266 results about "Vector quantization" patented technology

A method is provided for mapping the latent space of a Vector Quantized Variational AutoEncoder (VQ-VAE) to polynomial basis vectors. The method includes training a VQ-VAE model on a dataset to obtain a set of codebook vectors representing the latent space; defining a polynomial basis for the latent space, the polynomial basis containing terms up to a predetermined order; mapping each codebook vector to the polynomial basis by determining polynomial coefficients that represent each codebook vector in terms of the polynomial basis; and using the polynomial coefficients to reconstruct and manipulate latent space representations.

The invention relates to the technical field of cable aging diagnosis, in particular to a cable aging diagnosis monitoring method and system. The method comprises the following steps: collecting cable multi-physical field data; establishing a time synchronization mechanism according to the cable multi-physical field data to obtain a multi-physical field signal data set; extracting eigenentropy value characteristics of each physical field according to the multi-physical field signal data set; constructing an entropy feature vector based on the eigenentropy feature of each physical field; defining a cable initial state reference entropy according to the entropy feature vector; and based on pre-acquired historical data and the entropy value feature vector, quantifying the aging catalytic effect under the synergistic effect of the temperature field and the partial discharge field, identifying the microcrack propagation effect generated by the combined effect of the mechanical strain and the temperature field, and constructing an entropy value synergistic amplification coefficient. According to the method, the cooperative amplification coefficient is established based on the entropy theory, and the influence of the multi-physical field interaction on the cable aging process is accurately quantified.

The invention relates to integrated airborne communication antenna equipment and a communication method. The method comprises the following steps: carrying out CSI data acquisition and space-time focus modulation on a communication channel to obtain CSI space-time characteristic data; performing residual vector quantization operation to generate a quantized data stream, and performing recursive coding processing to obtain a target data stream; mapping the target data stream to a three-dimensional tensor space, constructing a real value tensor model, and performing iterative operation to obtain a CSI feature vector; performing fractional order wavelet transform decomposition on the CSI feature vector to obtain a radio frequency feature set, and inputting the radio frequency feature set into a federal fractional scattering network for feature processing to obtain an identity authentication result; and calculating an amplitude-phase weight coefficient of the antenna array, generating a beam parameter, and performing compensation matrix operation on the beam parameter according to the CSI feature vector to obtain a pre-coded signal. According to the invention, the dynamic adjustment of the antenna beam parameters is realized, and the accuracy and anti-interference capability of beam forming are improved.

The invention discloses a panoramic video frame insertion method based on a potential diffusion model. The method comprises the following steps: compressing an input image to a potential space by using a panoramic perception vector quantization variational auto-encoder to obtain potential features; constructing initial noise, and fusing the motion features extracted by the panoramic optical flow adapter to obtain condition information; performing iterative denoising operation on the potential features of the intermediate frame to obtain final potential features of the intermediate frame; and restoring the final potential features of the intermediate frame into a frame insertion image of a pixel space through a condition decoder. According to the panoramic video frame interpolation method provided by the invention, the potential diffusion model and the panoramic characteristic enhancement technology are combined, so that the perception quality of a frame interpolation result can be remarkably improved, details and complex textures of a pole region can be better reserved, and a high-quality time interpolation solution is provided for immersive panoramic video application.

The invention discloses a method and a system for generating 3D (three-dimensional) human motion under text driving by utilizing a 2D (two-dimensional) video. The method comprises the following steps of: acquiring the video and preprocessing to obtain a two-dimensional key point sequence and text description; the two-dimensional key point sequence passes through a spatiotemporal feature adapter to obtain a potential spatiotemporal feature sequence, a residual vector quantizer quantizes and outputs a three-dimensional SMP L parameter sequence, and meanwhile, potential spatiotemporal features and a discrete Token sequence are mapped; preprocessing a text to extract a semantic vector, partially covering a Token sequence of a basic quantization layer, reconstructing a prediction sequence through a predictor in combination with the semantic vector, and obtaining a complete sequence through a refiner; constructing a total loss function and a text-to-action loss function to train the module; and inputting the text description and the basic quantization layer Token to a trained module, outputting a three-dimensional SMPL parameter sequence, and rendering to generate a three-dimensional human body grid and animation. According to the method, the end-to-end generation from the text to the three-dimensional SMPL action is realized only by two-dimensional key points and text description.

The invention relates to the crossing field of natural language processing (NLP) and information retrieval (IR), in particular to an intelligent retrieval and question answering method and device, which is based on time window optimization and comprises the following steps: S1, knowledge preprocessing and time entity identification; s2, text vector quantization; s3, storing vectors, and retaining time information; s4, proposing a problem, and identifying a time entity; s5, user question vector quantization; s6, performing vector retrieval, and filtering time information; s7, combining time attenuation and similarity score; s8, merging the prompt statements; s9, returning an answer; and S10, answering the question. Compared with the prior art, the method has the advantages that each knowledge item in the knowledge base can be endowed with a clear time label, the question containing the time information can be answered more accurately and efficiently, and more timely and accurate answers can be provided.

The invention discloses a layered energy management method of a fuel cell hybrid tramcar with self-adaptive working condition perception. In the recognition layer, a sliding window mechanism is adopted to extract time domain and frequency domain features of load conditions, feature data are clustered based on a spectral clustering algorithm driven by a deep auto-encoder, a data set with category labels is obtained, and a deep dynamic learning vector quantization neural network classifier is trained; in the strategy layer, a double-delay depth deterministic strategy gradient reinforcement learning algorithm is adopted, a reward function is constructed, and lithium battery SOC fluctuation penalty term limit parameters in the reward function are adaptively adjusted according to the real-time load working condition category output by the recognition layer; training the reinforcement learning agent to obtain an optimal power distribution scheme between the multi-stack fuel cell power generation system and the lithium battery; and according to the performance degradation degrees of different fuel cell stacks, a distributed cooperative control strategy considering performance difference is adopted to distribute the output power of each stack, so that the coordinated control of the running state of the multi-stack fuel cell power generation system is realized.

The embodiment of the invention discloses a discrete voice coding method, electronic equipment and a storage medium, and the method comprises the steps: constructing a voice input and frequency spectrum output VAE, enabling the input voice to have speaker tone disturbance, and enabling the VAE structure to be used for multi-task learning; vector quantization is carried out on the VAE, and a VQ-VAE is initialized, so that a quantization layer is added to continue to reduce an information bottleneck; and additionally training a vocoder, and carrying out waveform reconstruction by starting from the code obtained in the VQ-VAE.

The invention belongs to the technical field of computer graphics and real-time 3D scene reconstruction, and relates to a real-time compression and optimization method based on a 3D Gaussian sputtering (3DGS) model, which realizes efficient storage, transmission and rendering of a three-dimensional scene through residual vector quantization (RVQ) and a dynamic pruning technology, and comprises the following steps: S1, residual quantization vector quantization; s2, enhancing a dynamic pruning strategy; and S3, index compression and coding optimization. The invention provides a real-time 3D Gaussian sputtering (3D Gaussian sputtering) model compression system combining residual vector quantization and dynamic pruning, which can realize high-quality real-time rendering and extremely high storage efficiency. Through three core technologies of a vector quantization strategy, dynamic pruning strategy enhancement and index compression and coding optimization, on the premise that the rendering quality is guaranteed, the purposes that the model storage amount is reduced by more than 45 times, and the rendering speed is increased by 3 times are achieved.

A method for performing a computing task includes: extracting one or more features from a user content; converting the features to a floating point query vector; quantizing the floating point query vector; obtaining a database vector including one or more floating point feature vectors; determining a compensation vector based on a data distribution of the floating point query vector; quantizing the floating point feature vectors; determining an error function based on a difference between data distributions of i) the quantized query vector compensated with the compensation vector, and ii) the floating point query vector; determining, based on the error function, values of the compensation vector corresponding to the quantized feature vectors; combining the quantized query vectors and the values of the compensation vector to obtain one or more compensated query vectors; and performing the computing task using the compensated query vectors and the quantized feature vectors to obtain an output.

The invention discloses a multi-scale feature fusion-based channel information feedback method for a large-scale MIMO (Multiple Input Multiple Output) system, which comprises the following steps of: 1, constructing a multi-scale feature fusion-based encoder and decoder network, and combining a novel selective state space with a multi-head attention model and hybrid convolution, the channel state information (CSI) remote modeling capability is further enhanced through an input selectivity mechanism, and multi-scale channel features are extracted for CSI (de-compression). 2, constructing a convolution enhanced vector quantization model, and further improving the feedback precision under the same feedback overhead by using dimension reduction convolution; and 3, a channel feature enhancement (CFE) module is constructed, and channel information loss is further reduced by utilizing ascending channel convolution to assist in decompressing full connection. According to the codec, multi-scale CSI features are extracted by using an input selectivity mechanism, multi-head attention and convolution, and high-precision channel information feedback is completed by combining a convolution enhanced vector quantization model and a CFE module.

The invention discloses a text-driven digital human action generation method, which comprises the following steps of: decoupling each body part of a human body when a training data set is constructed, generating a corresponding text description unit for each body part, and constructing different texts for describing the same action by randomly selecting the text description units. The influence of local action combination bias existing in a previous data set is relieved, and the model is helped to better associate a text with a corresponding local body action; an action encoder and an action decoder in the human body action vector quantization auto-encoder both adopt a graph convolutional network structure, so that the human body skeleton connection relationship is more effectively represented. The human body motion data actually should be of a topological graph structure instead of grid data, so that compared with a convolutional neural network, the graph convolutional neural network can more effectively utilize the topological structure of the human body and extract the internal relation between joints. The modal transformation model benefits from a loss function employing local semantic enhancement during training.

The invention relates to a video generation method, which comprises the following steps: S1, acquiring a condition prompt of a video to be generated, and reconstructing a content prompt of the video through an alignment model to obtain a condition content prompt covering a scene, an action and a camera motion, the content of the condition prompt comprising a text, an audio, a picture or a video; s2, encoding the content prompt by using a multi-modal encoder to obtain multi-modal vector embedding; s3, calling a visual automatic encoder subjected to space-time residual quantization processing, performing off-line vector quantization on the training set video, and obtaining a video token sequence and an encoding mode thereof; s4, inputting the video token sequence into a Mate module, and based on a short-range-long-range decoupling space-time attention and sliding time window strategy, outputting a representation which is in a linear relationship with the number of video tokens; and S5, performing cross attention fusion on the multi-modal vector embedding and Mate module output, sampling step by step to obtain a target video token sequence, and restoring the target video token sequence into a complete video through a decoder.

The invention relates to the technical field of wireless semantic communication, and discloses a semantic information transmission method, device and equipment and a storage medium, a sending end extracts features of to-be-transmitted data, and matches the features with vectors in a preset discrete vector codebook to obtain discrete feature vectors; and determining an orthogonal frequency division multiplexing time-frequency resource distribution grid according to the feature importance weights of the discrete feature vector and the initial feature vector, generating a sending signal, and transmitting the sending signal to a receiving end. And the receiving end inputs the semantic feature vector obtained after processing the received signal into the vector quantization variational self-decoder, and the semantic feature vector is matched with the vector in the discrete vector codebook again to correct the transmission error, thereby obtaining the reconstructed data to be transmitted. Discretization processing of the features is achieved through a discrete vector codebook, important semantic feature bits are quantized according to the importance weight of each feature and then distributed to orthogonal frequency division multiplexing for transmission, so that key semantic information is protected, and anti-interference transmission of the semantic information is achieved.

The invention relates to a pedestrian flow modeling and real-time simulation method, system and device and a medium, and the method comprises the following steps: carrying out the probability analysis of the moving direction of pedestrians in a railway platform environment based on a cellular automaton theory, and carrying out the modeling; based on the probability model of the pedestrian moving direction, according to the pedestrian decision-making process, modeling the pedestrian decision-making process by adopting a modeling method based on a multi-agent system; and simulating the pedestrian decision model by using an iterative vector quantization Q-Learning algorithm to obtain the overall distribution condition of the pedestrian flow for the next intelligent analysis of the digital twin system. The method can be widely applied to the computer and automation field and the railway video monitoring field.

A method and apparatus for classifying generated speech are disclosed. The method for classifying generated speech includes: applying a one-dimensional convolution operation to raw speech data to embed the raw speech data into a feature space and extract a feature vector; quantizing the feature vector by applying it to a residual vector quantizer; and applying the quantized result to a classifier model including a natural language processing model to output a classification label.

A method for training a model for generating a representation of long-term motion from a text input comprises: training a motion encoder of an autoencoder to compress and map an input motion into a latent representation comprising a sequence of latent vectors in a discrete latent space, each latent vector representing a fixed length of motion; training a quantization module to quantize the latent vectors to a sequence of quantized latent vectors in quantized latent space; and training a motion decoder to reconstruct the quantized sequence as a sequence of single-frame pose representations. A text encoder is trained to predict a latent sequence conditioned on a text input and a duration using the mapped latent representation as a target.

The invention belongs to the technical field of signal processing, is suitable for the medical field and the financial field, and discloses an audio signal encoding and decoding method and device based on a neural network, equipment and a medium. The method comprises the following steps: respectively inputting an initial amplitude spectrum and an initial phase spectrum corresponding to an audio signal to be processed into an amplitude encoder and a phase encoder for parallel feature coding; inputting a target coding vector of the audio signal to be processed into a residual vector quantization module for quantization processing to obtain an amplitude implicit vector and a phase implicit vector of the audio signal to be processed, and respectively inputting the amplitude implicit vector and the phase implicit vector into an amplitude decoder and a phase decoder for parallel decoding, obtaining an initial amplitude spectrum and an initial phase spectrum; and performing inverse short-time Fourier transform processing on the initial amplitude spectrum and the initial phase spectrum of the audio signal to be processed to generate a reconstructed audio signal. According to the invention, the tone quality and efficiency of the audio codec at a low bit rate are effectively improved.

The invention relates to an audio processing method, device and system and a storage medium, relates to the technical field of data processing, and aims to at least solve the problem of serious distortion of output audio in related technologies. The method comprises the following steps: inputting a to-be-processed audio into an audio conversion model, so as to adopt the audio conversion model to execute the following operations: calling an encoder to encode the to-be-processed audio into a target continuous encoding signal; quantizing the target continuous coded signal vector into a target discrete coded signal based on a preset codebook; calling a decoder to decode the target discrete coded signal into a target audio composed of continuous target output coded signals based on a preset codebook; wherein the preset codebook comprises an association mapping relation between a continuous coding signal and a discrete coding signal, and the audio conversion model is trained by taking reconstruction loss, codebook loss and retention loss of coding and decoding of a coder and a decoder as constraint targets; and outputting a target audio obtained after the operation of the audio conversion model is completed.

The invention provides a voice reconstruction method and system based on entropy coding residual quantization and frequency spectrum restoration, and relates to the technical field of artificial intelligence voice signal processing, and the method comprises the steps: obtaining an original voice waveform, inputting the voice waveform into a neural voice coding and decoding model, firstly entering a coder to map the input voice waveform into acoustic potential representation, and then entering a frequency spectrum restoration model; performing residual quantization on the acoustic potential characterization layer by layer through a residual vector quantization module, introducing a gating-based dynamic layer number selection mechanism and entropy regularization constraint, enabling bits to be adaptively distributed among different voice segments, reconstructing reconstructed acoustic features of the potential characterization, inputting the reconstructed acoustic features into a decoder, restoring the reconstructed acoustic features into a time domain waveform, and outputting the time domain waveform. And mapping to a logarithmic magnitude spectrum domain through a spectrum repairing module, predicting a residual error in the logarithmic magnitude spectrum domain and performing confidence gating fusion to obtain a complex spectrum, and outputting after time domain synthesis to obtain reconstructed speech. According to the invention, high fidelity, intelligibility and transmission reliability of the voice can be considered at an extremely low bit rate.

The invention discloses a power data cross-domain dynamic trust evaluation and access control method, and belongs to the field of power data security management and control. The method comprises the following steps: S1, dynamic trust evaluation: acquiring context information of a user and equipment in real time, generating a trust value and a risk value by using a random forest algorithm, and dynamically calculating a trust degree; s2, adapting to a sequence recommendation algorithm of a large language model based on a product quantization technology: generating an access authorization strategy through behavior sequence embedding generation and vector quantization in combination with credibility; s3, fine-grained dynamic access control: dynamically adjusting permission distribution according to the sensitivity of the power data and a real-time trust evaluation result; and S4, user abnormal behavior identification: monitoring user behaviors in real time based on a TextCNN model, identifying abnormity and triggering a safety protection mechanism. According to the method, the safety and reliability of cross-domain access of the power data can be effectively improved, data abuse is effectively prevented, and the method has wide industrial application value.

The invention discloses a high-fidelity three-dimensional Gaussian sputtering (3DGS) lightweight method for resource-constrained equipment. The method aims at solving the problems of high storage and computing resource consumption caused by the fact that a large number of parameters are stored in an existing 3DGS technology, and geometric distortion possibly occurring when details of a scene center are processed is overcome. The core of the method lies in a multi-stage progressive optimization framework, and the framework cooperatively applies four key technologies of Gaussian cutting and opacity regularization, dynamic spherical harmonic function adjustment, entropy constraint vector quantization and coordinate space shrinkage. Wherein in Gaussian clipping, redundant gauss are eliminated by quantifying the contribution degree of a Gaussian unit; the dynamic spherical harmonic function adjustment adaptively adjusts the order of color coding according to the scene complexity; the entropy constraint vector quantization is used for compressing a plurality of Gaussian attributes so as to realize more compact representation; and the coordinate space shrinkage is realized through nonlinear transformation, so that the rendering precision of details of the center of the scene is remarkably improved. According to the method, while the rendering precision and quality are kept, remarkable storage compression is realized, and the method is particularly suitable for deployment of resource-limited platforms such as mobile equipment.

The invention discloses a semantic-driven intelligent agent capability discovery method and device oriented to the intelligent agent Internet, and the method comprises the steps: firstly generating an intelligent agent structured portrait which covers the three-dimensional information of skills, roles and states; semantic coding is carried out on the portrait, and the portrait is converted into a high-dimensional semantic vector; then, the vector is partitioned, a codebook is generated in each subspace in a clustering mode, and the codebook comprises representative vectors and serial numbers of the representative vectors; and quantizing the sub-vectors into numbers based on a codebook, and splicing the numbers to form discrete identification codes of the agent index. And when the index is updated, incremental maintenance is executed according to the distance between the new agent sub-vector and the vector in the codebook. Meanwhile, a generative retrieval model is trained, task query is directly mapped into discrete identification codes, historical and new task samples are mixed for continuous learning in training, and stability constraints are introduced. According to the method, through semantic portraits, a quantitative index mechanism and memory enhancement continuous learning, an end-to-end retrieval and rapid updating capability discovery scheme is constructed.

The invention provides a global structure sensing vector quantization method based on optimal transmission, which comprises the following steps of: mapping input data to a continuous hidden variable space through an encoder to generate continuous representation; constructing a distance matrix between the continuous representation and the codebook, and solving a distribution matrix by using an optimal transmission problem; carrying out normalization processing on the distance matrix, and generating an initial value of a distribution matrix by adopting a special initialization method; performing row and column normalization iteration on the distribution matrix through a Sinkhorn-Knopp algorithm until the distribution matrix is converged; and after iteration is finished, quantitative characteristics are determined based on the maximum value of the distribution matrix, the quantitative characteristics are mapped back to an original data space through a decoder, and a reconstructed data result is obtained. Based on the scheme provided by the invention, a codebook utilization rate close to 100% and an excellent data reconstruction result can be ensured, and a powerful tool is provided for discrete compression scene reconstruction of continuous data such as images and videos.

The invention discloses a medical image generation method based on a diffusion model, which introduces a novel method based on a potential diffusion model LDM, and uses a tumor mask and a CT image as condition input to generate a high-quality PET image containing a tumor focus. According to the method, an image is coded by using a vector quantization variational auto-encoder VQVAE, denoising is performed in a hidden space by using a diffusion model, and a generation process is guided by using a conditional encoder. The real tumor with controllable shape, size and position can be generated, and the structure and metabolic activity of the real tumor can be simulated. A comparison experiment with a real PET image shows that the generated image has high fidelity, and the potential of the method for enhancing the performance of the AI algorithm in a data limited environment is highlighted.

Systems, apparatuses and methods may provide for technology that conducts, in accordance with a first instruction, a load of a block of data into a register, wherein the block of data is to include a plurality of lanes, conducts, in accordance with a second instruction, a first bitwise mask application to each lane in the plurality of lanes, and extracts a set of vector dimensions from the block of data based on the first bitwise mask application.

The invention provides a voice generation model construction method and device, electronic equipment and a readable medium, and the method comprises the steps: inputting a preset training voice into a preset vector quantizer, and obtaining a training semantic discrete feature of the training voice; the training semantic discrete features comprise the language style of the training voice; obtaining a training text corresponding to the training voice, and training a preset autoregression voice model by using the training text and the training semantic discrete features to obtain a semantic discrete feature generation model; obtaining a training Mel spectrogram corresponding to the training semantic discrete features; training a preset optimal transmission condition flow matching model by using the training semantic discrete features and the training Mel spectrogram to obtain a Mel spectrogram generation model; and constructing a voice generation model based on the Mel spectrogram generation model and the semantic discrete feature generation model. According to the invention, multi-language-style voice or dialect voice generation is realized through a single voice generation model, the annotation cost is reduced, and the mechanical feeling generated in audio generation is reduced.

A multi-modal human body action generation method based on a large language model comprises the steps that structural features are extracted based on 3D action data, body part-level atomic semantic description is generated in combination with the large language model, and a multi-modal alignment data set containing texts, voices and music is constructed; whole-body actions are decoupled according to parts, an independent vector quantization encoder is adopted to perform residual quantization, and an atomic action token strongly associated with a fine-grained text is generated; splicing the text description and the action token into a mixed action sentence containing a special mark according to a body structure; multi-modal input joint modeling is realized through a large language model, a fine-grained text and an action token sequence are synchronously generated, 3D actions conforming to semantics are output through decoding, and zero sample generation and part-level accurate control are supported. According to the method, the limitation of coarse-grained alignment of a traditional method is broken through, and the semantic consistency, multi-modal adaptability and local controllability of action generation are remarkably improved through fine-grained semantic mapping, decoupling action coding and unified sequence modeling.

A method of operating a wireless communication device includes calculating a first channel matrix for a downlink channel based on a CSI-RS, determining a stage level of a multi-stage vector quantization process based on an uplink channel, extracting, using a first machine learning model, a latent vector based on the first channel matrix, selecting a first codeword from a first codebook corresponding to a first stage, and generating, in the first stage, a first residual latent vector based on the latent vector, selecting, in a second stage corresponding to the determined stage level, a second codeword from a second codebook corresponding to the second stage, and generating, in the second stage, a second residual latent vector based on the first residual latent vector, wherein the second residual latent vector is based on a second codeword, and generating a bitstream based on the first codeword and the second codeword.

The invention relates to a video definition improving method and system, and specifically, the method comprises the steps: analyzing a video to obtain a frame image in the video, obtaining the gradient intensity and the gradient direction of each pixel point in the frame image, and carrying out the normalization of the gradient intensity and the gradient direction; the method comprises the following steps: encoding a frame image through an encoder to obtain a feature map, obtaining an effective receptive field of a feature vector on the feature map in the frame image, determining gradient intensity and gradient direction consistency corresponding to the feature vector according to the effective receptive field, weighting the feature vector by using the gradient intensity corresponding to the feature vector to obtain a weighted feature map, and obtaining a weighted image; quantizing the feature vector by using gradient intensity and gradient direction consistency corresponding to the feature vector to obtain a feature map quantization result; and decoding the quantized result of the feature map by using a decoder to obtain a high-resolution frame image, and synthesizing the high-resolution images of all the frame images in the video into the video.