235 results about "Vector quantization" patented technology

A method is provided for mapping the latent space of a Vector Quantized Variational AutoEncoder (VQ-VAE) to polynomial basis vectors. The method includes training a VQ-VAE model on a dataset to obtain a set of codebook vectors representing the latent space; defining a polynomial basis for the latent space, the polynomial basis containing terms up to a predetermined order; mapping each codebook vector to the polynomial basis by determining polynomial coefficients that represent each codebook vector in terms of the polynomial basis; and using the polynomial coefficients to reconstruct and manipulate latent space representations.

The invention relates to the technical field of cable aging diagnosis, in particular to a cable aging diagnosis monitoring method and system. The method comprises the following steps: collecting cable multi-physical field data; establishing a time synchronization mechanism according to the cable multi-physical field data to obtain a multi-physical field signal data set; extracting eigenentropy value characteristics of each physical field according to the multi-physical field signal data set; constructing an entropy feature vector based on the eigenentropy feature of each physical field; defining a cable initial state reference entropy according to the entropy feature vector; and based on pre-acquired historical data and the entropy value feature vector, quantifying the aging catalytic effect under the synergistic effect of the temperature field and the partial discharge field, identifying the microcrack propagation effect generated by the combined effect of the mechanical strain and the temperature field, and constructing an entropy value synergistic amplification coefficient. According to the method, the cooperative amplification coefficient is established based on the entropy theory, and the influence of the multi-physical field interaction on the cable aging process is accurately quantified.

The invention discloses a panoramic video frame insertion method based on a potential diffusion model. The method comprises the following steps: compressing an input image to a potential space by using a panoramic perception vector quantization variational auto-encoder to obtain potential features; constructing initial noise, and fusing the motion features extracted by the panoramic optical flow adapter to obtain condition information; performing iterative denoising operation on the potential features of the intermediate frame to obtain final potential features of the intermediate frame; and restoring the final potential features of the intermediate frame into a frame insertion image of a pixel space through a condition decoder. According to the panoramic video frame interpolation method provided by the invention, the potential diffusion model and the panoramic characteristic enhancement technology are combined, so that the perception quality of a frame interpolation result can be remarkably improved, details and complex textures of a pole region can be better reserved, and a high-quality time interpolation solution is provided for immersive panoramic video application.

The invention discloses a method and a system for generating 3D (three-dimensional) human motion under text driving by utilizing a 2D (two-dimensional) video. The method comprises the following steps of: acquiring the video and preprocessing to obtain a two-dimensional key point sequence and text description; the two-dimensional key point sequence passes through a spatiotemporal feature adapter to obtain a potential spatiotemporal feature sequence, a residual vector quantizer quantizes and outputs a three-dimensional SMP L parameter sequence, and meanwhile, potential spatiotemporal features and a discrete Token sequence are mapped; preprocessing a text to extract a semantic vector, partially covering a Token sequence of a basic quantization layer, reconstructing a prediction sequence through a predictor in combination with the semantic vector, and obtaining a complete sequence through a refiner; constructing a total loss function and a text-to-action loss function to train the module; and inputting the text description and the basic quantization layer Token to a trained module, outputting a three-dimensional SMPL parameter sequence, and rendering to generate a three-dimensional human body grid and animation. According to the method, the end-to-end generation from the text to the three-dimensional SMPL action is realized only by two-dimensional key points and text description.

The invention discloses a layered energy management method of a fuel cell hybrid tramcar with self-adaptive working condition perception. In the recognition layer, a sliding window mechanism is adopted to extract time domain and frequency domain features of load conditions, feature data are clustered based on a spectral clustering algorithm driven by a deep auto-encoder, a data set with category labels is obtained, and a deep dynamic learning vector quantization neural network classifier is trained; in the strategy layer, a double-delay depth deterministic strategy gradient reinforcement learning algorithm is adopted, a reward function is constructed, and lithium battery SOC fluctuation penalty term limit parameters in the reward function are adaptively adjusted according to the real-time load working condition category output by the recognition layer; training the reinforcement learning agent to obtain an optimal power distribution scheme between the multi-stack fuel cell power generation system and the lithium battery; and according to the performance degradation degrees of different fuel cell stacks, a distributed cooperative control strategy considering performance difference is adopted to distribute the output power of each stack, so that the coordinated control of the running state of the multi-stack fuel cell power generation system is realized.

The invention belongs to the technical field of computer graphics and real-time 3D scene reconstruction, and relates to a real-time compression and optimization method based on a 3D Gaussian sputtering (3DGS) model, which realizes efficient storage, transmission and rendering of a three-dimensional scene through residual vector quantization (RVQ) and a dynamic pruning technology, and comprises the following steps: S1, residual quantization vector quantization; s2, enhancing a dynamic pruning strategy; and S3, index compression and coding optimization. The invention provides a real-time 3D Gaussian sputtering (3D Gaussian sputtering) model compression system combining residual vector quantization and dynamic pruning, which can realize high-quality real-time rendering and extremely high storage efficiency. Through three core technologies of a vector quantization strategy, dynamic pruning strategy enhancement and index compression and coding optimization, on the premise that the rendering quality is guaranteed, the purposes that the model storage amount is reduced by more than 45 times, and the rendering speed is increased by 3 times are achieved.

A method for performing a computing task includes: extracting one or more features from a user content; converting the features to a floating point query vector; quantizing the floating point query vector; obtaining a database vector including one or more floating point feature vectors; determining a compensation vector based on a data distribution of the floating point query vector; quantizing the floating point feature vectors; determining an error function based on a difference between data distributions of i) the quantized query vector compensated with the compensation vector, and ii) the floating point query vector; determining, based on the error function, values of the compensation vector corresponding to the quantized feature vectors; combining the quantized query vectors and the values of the compensation vector to obtain one or more compensated query vectors; and performing the computing task using the compensated query vectors and the quantized feature vectors to obtain an output.

The invention relates to a video generation method, which comprises the following steps: S1, acquiring a condition prompt of a video to be generated, and reconstructing a content prompt of the video through an alignment model to obtain a condition content prompt covering a scene, an action and a camera motion, the content of the condition prompt comprising a text, an audio, a picture or a video; s2, encoding the content prompt by using a multi-modal encoder to obtain multi-modal vector embedding; s3, calling a visual automatic encoder subjected to space-time residual quantization processing, performing off-line vector quantization on the training set video, and obtaining a video token sequence and an encoding mode thereof; s4, inputting the video token sequence into a Mate module, and based on a short-range-long-range decoupling space-time attention and sliding time window strategy, outputting a representation which is in a linear relationship with the number of video tokens; and S5, performing cross attention fusion on the multi-modal vector embedding and Mate module output, sampling step by step to obtain a target video token sequence, and restoring the target video token sequence into a complete video through a decoder.

The invention relates to the technical field of wireless semantic communication, and discloses a semantic information transmission method, device and equipment and a storage medium, a sending end extracts features of to-be-transmitted data, and matches the features with vectors in a preset discrete vector codebook to obtain discrete feature vectors; and determining an orthogonal frequency division multiplexing time-frequency resource distribution grid according to the feature importance weights of the discrete feature vector and the initial feature vector, generating a sending signal, and transmitting the sending signal to a receiving end. And the receiving end inputs the semantic feature vector obtained after processing the received signal into the vector quantization variational self-decoder, and the semantic feature vector is matched with the vector in the discrete vector codebook again to correct the transmission error, thereby obtaining the reconstructed data to be transmitted. Discretization processing of the features is achieved through a discrete vector codebook, important semantic feature bits are quantized according to the importance weight of each feature and then distributed to orthogonal frequency division multiplexing for transmission, so that key semantic information is protected, and anti-interference transmission of the semantic information is achieved.

A method and apparatus for classifying generated speech are disclosed. The method for classifying generated speech includes: applying a one-dimensional convolution operation to raw speech data to embed the raw speech data into a feature space and extract a feature vector; quantizing the feature vector by applying it to a residual vector quantizer; and applying the quantized result to a classifier model including a natural language processing model to output a classification label.

A method for training a model for generating a representation of long-term motion from a text input comprises: training a motion encoder of an autoencoder to compress and map an input motion into a latent representation comprising a sequence of latent vectors in a discrete latent space, each latent vector representing a fixed length of motion; training a quantization module to quantize the latent vectors to a sequence of quantized latent vectors in quantized latent space; and training a motion decoder to reconstruct the quantized sequence as a sequence of single-frame pose representations. A text encoder is trained to predict a latent sequence conditioned on a text input and a duration using the mapped latent representation as a target.

The invention belongs to the technical field of signal processing, is suitable for the medical field and the financial field, and discloses an audio signal encoding and decoding method and device based on a neural network, equipment and a medium. The method comprises the following steps: respectively inputting an initial amplitude spectrum and an initial phase spectrum corresponding to an audio signal to be processed into an amplitude encoder and a phase encoder for parallel feature coding; inputting a target coding vector of the audio signal to be processed into a residual vector quantization module for quantization processing to obtain an amplitude implicit vector and a phase implicit vector of the audio signal to be processed, and respectively inputting the amplitude implicit vector and the phase implicit vector into an amplitude decoder and a phase decoder for parallel decoding, obtaining an initial amplitude spectrum and an initial phase spectrum; and performing inverse short-time Fourier transform processing on the initial amplitude spectrum and the initial phase spectrum of the audio signal to be processed to generate a reconstructed audio signal. According to the invention, the tone quality and efficiency of the audio codec at a low bit rate are effectively improved.

The invention provides a voice reconstruction method and system based on entropy coding residual quantization and frequency spectrum restoration, and relates to the technical field of artificial intelligence voice signal processing, and the method comprises the steps: obtaining an original voice waveform, inputting the voice waveform into a neural voice coding and decoding model, firstly entering a coder to map the input voice waveform into acoustic potential representation, and then entering a frequency spectrum restoration model; performing residual quantization on the acoustic potential characterization layer by layer through a residual vector quantization module, introducing a gating-based dynamic layer number selection mechanism and entropy regularization constraint, enabling bits to be adaptively distributed among different voice segments, reconstructing reconstructed acoustic features of the potential characterization, inputting the reconstructed acoustic features into a decoder, restoring the reconstructed acoustic features into a time domain waveform, and outputting the time domain waveform. And mapping to a logarithmic magnitude spectrum domain through a spectrum repairing module, predicting a residual error in the logarithmic magnitude spectrum domain and performing confidence gating fusion to obtain a complex spectrum, and outputting after time domain synthesis to obtain reconstructed speech. According to the invention, high fidelity, intelligibility and transmission reliability of the voice can be considered at an extremely low bit rate.

The invention discloses a power data cross-domain dynamic trust evaluation and access control method, and belongs to the field of power data security management and control. The method comprises the following steps: S1, dynamic trust evaluation: acquiring context information of a user and equipment in real time, generating a trust value and a risk value by using a random forest algorithm, and dynamically calculating a trust degree; s2, adapting to a sequence recommendation algorithm of a large language model based on a product quantization technology: generating an access authorization strategy through behavior sequence embedding generation and vector quantization in combination with credibility; s3, fine-grained dynamic access control: dynamically adjusting permission distribution according to the sensitivity of the power data and a real-time trust evaluation result; and S4, user abnormal behavior identification: monitoring user behaviors in real time based on a TextCNN model, identifying abnormity and triggering a safety protection mechanism. According to the method, the safety and reliability of cross-domain access of the power data can be effectively improved, data abuse is effectively prevented, and the method has wide industrial application value.

The invention discloses a high-fidelity three-dimensional Gaussian sputtering (3DGS) lightweight method for resource-constrained equipment. The method aims at solving the problems of high storage and computing resource consumption caused by the fact that a large number of parameters are stored in an existing 3DGS technology, and geometric distortion possibly occurring when details of a scene center are processed is overcome. The core of the method lies in a multi-stage progressive optimization framework, and the framework cooperatively applies four key technologies of Gaussian cutting and opacity regularization, dynamic spherical harmonic function adjustment, entropy constraint vector quantization and coordinate space shrinkage. Wherein in Gaussian clipping, redundant gauss are eliminated by quantifying the contribution degree of a Gaussian unit; the dynamic spherical harmonic function adjustment adaptively adjusts the order of color coding according to the scene complexity; the entropy constraint vector quantization is used for compressing a plurality of Gaussian attributes so as to realize more compact representation; and the coordinate space shrinkage is realized through nonlinear transformation, so that the rendering precision of details of the center of the scene is remarkably improved. According to the method, while the rendering precision and quality are kept, remarkable storage compression is realized, and the method is particularly suitable for deployment of resource-limited platforms such as mobile equipment.

The invention discloses a semantic-driven intelligent agent capability discovery method and device oriented to the intelligent agent Internet, and the method comprises the steps: firstly generating an intelligent agent structured portrait which covers the three-dimensional information of skills, roles and states; semantic coding is carried out on the portrait, and the portrait is converted into a high-dimensional semantic vector; then, the vector is partitioned, a codebook is generated in each subspace in a clustering mode, and the codebook comprises representative vectors and serial numbers of the representative vectors; and quantizing the sub-vectors into numbers based on a codebook, and splicing the numbers to form discrete identification codes of the agent index. And when the index is updated, incremental maintenance is executed according to the distance between the new agent sub-vector and the vector in the codebook. Meanwhile, a generative retrieval model is trained, task query is directly mapped into discrete identification codes, historical and new task samples are mixed for continuous learning in training, and stability constraints are introduced. According to the method, through semantic portraits, a quantitative index mechanism and memory enhancement continuous learning, an end-to-end retrieval and rapid updating capability discovery scheme is constructed.

The invention provides a global structure sensing vector quantization method based on optimal transmission, which comprises the following steps of: mapping input data to a continuous hidden variable space through an encoder to generate continuous representation; constructing a distance matrix between the continuous representation and the codebook, and solving a distribution matrix by using an optimal transmission problem; carrying out normalization processing on the distance matrix, and generating an initial value of a distribution matrix by adopting a special initialization method; performing row and column normalization iteration on the distribution matrix through a Sinkhorn-Knopp algorithm until the distribution matrix is converged; and after iteration is finished, quantitative characteristics are determined based on the maximum value of the distribution matrix, the quantitative characteristics are mapped back to an original data space through a decoder, and a reconstructed data result is obtained. Based on the scheme provided by the invention, a codebook utilization rate close to 100% and an excellent data reconstruction result can be ensured, and a powerful tool is provided for discrete compression scene reconstruction of continuous data such as images and videos.

The invention discloses a medical image generation method based on a diffusion model, which introduces a novel method based on a potential diffusion model LDM, and uses a tumor mask and a CT image as condition input to generate a high-quality PET image containing a tumor focus. According to the method, an image is coded by using a vector quantization variational auto-encoder VQVAE, denoising is performed in a hidden space by using a diffusion model, and a generation process is guided by using a conditional encoder. The real tumor with controllable shape, size and position can be generated, and the structure and metabolic activity of the real tumor can be simulated. A comparison experiment with a real PET image shows that the generated image has high fidelity, and the potential of the method for enhancing the performance of the AI algorithm in a data limited environment is highlighted.

The invention provides a voice generation model construction method and device, electronic equipment and a readable medium, and the method comprises the steps: inputting a preset training voice into a preset vector quantizer, and obtaining a training semantic discrete feature of the training voice; the training semantic discrete features comprise the language style of the training voice; obtaining a training text corresponding to the training voice, and training a preset autoregression voice model by using the training text and the training semantic discrete features to obtain a semantic discrete feature generation model; obtaining a training Mel spectrogram corresponding to the training semantic discrete features; training a preset optimal transmission condition flow matching model by using the training semantic discrete features and the training Mel spectrogram to obtain a Mel spectrogram generation model; and constructing a voice generation model based on the Mel spectrogram generation model and the semantic discrete feature generation model. According to the invention, multi-language-style voice or dialect voice generation is realized through a single voice generation model, the annotation cost is reduced, and the mechanical feeling generated in audio generation is reduced.

A multi-modal human body action generation method based on a large language model comprises the steps that structural features are extracted based on 3D action data, body part-level atomic semantic description is generated in combination with the large language model, and a multi-modal alignment data set containing texts, voices and music is constructed; whole-body actions are decoupled according to parts, an independent vector quantization encoder is adopted to perform residual quantization, and an atomic action token strongly associated with a fine-grained text is generated; splicing the text description and the action token into a mixed action sentence containing a special mark according to a body structure; multi-modal input joint modeling is realized through a large language model, a fine-grained text and an action token sequence are synchronously generated, 3D actions conforming to semantics are output through decoding, and zero sample generation and part-level accurate control are supported. According to the method, the limitation of coarse-grained alignment of a traditional method is broken through, and the semantic consistency, multi-modal adaptability and local controllability of action generation are remarkably improved through fine-grained semantic mapping, decoupling action coding and unified sequence modeling.

A method of operating a wireless communication device includes calculating a first channel matrix for a downlink channel based on a CSI-RS, determining a stage level of a multi-stage vector quantization process based on an uplink channel, extracting, using a first machine learning model, a latent vector based on the first channel matrix, selecting a first codeword from a first codebook corresponding to a first stage, and generating, in the first stage, a first residual latent vector based on the latent vector, selecting, in a second stage corresponding to the determined stage level, a second codeword from a second codebook corresponding to the second stage, and generating, in the second stage, a second residual latent vector based on the first residual latent vector, wherein the second residual latent vector is based on a second codeword, and generating a bitstream based on the first codeword and the second codeword.

The invention relates to a video definition improving method and system, and specifically, the method comprises the steps: analyzing a video to obtain a frame image in the video, obtaining the gradient intensity and the gradient direction of each pixel point in the frame image, and carrying out the normalization of the gradient intensity and the gradient direction; the method comprises the following steps: encoding a frame image through an encoder to obtain a feature map, obtaining an effective receptive field of a feature vector on the feature map in the frame image, determining gradient intensity and gradient direction consistency corresponding to the feature vector according to the effective receptive field, weighting the feature vector by using the gradient intensity corresponding to the feature vector to obtain a weighted feature map, and obtaining a weighted image; quantizing the feature vector by using gradient intensity and gradient direction consistency corresponding to the feature vector to obtain a feature map quantization result; and decoding the quantized result of the feature map by using a decoder to obtain a high-resolution frame image, and synthesizing the high-resolution images of all the frame images in the video into the video.

An image manipulation device including an interface configured to receive an image; an encoder configured to transform the received image into a vector Ws; a quantifier configured to output a source quantitative value Qs for the received image based on a training data set including a plurality of images arranged into classes, and receive a target quantitative value Qt different than the source quantitative value Qs; a navigator configured to receive the source quantitative value Qs, target quantitative value Qt and the vector Ws, and output a vector Wt indicating a modified vector of the vector Ws based on the source quantitative value Qs and target quantitative value Qt; a decoder configured to decode the output vector Wt into a modified image It corresponding to a modification of the received image; and a display configured to display the modified image.

The invention provides a semi-supervised signal modulation identification method based on multi-codebook discrete virtual adversarial training, and the method comprises the steps: carrying out the high-dimensional potential representation coding of input labeled and unlabeled sample signals through employing a vector quantization generative adversarial network, so as to obtain a continuous potential representation; mapping the continuous potential representation into a discrete vector sequence through a quantizer; virtual confrontation disturbance meeting the condition that the L2 norm does not exceed epsilon is applied to the unlabeled sample at the input end, and a disturbance sample is generated; respectively inputting a labeled sample, a label-free sample and a disturbance sample into a classifier, calculating cross entropy loss and KL divergence consistency loss, and jointly optimizing parameters of an encoder, a codebook and the classifier in a weighted sum form; therefore, the discrete potential representation is obtained through the vector quantization generative adversarial network, disturbance is constructed for the unlabeled sample at the input end, and joint optimization is performed in combination with the cross entropy and the KL divergence loss, so that the recognition accuracy and robustness under the conditions of few labels and low signal-to-noise ratio are improved.

Systems and techniques are described for quantizing parameters (e.g., post-training vectors) associated with a pre-trained model. For example, a device can obtain a codebook for a group of weights of a pre-trained machine learning model. The device can determine a compression ratio based on the codebook and at least one of a vector quantization dimensionality, a group size, a codebook bit-width, or a scale group size. The device can quantize, via a vector quantization engine, the group of weights of the pre-trained machine learning model a plurality of columns at a time according to the compression ratio to generate a quantized pre-trained model.

The embodiment of the invention provides a CSI (Channel State Information) feedback method based on Transform and entropy constraint vector quantization. The method is applied to the technical field of wireless communication. The method comprises the following steps: receiving an input angle-time delay domain channel matrix, and performing feature extraction on the channel matrix through a convolutional layer to obtain a first feature tensor; the first feature tensor is input into a trained Transform encoder for analysis processing, and a latent vector is obtained; performing quantization processing on the latent vector through the trained vector quantization variational auto-encoder to obtain discretization representation of the latent vector; performing dequantization processing on the discretized representation of the latent vector to obtain a dequantized latent vector; performing feature extraction on the dequantized latent vector through a convolutional layer to generate an initial channel feature; the initial channel characteristics are input into a trained Transform decoder for analysis processing, and final channel characteristics are obtained; and the final channel feature is mapped to the target resolution to obtain a reconstructed channel matrix, so that the reconstruction precision of the channel matrix is improved.

The invention discloses a zero-sample semantic segmentation map construction method combining a road visual language model and a dynamic object, and belongs to the technical field of zero-sample semantic segmentation map construction methods.According to the method, feature recovery is carried out on a degraded look-around image in severe weather through a vector quantized image restoration model, and the feature of the degraded look-around image is restored; therefore, the image features under the normal weather condition are recovered. Then, analyzing the restored image by using a road visual language model, generating semantic description, and extracting information of key elements on the road;

The invention relates to the technical field of speech synthesis, financial science and technology and medical health, and discloses a speech synthesis method, device and equipment of potential rhythm based on diffusion and a medium, and the method comprises the steps: generating a reference rhythm vector corresponding to each frequency band of an initial audio according to a Mel spectrogram and a preset real phoneme duration; according to the text hidden representation and the speaker hidden representation, constructing an analysis phoneme duration corresponding to the reference rhythm vector; constructing an analysis rhythm vector by using the text hidden representation, the speaker hidden representation and the analysis phoneme duration; generating a potential rhythm vector according to the reference rhythm vector and the analysis rhythm vector; and performing speech synthesis by using the text hidden representation, the speaker hidden representation and the potential rhythm vector to obtain a synthesized speech. The time step number is remarkably reduced through model analysis, the generation speed is increased, and meanwhile, the smoothness of the generated voice is remarkably improved through rhythm vector quantization.

The invention provides a voice synthesis method and device fusing visual information, electronic equipment and a medium. The method comprises the steps of extracting text features of target text information based on a text encoder; performing feature extraction processing, vector quantization coding processing and cross attention processing on the target video information to determine video features in the target video information; performing cross attention processing on the text features and the video features to determine joint features; and inputting the joint features into a pre-trained voice generation model, carrying out modeling processing on the joint features to generate video text features, then carrying out random rhythm prediction processing, feature expansion processing and decoding processing on the video text features, and generating synthetic voice corresponding to the target text information. Through fusing the information in the video, the voice with more expressive force is generated. According to the method, the defects in the aspect of emotion control in the prior art are overcome, and high-quality speech synthesis can be realized by using a small amount of sample data.

The invention discloses a hidden space adversarial sample generation method and system based on multi-scale feature separation, and the method comprises the steps: employing a neural network quantization training method based on straight-through estimation, and training a hierarchical vector quantization variational auto-encoder; carrying out differentiable Haar wavelet transformation on the input image by adopting a wavelet packet transformation algorithm, decomposing the input image into a low-frequency component and a high-frequency component, and realizing multi-scale feature separation; inputting the high-frequency component into a hierarchical vector quantization variational auto-encoder, and extracting and quantizing global high-frequency features and local high-frequency detail features; in the potential space, a learnable disturbance variable is introduced, a potential vector after disturbance is constructed, and the potential vector is reconstructed into an adversarial sample through a decoder; and based on a preset disturbance target, carrying out iterative optimization on the disturbance vector until a confrontation sample which satisfies an attack success condition and is optimized in visual quality is generated. According to the method, a wavelet domain variational auto-encoder and a hidden space iterative attack algorithm are fused, and an adversarial sample with high fidelity and clear interpretation is generated.