85 results about "Vector quantisation" patented technology

The present disclosure describes techniques for generating latent representations of images using a machine learning model. An image is split and flattened into a series of patches. The series of patches is concatenated with a sequence of latent tokens. The concatenated patches and latent tokens are input into an encoder of the machine learning model. A one-dimensional (1D) latent representation of the image is generated by the encoder. Vector quantization is performed on the 1D latent representation of the image by a vector quantizer of the machine learning model to generate quantized latent tokens. The image is reconstructed based on the quantized latent tokens by a decoder of the machine learning model.

The embodiment of the invention provides a self-supervised emotion recognition method based on a heart and brain combined codebook and related equipment, and belongs to the technical field of physiological signal processing and artificial intelligence. The method comprises the following steps: respectively defining heart beats of electrocardiosignals and electroencephalogram signal segments with equal lengths as words, and constructing sentences; a shared heart and brain joint codebook is created and trained, and electrocardio and electroencephalogram words are mapped to a unified discrete semantic space through vector quantization so as to learn cross-subject general characterization; then, discretizing a signal sentence by using the codebook, and combining space and position embedding and inputting a Transform encoder to carry out mask pre-training so as to learn context semantics of the signal; and finally, finely tuning the pre-training model for an emotion recognition task. According to the method, deep semantic fusion of heart and brain signals is realized through signal structuring and codebook sharing, dependence on labeled data is effectively overcome, and emotion recognition accuracy and cross-subject generalization ability are remarkably improved.

The invention discloses a multi-modal time sequence fusion voice-driven gesture generation method, which comprises the following steps of: firstly, learning compact discrete representation of gesture motion through a vector quantization variational auto-encoder model, and constructing a quantized potential space for a subsequent generation task; extracting audio features of the voice audio through an audio encoder; performing low-dimensional embedding on the identity of the speaker to obtain an identity feature; performing time sequence alignment on the audio features and the historical gesture sequence and then splicing the audio features and the historical gesture sequence along feature dimensions to form multi-modal initial representation; deep feature fusion is carried out through a multi-modal time sequence fusion module integrating a self-attention mechanism, a cross attention mechanism taking identity features as conditions and a Mamba module; and finally, reconstructing a target gesture sequence through a pre-trained decoder. According to the method, the technical problems of insufficient multi-modal fusion, low calculation efficiency and single generated action are solved, and the gesture animation which is natural, smooth and personalized and meets the real-time interaction requirement can be generated.

A system and method for a wireless device to derive a channel state information (CSI) codebook based on a decoder and a vector quantization codebook, receive a reference signal from a base station, derive an estimated channel from the reference signal, select an entry from the CSI codebook based on the estimated channel and a selection criterion, and report an index of the selected entry to the base station. The wireless device may receive a subset indication, derive therefrom a second CSI codebook, and select the CSI codebook entry from the second CSI codebook. A CSI compression machine learning (ML) system and vector quantization codebook may be obtained by a network controller. The decoder may be a part of the CSI compression ML system, the vector quantization codebook may be based on an encoder of the CSI compression ML system, and they may be sent to the wireless device.

A UE-associated entity may train an encoder to encode uplink control information. The UE-associated entity may determine a quantization codebook to be applied to the encoded uplink control information. The UE-associated entity may share a sequential training dataset with a base station-associated entity, the sequential training dataset including: one of an input vector set or an output vector set; and one of an encoded and unquantized intermediate vector set or an encoded and quantized intermediate vector set. The base station-associated entity may train a decoder based on a quantization codebook and at least the sequential training dataset from the UE-associated entity. When the base station-associated entity receives multiple sequential training datasets for different vendors, the base station-associated entity may train a multi-vendor decoder based on the multiple sequential training datasets.

An example computing device may include memory and one or more processors. The one or more processors are configured to determine a Cartesian grid in a space. The Cartesian grid may correspond to a precision of an arithmetic processing unit. The one or more processors are configured to, for each of a plurality of neural network weights, determine a respective vector in the space, such that each respective vector belongs to a corresponding predefined vector on the Cartesian grid and represents a respective compressed neural network weight of a plurality of compressed neural network weights. The one or more processors are configured to execute, based on the plurality of compressed network weights, a neural network. The one or more processors are configured to generate, based on executing the neural network, an output.

The invention discloses a dense monitoring Internet of Things remote estimation method based on multi-cell semantic enhancement, and belongs to the field of wireless communication. The method comprises the following steps: firstly, an edge device observes a reasoning target, encodes an observation value according to a semantic codebook, and transmits the observation value to an edge server; secondly, the edge server carries out vector quantization on the received signal and sends a quantized code word index to a cloud server through a forward link; and finally, the cloud server performs dequantization and joint decoding on the received quantization index to obtain an estimated value of the reasoning target in the multiple cells, and a remote estimation task is completed. In addition, a loss function is constructed based on an information bottleneck theory and a straight-through estimation gradient approximation method, and the system is trained to be optimal by adopting a two-stage training strategy. According to the method, semantic information extraction of the remote estimation task in a multi-cell scene is realized, the performance of the remote estimation task can be improved while code word redundancy is inhibited, and the utilization efficiency of spectrum resources is remarkably improved.

The application provides a generative image compression method based on vector quantization, and belongs to the technical field of image and video compression. The method comprises the following steps: obtaining a continuous latent representation of an input image through an analysis transformation module; then performing vector quantization processing to obtain discrete indexes of the continuous latent representation and corresponding quantized features; constructing a continuous index probability distribution; predicting a conditional probability distribution of the discrete indexes through a conditional autoregressive entropy model; calculating a coding rate based on the continuous index probability distribution and the conditional probability distribution; reconstructing an image based on the quantized features and constructing a distortion loss; constructing a rate-distortion loss function and jointly training an image compression model comprising the analysis transformation module, the vector quantization module, the conditional autoregressive entropy model and a synthesis transformation module to obtain a trained image compression model; and compressing the input image by using the trained image compression model. The application can realize rate-distortion joint optimization and collaborative improvement of compression efficiency and reconstruction quality.

This provides a method for compressing audio waveforms using machine learning models. [Solution] The method includes the steps of: receiving an audio waveform containing an audio sample for each of a plurality of time steps; processing the audio waveform using an encoder neural network to generate a plurality of feature vectors representing the audio waveform; and generating a coded representation of each of the plurality of feature vectors using a plurality of vector quantizers, each associated with a codebook of the code vectors. Each coded representation of each feature vector identifies a plurality of code vectors containing the respective code vectors from the codebooks of each vector quantizer. The method also includes the step of compressing each coded representation of the plurality of feature vectors to generate a compressed representation of the audio waveform.

The specification provides a method and device for generating a three-dimensional dental model. The method comprises: preprocessing point cloud data of a three-dimensional dental model after orthodontic treatment in an existing data set, and extracting geometric features from the preprocessed point cloud data; obtaining a vector quantization variational autoencoder model to be trained, and training the model by: encoding the extracted geometric features into a hidden space based on the encoder of the vector quantization variational autoencoder model, and further reconstructing the encoded information in the hidden space into the point cloud data of the three-dimensional dental model after orthodontic treatment through the decoder; and training a diffusion model in the hidden space of the vector quantization variational autoencoder model, wherein the diffusion model is used for diffusion processing of the encoded information input into the hidden space to generate new encoded information, so that the decoder generates new point cloud data of the three-dimensional dental model after orthodontic treatment according to the new encoded information.

A distributed photovoltaic cluster rapid clustering method, comprising: S1, extracting the historical meteorological data of the grid-connected point of the distributed photovoltaic power station and the operating characteristic parameter value of the distributed photovoltaic power station, and normalizing the same; S2, performing rapid filtering on the distributed photovoltaic power station based on the approximate k-means vector quantization technology, and obtaining representative typical distributed photovoltaic power stations; S3, screening out the distributed photovoltaic power stations with similar relations between each other through the KNN algorithm; S4, obtaining the adjacent matrix and degree matrix information of the typical distributed photovoltaic power stations by means of the Gaussian kernel calculation method, and constructing a Laplace graph; S5, adopting the "average cut" graph cutting method, obtaining the clustering result of the typical distributed photovoltaic power stations; S6, calculating the distance from each distributed photovoltaic power station to all typical distributed photovoltaic power stations, and distributing to a similar class according to the distance size, and completing the clustering. The scheme realizes rapid and accurate clustering of the distributed photovoltaic cluster.

The application discloses an image vector quantization encoding method and device, a text-image model training method and device, and a text-image model using method and device. The method comprises the following steps: inputting an image into an encoder to obtain intermediate feature vectors corresponding to each image block contained in the image; searching for indexes of image representations closest to the intermediate feature vectors corresponding to each image block in the image in a first codebook; the first codebook contains multiple rows of image representations and corresponding indexes, and the positions of the indexes corresponding to the similar image representations in the first codebook are also adjacent; and replacing the intermediate feature vectors of each image block in the image with the indexes searched to obtain vector quantization encoding corresponding to each image block in the image. The application can greatly save the calculation amount, improve the speed and efficiency of vector quantization encoding, improve the training efficiency of the model, and reduce the consumption of computing resources.

The application provides a speech conversion method, a speech conversion device, an electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: obtaining original speech data of a target speaker; performing segmentation processing on the original speech data to obtain first speech data and second speech data; performing encoding processing on the first speech data and the second speech data through a vector quantization coding network of a speech conversion model to obtain a first text vector, a first speech feature vector, a second text vector and a second speech feature vector; the first speech feature vector and the second speech feature vector are used for representing speech characteristics of the target speaker; performing splicing processing on the first text vector, the first speech feature vector, the second text vector and the second speech feature vector to obtain a target speech vector; and performing decoding processing on the target speech vector through a decoding network of the speech conversion model to obtain target speech data. The application can improve the speech conversion effect.

The present invention provides a method for compressing a signal, the method comprising: obtaining a main signal via a signal recording module; modeling a model signal of the main signal via a processor by: obtaining a sampled signal via the processor; obtaining a windowed signal via the processor; and extracting, via the processor: a fundamental frequency waveform having a fundamental magnitude and a fundamental phase; and at least one harmonic frequency waveform having a harmonic magnitude and a harmonic phase; wherein the model signal comprises the fundamental frequency waveform and the at least one harmonic frequency waveform; calculating, via the processor, an error signal between a reconstructed signal and the main signal; determining, via the processor, an optimal gain according to at least: an averaging step providing an average, a predefined threshold, and a scaling signal, wherein the scaling signal is a historical error signal scaled by iteratively: averaging a difference between the error signal and the scaling signal, wherein the optimal gain comprises a predefined gain when the average satisfies the predefined threshold; determining, via the processor, an index from a residual signal by: determining the residual signal; vector quantizing the residual signal; and indexing the vector quantized residual signal; synthesizing, via the processor, a compressed signal, wherein the compressed signal comprises: the fundamental phase; the fundamental magnitude; the harmonic phase; the harmonic magnitude the optimal gain; the index. Thus, the compression method preferably overcomes problems associated with current compression techniques, and provides a suitable technique for compressing a signal that can be used to infer a type of load on a circuit.

The application discloses an audio encoding method, an audio decoding method and an audio encoding and decoding system. The method comprises the following steps: obtaining audio to be transmitted and a transmission bit rate; extracting a semantic feature sequence of the audio, and performing vector quantization on the semantic feature sequence based on a semantic codebook to obtain a discrete semantic token sequence encoded based on a first bit rate; extracting a first acoustic feature of the audio, determining a second bit rate which satisfies a transmission bit rate constraint together with the first bit rate, performing downsampling processing on the first acoustic feature based on the second bit rate to obtain a second acoustic feature, and performing vector quantization on the second acoustic feature based on an acoustic codebook to obtain a discrete acoustic token sequence encoded based on the second bit rate; and transmitting the discrete semantic token sequence and the discrete acoustic token sequence to an audio decoding device to reconstruct the audio. The application solves the technical problem that a traditional audio encoding and decoding scheme is difficult to simultaneously consider strong semantic representation and high-fidelity reconstruction in an ultralow bit rate scene.

This invention provides a vector quantized image modeling method and system. [Solution] The method provides a vector quantized image modeling (VIM) technique that includes the step of pre-training a machine learning model (e.g., a transformer model) to autoregressively predict rasterized image tokens. Individual image tokens are encoded from a trained vision transformer-based VQGAN (ViT-VQGAN). [Effects] This paper proposes multiple improvements to vanilla VQGAN, from architecture to codebook learning, resulting in better performance and reconstruction fidelity. The improved ViT-VQGAN further enhances vector quantized image modeling tasks, including image generation and unsupervised representation learning.

The present invention reduces the number of encoded bits in vector quantization. This encoding device is provided with: a quantizing circuit which generates quantization parameters including first information on a vector quantization codebook, and second information on code vectors included in the codebook; and a control circuit which employs the second number of bits based on the difference between the first number of bits available for encoding of a sub-vector in the vector quantization, and the number of bits for the sub-vector quantization parameters, to control encoding of the first information with respect to the sub-vector.

This application discloses an autonomous driving scene generation method based on a spatiotemporal decoupled world model, belonging to the field of autonomous driving technology. The method includes: firstly, acquiring a multimodal autonomous driving dataset; after preprocessing to construct a state sequence; then, discretizing the state sequence into a pose word sequence and an image word sequence using an equidistant binning strategy and an improved time-aware vector quantization encoder; subsequently, inputting the word sequence into a spatiotemporal multimodal fusion module; and decoupling spatiotemporal information and predicting the potential state features of the next moment by alternately stacking temporal Transformer layers and spatial multimodal Transformer layers; during the training phase, employing a random masking strategy to prevent long-term generation drift, and calculating the cross-entropy loss between predicted words and ground truth words generated by the internal state autoregression module to update the model parameters; finally, generating autonomous driving scene data through binning inverse operation and a time-aware decoder; the method of this invention has the ability to generate long-term, high-fidelity videos.

The invention discloses an image anti-counterfeiting method based on vector quantization and a phase mask, and the method comprises the steps: firstly, selecting a two-dimensional code which is simple in structure and only has transverse and longitudinal black and white change characteristics as an information medium, and simplifying a subsequent phase modulation structure while keeping the recognition characteristics clear; a vector quantization image compression technology is utilized to effectively code a high-definition image with rich content into a simplified graphic expression in which a two-dimensional code can be embedded, and effective mapping and compression from image complexity to a barcode structure are realized; and finally, a low-order pure-phase optical mask is generated based on the two-dimensional code, embedding and optical reconstruction of a high-definition image are realized under the condition that the phase quantization order is relatively low, and both the image identification degree and the mask preparation feasibility are considered. Therefore, the innovative image-level anti-counterfeiting coding method which integrates physical anti-counterfeiting and digital anti-counterfeiting advantages and has high information bearing capacity and strong physical anti-counterfeiting performance is constructed. The invention provides a feasible, efficient and low-cost new path for physical anti-counterfeiting of high-definition images.

This application provides a method, apparatus, and computer equipment for repairing missing areas in well logging electrical imaging, belonging to the field of oil and gas exploration and development technology. The method includes: preprocessing the obtained current well logging electrical imaging image and creating a mask for the missing areas in the preprocessed image; inputting the preprocessed image and the mask into a pre-constructed completion model to fill the missing areas in the image, and outputting a completed image. The completion model is obtained by training an initially constructed denoising diffusion probability model in stages. The network used for denoising inverse diffusion in the completion model is a vector quantization variational autoencoder, and a noisy image is introduced as prior information of the known region to constrain the denoising inverse diffusion process. This method, combining a denoising diffusion probability model, a vector quantization variational autoencoder, and prior information of the known region based on the noisy image, achieves accurate filling of blank strips.

A method for compressing a video stream includes retrieving a plurality of frames corresponding to the video stream. For each of two or more sequential frames of the plurality of frames of the video stream, the method includes extracting Key Point Descriptors (KPDs) for the respective frame and processing the respective frame using Principle Component Analysis (PCA) followed by vector quantization, resulting in a quantized explained variance matrix for the respective frame. The quantized explained variance matrix for the respective frame is stored. The KPDs for the respective frame are stored.

A system and methods for upsampling compressed data using a jointly trained Vector Quantized Variational Autoencoder (VQ-VAE) and neural upsampler. The system compresses input data into a discrete latent space using a VQ-VAE encoder, reconstructs the data using a VQ-VAE decoder, and enhances the reconstructed data using a neural upsampler. The VQ-VAE and neural upsampler are jointly trained using a combined loss function, enabling end-to-end optimization. The system allows for efficient compression and high-quality reconstruction of various data types, including financial time-series, images, audio, video, sensor data, and text. The learned discrete latent space can be explored and manipulated using techniques such as interpolation, extrapolation, and vector arithmetic to generate new or modified data samples. The system finds applications in data storage, transmission, analysis, and generation across multiple domains.

A two-stage approach for learning and generating an expressive text-to-motion animation from partially annotated datasets (T2M-X). In an example implementation, T2M-X builds a unified motion dataset based on partially annotated datasets. In the first stage, T2M-X uses the unified motion dataset to train three vector-quantized variational autoencoders (VQ-VAE) for body, hand, and face, respectively, and generate high-quality motion outputs. In the second stage, T2M-X uses the high-quality motion outputs to train a multi-indexing generative pre-trained transformer (GPT) model that includes motion consistency loss and sequence length consistency for learning and then generating coordinated and expressive animations.

The invention provides a side channel analysis method based on frequency domain loss auto-encoder data enhancement, which comprises the following steps of: adding a frequency domain loss item to a reconstruction loss part of a vector quantization variational auto-encoder loss function, and synchronously utilizing frequency domain and time domain information to improve the loss function of an auto-encoder; constructing an auto-encoder model based on the improved loss function, and training the improved auto-encoder model by using original data; performing data enhancement by using the trained model to generate a new energy trace; and finally, performing side channel analysis by using the newly generated energy trace. The method can better help the model to learn the periodic structure and frequency characteristics of the energy trace, thereby generating more real energy trace data, expanding the scale of a data set for side channel analysis, improving the generalization ability of a deep learning side channel analysis model, and improving the effect of side channel analysis.

An audio quantizer for quantizing a plurality of audio information elements comprises: a first-stage vector quantizer (141, 143) for quantizing the plurality of audio information elements to determine a first-stage vector quantization result and a plurality of intermediate quantized elements corresponding to the first-stage vector quantization result; a residual element determinator (142) for calculating a plurality of residual elements from the plurality of intermediate quantized elements and the plurality of audio information elements;and a second-stage vector quantizer (145) for quantizing the plurality of residual elements to obtain a second-stage vector quantization result, wherein the first-stage vector quantization result and the second-stage vector quantization result are a quantized representation of the plurality of audio information elements.;

The invention discloses a channel state information acquisition method and device, equipment and a medium, and belongs to the field of communication. The method is executed by a first node, and the method comprises the steps that a first signal from a second node is received, and the second node comprises a low-power-consumption terminal or a low-power-consumption module; and acquiring first channel state information (CSI) between the first node and the second node based on a measurement result of the first signal. Since the acquisition of the first CSI does not depend on the measurement of the second node, the power consumption required by the second node to measure the reference signal is reduced. Moreover, the acquisition of the first CSI does not depend on the second node to feed back the CSI information, so that the resource overhead required by the second node to feed back the CSI information is reduced. Moreover, the CSI information does not need to be fed back based on a vector quantization or codebook method, so that the loss of the precision and accuracy of the CSI information is avoided.

The application relates to the technical field of medical image processing, in particular to a microstructure model optimization method and device based on diffusion prior and electronic equipment, which comprises the following steps: encoding microstructure parameters of a microstructure model into a latent space through a vector quantization autoencoder; learning a diffusion parameter distribution prior of the microstructure parameters in the latent space by using a latent diffusion model; determining noise latent features of a previous time step according to the diffusion parameter distribution prior, denoising the noise latent features of the previous time step to obtain noise latent features of a current time step, and optimizing the noise latent features of the current time step according to actually collected diffusion magnetic resonance data; denoising the optimized noise latent features to obtain noise-free latent features, generating synthetic data by using a microstructure model and decoded noise-free latent feature parameters, and optimizing the microstructure parameters of the microstructure model according to the synthetic data and the diffusion magnetic resonance data. Thus, the problems of insufficient generalization ability and poor universality of related technologies are solved.

The invention discloses a grouping air federated learning method and system based on static feature distribution. The method comprises the following steps: dividing a global model update vector into a plurality of signal sub-segments according to dimensions; static grouping is carried out according to local data distribution characteristics of the clients, and each group of clients is only responsible for transmission tasks of partial sub-segments; after locally completing model training, each client carries out vector quantization coding on an update vector, extracts a code word index, and sends a coded sub-segment signal to the server; after the server receives each group of uploaded signals, the signals are recovered under the condition of considering channel distortion and noise, the atomic segment content is returned based on the recovered quantization index vector, and the global model is updated in combination with an error feedback mechanism; according to the process, aggregation optimization of the whole model can be completed through rotary sub-segment scheduling and multi-round iteration. According to the method, the overhead of each round of communication is effectively reduced, the anti-noise capability is enhanced, and the overall communication efficiency and the model convergence performance of the federated learning system under the wireless channel are improved.

The application discloses a kind of space-time coding matrix generation methods based on physical drive vector quantization automatic encoder, with target harmonic scattering mode as input, physical drive vector quantization automatic encoder can quickly output optimal discrete space-time coding matrix.The physical drive vector quantization automatic encoder includes an encoder module, a vector quantization layer and a physically driven decoder module, the output of the encoder module is passed to the vector quantization layer, which is converted into a discrete vector to obtain a discrete space-time coding matrix, while the physical operating mechanism between the space-time coding matrix and the harmonic scattering mode is introduced into the decoder module of the automatic encoder for reconstructing the harmonic scattering mode, so the physical drive vector quantization automatic encoder is trained in an unsupervised manner, without preparing a large amount of artificially labeled data.The application can help space-time coding digital metasurface to realize flexible real-time multi-harmonic beamforming.

A method for generating vector-quantized deep neural networks includes receiving a training dataset that includes one or more image, text information, sound information, training a neural model with the training dataset to adjust one or more parameters associated with one or more deep neural network layers, segmenting the one or more parameters associated with each of the one or more deep neural network layers into one or more segments based on a type and a size of each of the one or more deep neural network layers, generating one or more fixed codebook, wherein the fixed codebook include a predetermined number of codewords, replacing each of the one or more segments with one of the codewords, and in response to replacing the one or more segments with one of the codewords, outputting a trained neural model that utilizes the one or more fixed codebooks.