114 results about "Vector quantisation" patented technology

A system and method for multi-scale temporal attention processing in multimodal technology deep learning systems. This system processes time-series, textual, sentiment, and structured tabular data across three hierarchically-organized temporal streams—quarterly, weekly, and intraday levels—with bidirectional cross-temporal information flow. Scale-specific attention mechanisms are optimized for respective temporal granularities, while an adaptive controller dynamically weights each temporal level based on real-time market volatility indicators. A multi-scale fusion processor integrates attention-weighted representations to generate temporally unified representations preserving both short-term market dynamics and long-term trends. This approach enables superior forecasting and risk assessment by leveraging temporal correlations across multiple time scales while automatically adapting to changing market conditions. The system facilitates interpretable AI analysis through attention visualization and enables synthetic scenario generation for model testing.

The present disclosure describes techniques for generating latent representations of images using a machine learning model. An image is split and flattened into a series of patches. The series of patches is concatenated with a sequence of latent tokens. The concatenated patches and latent tokens are input into an encoder of the machine learning model. A one-dimensional (1D) latent representation of the image is generated by the encoder. Vector quantization is performed on the 1D latent representation of the image by a vector quantizer of the machine learning model to generate quantized latent tokens. The image is reconstructed based on the quantized latent tokens by a decoder of the machine learning model.

The invention discloses a signal feature compression method based on codebook discrete quantization and multi-task learning, and belongs to the technical field of crossing of signal processing and artificial intelligence. In order to solve the problem that in the prior art, compression efficiency, semantic retention and calculation complexity are difficult to consider in high-dimensional IQ signal compression, high-dimensional continuous features of an original IQ signal are extracted through an encoder; carrying out vector quantization by utilizing the learnable codebook to generate a discrete index and calculating quantization loss; inputting the discrete features into a decoder branch reconstruction signal to calculate reconstruction loss, and inputting a classifier branch prediction category to calculate classification loss; processing traditional features and coding features by adopting a multi-layer perceptron, and calculating comparison loss through cosine similarity; combining optimization quantization loss, reconstruction loss, classification loss and comparison loss to train a model; and finally outputting a discrete index as a compression feature. The method realizes efficient compression and classification semantic reservation, and is suitable for wireless communication, Internet of Things and other scenes.

The invention provides a semantic information transmission communication method and system based on VQGAN optimization. The method comprises the following steps: establishing a VQGAN model for an image; inputting an image to be transmitted into an encoder of the VQGAN model, and extracting image features corresponding to the image by the encoder; searching the index represented by the discrete potential space with the nearest distance according to the feature vector of the image feature; the index of the embedded vector is recorded, channel coding and modulation are carried out, and transmission is carried out through a channel; after channel transmission, the index is recovered through demodulation and channel decoding; and finding the spatial representation in the codebook according to the index, and performing image reconstruction by using a decoder of a receiving end to obtain a reconstructed image corresponding to the to-be-transmitted image. According to the method, a discrete codebook is established in a vector quantization mode, transmission content is reduced by only transmitting an index of the codebook, and high-resolution image reconstruction is performed based on a generative adversarial network and dynamic change of a confrontation channel and compatibility among different devices based on traditional channel coding.

The embodiment of the invention provides a self-supervised emotion recognition method based on a heart and brain combined codebook and related equipment, and belongs to the technical field of physiological signal processing and artificial intelligence. The method comprises the following steps: respectively defining heart beats of electrocardiosignals and electroencephalogram signal segments with equal lengths as words, and constructing sentences; a shared heart and brain joint codebook is created and trained, and electrocardio and electroencephalogram words are mapped to a unified discrete semantic space through vector quantization so as to learn cross-subject general characterization; then, discretizing a signal sentence by using the codebook, and combining space and position embedding and inputting a Transform encoder to carry out mask pre-training so as to learn context semantics of the signal; and finally, finely tuning the pre-training model for an emotion recognition task. According to the method, deep semantic fusion of heart and brain signals is realized through signal structuring and codebook sharing, dependence on labeled data is effectively overcome, and emotion recognition accuracy and cross-subject generalization ability are remarkably improved.

The invention discloses a multi-modal time sequence fusion voice-driven gesture generation method, which comprises the following steps of: firstly, learning compact discrete representation of gesture motion through a vector quantization variational auto-encoder model, and constructing a quantized potential space for a subsequent generation task; extracting audio features of the voice audio through an audio encoder; performing low-dimensional embedding on the identity of the speaker to obtain an identity feature; performing time sequence alignment on the audio features and the historical gesture sequence and then splicing the audio features and the historical gesture sequence along feature dimensions to form multi-modal initial representation; deep feature fusion is carried out through a multi-modal time sequence fusion module integrating a self-attention mechanism, a cross attention mechanism taking identity features as conditions and a Mamba module; and finally, reconstructing a target gesture sequence through a pre-trained decoder. According to the method, the technical problems of insufficient multi-modal fusion, low calculation efficiency and single generated action are solved, and the gesture animation which is natural, smooth and personalized and meets the real-time interaction requirement can be generated.

A system and method for a wireless device to derive a channel state information (CSI) codebook based on a decoder and a vector quantization codebook, receive a reference signal from a base station, derive an estimated channel from the reference signal, select an entry from the CSI codebook based on the estimated channel and a selection criterion, and report an index of the selected entry to the base station. The wireless device may receive a subset indication, derive therefrom a second CSI codebook, and select the CSI codebook entry from the second CSI codebook. A CSI compression machine learning (ML) system and vector quantization codebook may be obtained by a network controller. The decoder may be a part of the CSI compression ML system, the vector quantization codebook may be based on an encoder of the CSI compression ML system, and they may be sent to the wireless device.

A UE-associated entity may train an encoder to encode uplink control information. The UE-associated entity may determine a quantization codebook to be applied to the encoded uplink control information. The UE-associated entity may share a sequential training dataset with a base station-associated entity, the sequential training dataset including: one of an input vector set or an output vector set; and one of an encoded and unquantized intermediate vector set or an encoded and quantized intermediate vector set. The base station-associated entity may train a decoder based on a quantization codebook and at least the sequential training dataset from the UE-associated entity. When the base station-associated entity receives multiple sequential training datasets for different vendors, the base station-associated entity may train a multi-vendor decoder based on the multiple sequential training datasets.

The invention relates to the technical field of wireless communication, and discloses a robust semantic communication method, device and equipment based on sparse vector coding, and the method comprises the steps: extracting continuous semantic features of original data through a semantic encoder model; discretizing the continuous semantic features into index bits through a vector quantization method by using a learnable feature dictionary; the index bit is mapped into a sparse vector by adopting sparse vector coding, and the sparse vector is sent after being expanded by a codebook; and a receiving end identifies the index through a multipath matching pursuit algorithm and reconstructs the original data. According to the method, a semantic feature extraction model, a vector quantization technology and sparse vector coding are fused, and gradient approximation, joint loss function optimization and a transmission parameter dynamic adjustment mechanism are designed, so that the problems of poor digital transmission compatibility and weak channel adaptability of traditional semantic communication are effectively solved; and the semantic transmission reliability and the system resource utilization efficiency under the dynamic channel condition are improved.

An example computing device may include memory and one or more processors. The one or more processors are configured to determine a Cartesian grid in a space. The Cartesian grid may correspond to a precision of an arithmetic processing unit. The one or more processors are configured to, for each of a plurality of neural network weights, determine a respective vector in the space, such that each respective vector belongs to a corresponding predefined vector on the Cartesian grid and represents a respective compressed neural network weight of a plurality of compressed neural network weights. The one or more processors are configured to execute, based on the plurality of compressed network weights, a neural network. The one or more processors are configured to generate, based on executing the neural network, an output.

A computing system can include one or more processors and one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the computing system to perform operations including obtaining model structure data indicative of a plurality of parameters of a machine-learned model; determining a codebook comprising a plurality of centroids, the plurality of centroids having a respective index of a plurality of indices indicative of an ordering of the codebook; determining a plurality of codes respective to the plurality of parameters, the plurality of codes respectively comprising a code index of the plurality of indices corresponding to a closest centroid of the plurality of centroids to a respective parameter of the plurality of parameters; and providing encoded data as an encoded representation of the plurality of parameters of the machine-learned model, the encoded data comprising the codebook and the plurality of codes.

A communication method and apparatus. A terminal device determines channel state information based on S indexes of S vectors. Each of the S vectors is included in a first vector quantization dictionary. The first vector quantization dictionary includes N1 vectors, where N1 and S are both positive integers. The terminal device sends the channel state information to a network device. Because a vector included in the vector quantization dictionary usually has a relatively large dimension, quantizing channel information by using the first vector quantization dictionary is equivalent to performing dimension expansion on the channel information or maintaining a relatively high dimension. High-precision feedback is implemented by using relatively low signaling overheads.

The invention discloses a dense monitoring Internet of Things remote estimation method based on multi-cell semantic enhancement, and belongs to the field of wireless communication. The method comprises the following steps: firstly, an edge device observes a reasoning target, encodes an observation value according to a semantic codebook, and transmits the observation value to an edge server; secondly, the edge server carries out vector quantization on the received signal and sends a quantized code word index to a cloud server through a forward link; and finally, the cloud server performs dequantization and joint decoding on the received quantization index to obtain an estimated value of the reasoning target in the multiple cells, and a remote estimation task is completed. In addition, a loss function is constructed based on an information bottleneck theory and a straight-through estimation gradient approximation method, and the system is trained to be optimal by adopting a two-stage training strategy. According to the method, semantic information extraction of the remote estimation task in a multi-cell scene is realized, the performance of the remote estimation task can be improved while code word redundancy is inhibited, and the utilization efficiency of spectrum resources is remarkably improved.

The invention relates to the technical field of speech synthesis, can be applied to the fields of financial science and technology and medical health, and discloses a speech synthesis method, device, equipment and medium, and the method comprises the steps: obtaining original speech, and extracting high-dimensional features from the original speech to obtain high-dimensional speech features; inputting the high-dimensional language features into a pre-trained vector quantizer for discretization to obtain a plurality of discrete Tokens; a prediction Token sequence is generated through a TTS generator according to text information corresponding to the original voice and the multiple discrete Token, and the TTS generator is obtained by adopting a sample set to train and verify a large language model; and inputting the predicted Token sequence into a voice decoder for voice synthesis to obtain target voice. And the quality and the accuracy of the synthesized speech are improved.

The application provides a generative image compression method based on vector quantization, and belongs to the technical field of image and video compression. The method comprises the following steps: obtaining a continuous latent representation of an input image through an analysis transformation module; then performing vector quantization processing to obtain discrete indexes of the continuous latent representation and corresponding quantized features; constructing a continuous index probability distribution; predicting a conditional probability distribution of the discrete indexes through a conditional autoregressive entropy model; calculating a coding rate based on the continuous index probability distribution and the conditional probability distribution; reconstructing an image based on the quantized features and constructing a distortion loss; constructing a rate-distortion loss function and jointly training an image compression model comprising the analysis transformation module, the vector quantization module, the conditional autoregressive entropy model and a synthesis transformation module to obtain a trained image compression model; and compressing the input image by using the trained image compression model. The application can realize rate-distortion joint optimization and collaborative improvement of compression efficiency and reconstruction quality.

This provides a method for compressing audio waveforms using machine learning models. [Solution] The method includes the steps of: receiving an audio waveform containing an audio sample for each of a plurality of time steps; processing the audio waveform using an encoder neural network to generate a plurality of feature vectors representing the audio waveform; and generating a coded representation of each of the plurality of feature vectors using a plurality of vector quantizers, each associated with a codebook of the code vectors. Each coded representation of each feature vector identifies a plurality of code vectors containing the respective code vectors from the codebooks of each vector quantizer. The method also includes the step of compressing each coded representation of the plurality of feature vectors to generate a compressed representation of the audio waveform.

The specification provides a method and device for generating a three-dimensional dental model. The method comprises: preprocessing point cloud data of a three-dimensional dental model after orthodontic treatment in an existing data set, and extracting geometric features from the preprocessed point cloud data; obtaining a vector quantization variational autoencoder model to be trained, and training the model by: encoding the extracted geometric features into a hidden space based on the encoder of the vector quantization variational autoencoder model, and further reconstructing the encoded information in the hidden space into the point cloud data of the three-dimensional dental model after orthodontic treatment through the decoder; and training a diffusion model in the hidden space of the vector quantization variational autoencoder model, wherein the diffusion model is used for diffusion processing of the encoded information input into the hidden space to generate new encoded information, so that the decoder generates new point cloud data of the three-dimensional dental model after orthodontic treatment according to the new encoded information.

A method for compressing a video stream includes retrieving a plurality of frames corresponding to the video stream. For each of two or more sequential frames of the plurality of frames of the video stream, the method includes extracting Key Point Descriptors (KPDs) for the respective frame and processing the respective frame using Principle Component Analysis (PCA) followed by vector quantization, resulting in a quantized explained variance matrix for the respective frame. The quantized explained variance matrix for the respective frame is stored. The KPDs for the respective frame are stored.

A distributed photovoltaic cluster rapid clustering method, comprising: S1, extracting the historical meteorological data of the grid-connected point of the distributed photovoltaic power station and the operating characteristic parameter value of the distributed photovoltaic power station, and normalizing the same; S2, performing rapid filtering on the distributed photovoltaic power station based on the approximate k-means vector quantization technology, and obtaining representative typical distributed photovoltaic power stations; S3, screening out the distributed photovoltaic power stations with similar relations between each other through the KNN algorithm; S4, obtaining the adjacent matrix and degree matrix information of the typical distributed photovoltaic power stations by means of the Gaussian kernel calculation method, and constructing a Laplace graph; S5, adopting the "average cut" graph cutting method, obtaining the clustering result of the typical distributed photovoltaic power stations; S6, calculating the distance from each distributed photovoltaic power station to all typical distributed photovoltaic power stations, and distributing to a similar class according to the distance size, and completing the clustering. The scheme realizes rapid and accurate clustering of the distributed photovoltaic cluster.

The application discloses an image vector quantization encoding method and device, a text-image model training method and device, and a text-image model using method and device. The method comprises the following steps: inputting an image into an encoder to obtain intermediate feature vectors corresponding to each image block contained in the image; searching for indexes of image representations closest to the intermediate feature vectors corresponding to each image block in the image in a first codebook; the first codebook contains multiple rows of image representations and corresponding indexes, and the positions of the indexes corresponding to the similar image representations in the first codebook are also adjacent; and replacing the intermediate feature vectors of each image block in the image with the indexes searched to obtain vector quantization encoding corresponding to each image block in the image. The application can greatly save the calculation amount, improve the speed and efficiency of vector quantization encoding, improve the training efficiency of the model, and reduce the consumption of computing resources.

The application provides a speech conversion method, a speech conversion device, an electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: obtaining original speech data of a target speaker; performing segmentation processing on the original speech data to obtain first speech data and second speech data; performing encoding processing on the first speech data and the second speech data through a vector quantization coding network of a speech conversion model to obtain a first text vector, a first speech feature vector, a second text vector and a second speech feature vector; the first speech feature vector and the second speech feature vector are used for representing speech characteristics of the target speaker; performing splicing processing on the first text vector, the first speech feature vector, the second text vector and the second speech feature vector to obtain a target speech vector; and performing decoding processing on the target speech vector through a decoding network of the speech conversion model to obtain target speech data. The application can improve the speech conversion effect.

The present invention provides a method for compressing a signal, the method comprising: obtaining a main signal via a signal recording module; modeling a model signal of the main signal via a processor by: obtaining a sampled signal via the processor; obtaining a windowed signal via the processor; and extracting, via the processor: a fundamental frequency waveform having a fundamental magnitude and a fundamental phase; and at least one harmonic frequency waveform having a harmonic magnitude and a harmonic phase; wherein the model signal comprises the fundamental frequency waveform and the at least one harmonic frequency waveform; calculating, via the processor, an error signal between a reconstructed signal and the main signal; determining, via the processor, an optimal gain according to at least: an averaging step providing an average, a predefined threshold, and a scaling signal, wherein the scaling signal is a historical error signal scaled by iteratively: averaging a difference between the error signal and the scaling signal, wherein the optimal gain comprises a predefined gain when the average satisfies the predefined threshold; determining, via the processor, an index from a residual signal by: determining the residual signal; vector quantizing the residual signal; and indexing the vector quantized residual signal; synthesizing, via the processor, a compressed signal, wherein the compressed signal comprises: the fundamental phase; the fundamental magnitude; the harmonic phase; the harmonic magnitude the optimal gain; the index. Thus, the compression method preferably overcomes problems associated with current compression techniques, and provides a suitable technique for compressing a signal that can be used to infer a type of load on a circuit.

The invention relates to the technical field of data management, in particular to an online centralized management method and system for electric power marketing, and the method comprises the steps: constructing typical power consumption behavior characteristics of various electric power fee packages in advance, and then directly extracting the power consumption behavior characteristics affecting the selection decision of the electric power fee packages. Matching the power consumption behavior characteristics of the power consumer in the current test period with the typical power consumption behavior characteristics of the power fee package currently selected by the power consumer, and if not, further matching to obtain a matched power fee package conforming to the current monthly power consumption behavior characteristics; and the data are transmitted to a marketing decision-making department as basic data of power marketing. Due to the pre-constructed typical power consumption behavior characteristics, vector quantization processing can be carried out on a large number of extracted user behavior characteristics by using the typical power consumption behavior characteristics as reference data, the storage data volume can be effectively reduced, a special power marketing database is constructed, and a data basis is provided for a power marketing decision-making department.

The application discloses an audio encoding method, an audio decoding method and an audio encoding and decoding system. The method comprises the following steps: obtaining audio to be transmitted and a transmission bit rate; extracting a semantic feature sequence of the audio, and performing vector quantization on the semantic feature sequence based on a semantic codebook to obtain a discrete semantic token sequence encoded based on a first bit rate; extracting a first acoustic feature of the audio, determining a second bit rate which satisfies a transmission bit rate constraint together with the first bit rate, performing downsampling processing on the first acoustic feature based on the second bit rate to obtain a second acoustic feature, and performing vector quantization on the second acoustic feature based on an acoustic codebook to obtain a discrete acoustic token sequence encoded based on the second bit rate; and transmitting the discrete semantic token sequence and the discrete acoustic token sequence to an audio decoding device to reconstruct the audio. The application solves the technical problem that a traditional audio encoding and decoding scheme is difficult to simultaneously consider strong semantic representation and high-fidelity reconstruction in an ultralow bit rate scene.

The application relates to the technical field of wireless communication, and discloses a robust semantic communication method, device and equipment based on sparse vector coding, which comprises the following steps: extracting continuous semantic features of original data through a semantic encoder model; discretizing the continuous semantic features into index bits through a vector quantization method by using a learnable feature dictionary; mapping the index bits into sparse vectors by using sparse vector coding, and transmitting the sparse vectors after codebook extension; and identifying the index and reconstructing the original data by using a multi-path matching tracking algorithm at a receiving end. By fusing a semantic feature extraction model, a vector quantization technology and sparse vector coding, and by designing a gradient approximation, a joint loss function optimization and a transmission parameter dynamic adjustment mechanism, the application effectively solves the problems of poor compatibility and weak channel adaptability of traditional semantic communication digital transmission, and improves the semantic transmission reliability and system resource utilization efficiency under dynamic channel conditions.

This invention provides a vector quantized image modeling method and system. [Solution] The method provides a vector quantized image modeling (VIM) technique that includes the step of pre-training a machine learning model (e.g., a transformer model) to autoregressively predict rasterized image tokens. Individual image tokens are encoded from a trained vision transformer-based VQGAN (ViT-VQGAN). [Effects] This paper proposes multiple improvements to vanilla VQGAN, from architecture to codebook learning, resulting in better performance and reconstruction fidelity. The improved ViT-VQGAN further enhances vector quantized image modeling tasks, including image generation and unsupervised representation learning.

The present invention reduces the number of encoded bits in vector quantization. This encoding device is provided with: a quantizing circuit which generates quantization parameters including first information on a vector quantization codebook, and second information on code vectors included in the codebook; and a control circuit which employs the second number of bits based on the difference between the first number of bits available for encoding of a sub-vector in the vector quantization, and the number of bits for the sub-vector quantization parameters, to control encoding of the first information with respect to the sub-vector.

This application discloses an autonomous driving scene generation method based on a spatiotemporal decoupled world model, belonging to the field of autonomous driving technology. The method includes: firstly, acquiring a multimodal autonomous driving dataset; after preprocessing to construct a state sequence; then, discretizing the state sequence into a pose word sequence and an image word sequence using an equidistant binning strategy and an improved time-aware vector quantization encoder; subsequently, inputting the word sequence into a spatiotemporal multimodal fusion module; and decoupling spatiotemporal information and predicting the potential state features of the next moment by alternately stacking temporal Transformer layers and spatial multimodal Transformer layers; during the training phase, employing a random masking strategy to prevent long-term generation drift, and calculating the cross-entropy loss between predicted words and ground truth words generated by the internal state autoregression module to update the model parameters; finally, generating autonomous driving scene data through binning inverse operation and a time-aware decoder; the method of this invention has the ability to generate long-term, high-fidelity videos.

An audio quantizer for quantizing a plurality of audio information items has: a first stage vector quantizer for quantizing the plurality of audio information items to determine a first stage vector quantization result and a plurality of intermediate quantized items corresponding to the first stage vector quantization result; a residual item determiner for calculating a plurality of residual items from the plurality of intermediate quantized items and the plurality of audio information items; and a second stage vector quantizer for quantizing the plurality of residual items to obtain a second stage vector quantization result, wherein the first stage vector quantization result and the second stage vector quantization result are a quantized representation of the plurality of audio information items.

The invention discloses an image anti-counterfeiting method based on vector quantization and a phase mask, and the method comprises the steps: firstly, selecting a two-dimensional code which is simple in structure and only has transverse and longitudinal black and white change characteristics as an information medium, and simplifying a subsequent phase modulation structure while keeping the recognition characteristics clear; a vector quantization image compression technology is utilized to effectively code a high-definition image with rich content into a simplified graphic expression in which a two-dimensional code can be embedded, and effective mapping and compression from image complexity to a barcode structure are realized; and finally, a low-order pure-phase optical mask is generated based on the two-dimensional code, embedding and optical reconstruction of a high-definition image are realized under the condition that the phase quantization order is relatively low, and both the image identification degree and the mask preparation feasibility are considered. Therefore, the innovative image-level anti-counterfeiting coding method which integrates physical anti-counterfeiting and digital anti-counterfeiting advantages and has high information bearing capacity and strong physical anti-counterfeiting performance is constructed. The invention provides a feasible, efficient and low-cost new path for physical anti-counterfeiting of high-definition images.