An image adaptive quantization communication method and system

Abstract

The application provides an image adaptive quantization communication method and system, relates to the technical field of artificial intelligence and semantic communication, and comprises the following steps: performing feature extraction on an input image to form an initial code word representation; adopting an adaptive dynamic quantization mechanism to perform quantization processing on the initial code word representation to generate a code word index as a quantization feature; after the receiving end obtains the code word index of the sending end, performing inverse quantization by using a codebook knowledge base, and reconstructing a recovered image by using a decoder. The method and system provided by the application avoid representation collapse and realize adaptive matching and dynamic updating between codebook elements and semantic features through multi-codebook cooperation, a learnable linear transformation and a dynamic selection mechanism based on attention; and through quantization representation and transmission of the image and then decoding and recovery, the efficiency of image transmission is greatly improved, and an efficient and intelligent basic framework is provided for a next-generation semantic-oriented image communication system.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and semantic communication technology, and in particular to an image adaptive quantization communication method and system. Background Technology

[0002] In today's era of the Internet of Things, high-resolution image data is exploding, posing unprecedented challenges to the transmission efficiency of communication networks. Against this backdrop, semantic communication systems for image transmission have emerged, with the core objective of shifting from traditional pixel-level fidelity optimization to the efficient and robust representation and reconstruction of the deep semantic content of images. Vector Quantized Variational Autoencoders (VQ-VAEs) map continuous image features to discrete codebooks, forming a shareable "semantic dictionary," providing a highly promising technical architecture for the compression and transmission of image semantics.

[0003] However, existing VQ-VAE-based technical solutions have fundamental flaws in semantic communication applications, resulting in limited performance when facing diverse application scenarios and hindering the practical promotion and application value of this technology. First, to alleviate the codebook "representation collapse" problem, mainstream solutions generally rely on external pre-trained models to define the latent space, causing semantic mismatch in specialized image fields such as medical imaging and remote sensing monitoring, and severely limiting generalization ability. Simultaneously, the performance improvement brought about by codebook expansion has significant bottlenecks, resulting in inefficient use of storage resources. Second, traditional VQ-VAE adopts a static codebook configuration strategy; the rigid structure results in a fixed number and precision of semantic units, making it difficult to achieve differentiated representations for different image content, leading to low semantic information encoding efficiency.

[0004] Therefore, how to achieve efficient transmission and high-quality reconstruction of image data under low-bandwidth channels has become a technical problem that the industry urgently needs to solve. Summary of the Invention

[0005] This invention provides an image adaptive quantization communication method and system to solve the problem of low efficiency in image semantic information encoding in the prior art, and to achieve efficient transmission and high-quality reconstruction of image data under low bandwidth channels.

[0006] This invention provides an image adaptive quantization communication method, comprising:

[0007] The sending end extracts features from the input image using an encoder to obtain the image's deep semantic features, forming an initial codeword representation; the dimensional structure of the input image is as follows: ,in, Indicates the number of channels. Indicates altitude, Indicates width;

[0008] The transmitting end uses an adaptive dynamic quantization mechanism to quantize the initial codeword representation and generate a codeword index as a quantization feature. The adaptive dynamic quantization mechanism refers to the adaptive mapping of continuous deep semantic features into discrete quantization representations through multi-codebook collaboration, learnable linear transformation, and attention-based dynamic selection mechanism.

[0009] The sending end sends the codeword index to the receiving end;

[0010] After obtaining the codeword index, the receiving end performs inverse quantization using the codebook knowledge base and reconstructs the restored image using the decoder.

[0011] In some embodiments, the encoder consists of a convolution operation module, a residual stack, and a ReLU activation function; during each encoding, the encoder configures the number of input channels, the number of output channels, and the feature extraction density of the convolutional layer for the input data.

[0012] In some embodiments, the main feature convolutional layer of the convolution operation module uses a 3×3 convolutional kernel; each convolutional layer is followed by a ReLU activation function.

[0013] In some embodiments, the residual stack consists of two sequentially connected normalized residual blocks; each residual block contains three convolutional layers with kernel sizes of 1×1, 3×3 and 1×1 respectively, and a ReLU activation function is connected after each convolutional layer.

[0014] In some embodiments, the encoder is composed of three convolutional operation modules, two residual stacks, and a ReLU activation function in sequence, wherein the two residual stacks are embedded sequentially between the first two convolutional operation modules; the encoder adopts a hierarchical downsampling architecture, where the first two convolutional layers of the encoder progressively compress the spatial resolution of the input data to 1 / 4 of the original through convolutional operations with a stride of 2, while increasing the channel dimension to 128 dimensions; the last convolutional layer keeps the feature map size unchanged, and the output shape is... .

[0015] In some embodiments, the specific steps of the adaptive dynamic quantization mechanism include:

[0016] S1: Feature Projection and Dimensional Adaptation: Semantic Features Extracted by the Encoder Perform a set of parallel two-dimensional convolution operations The semantic features The channel dimension from a unified dimension Projected onto the independent embedding dimensions corresponding to each codebook. Generate a set of feature vectors And perform a dimension flattening operation on the projected feature vectors. Merge spatial dimensions, keeping the last dimension as 1. ;

[0017] S2: Parameter Initialization: Randomly initialize the network parameter key vector Initialize the linear projection matrix with a normal distribution ;

[0018] S3: Dynamic codebook selection: Selecting the semantic features extracted by the encoder. as query vector Input multi-head attention module, based on learnable key vectors A set of codebook selection weights is calculated; these weights are then input into the Gumbel-Softmax function for processing, generating an approximate one-hot distribution. Three-dimensional codebook selection score;

[0019] S4: Nearest Neighbor Quantization and Straight-Through Estimation: For each codebook branch The projected feature vector and the code vector after online linear transformation Perform a nearest neighbor search among them to obtain the code vector index matrix with the minimum Euclidean distance. ,in, The pass-through estimator technique is used to output the quantization results during forward propagation. In backpropagation, the gradient of the original feature is directly transmitted, i.e. ;

[0020] S5: Quantization Dimension Alignment: Align the quantization results of each codebook branch. Transform into dimensional reshaping A three-dimensional matrix is ​​formed, and the channel dimensions are restored to the unified dimension through a convolutional projection layer. ,Right now ;in, Indicates the first i The quantization result of each codebook branch through dimension reshaping and channel dimension alignment after 1×1 convolutional projection;

[0021] S6: Quantization result stitching: ... One recovery to dimension The subsequent codebook features are concatenated along the feature dimensions to form dimensional comprehensive feature matrix ;

[0022] S7: Quantization Result Fusion: Using the codebook, select scores and perform batch matrix multiplication (BMM) on the comprehensive feature matrix. Perform weighted fusion to generate the final result. Quantization characteristics of dimension ;

[0023] S8: Quantization Loss Calculation: During training, the arithmetic mean of the vector quantization losses based on all codebooks is calculated as an additional loss term. Image reconstruction loss Determine the total loss The model is optimized; the vector quantization loss includes codebook loss and commitment loss; the codebook loss drives the vectors in the codebook to converge towards the encoder output features; the commitment loss constrains the encoder output features to converge towards their corresponding nearest neighbor code vectors, i.e. ;in, To reconstruct the image; MSE refers to mean squared error; Weighting parameters for adjusting the quantization loss ratio;

[0024] S9: Convergence Judgment and Update: If the total loss converges, output the reconstructed image and codeword index, and transmit the codeword index to the receiving end; if the total loss does not converge, update the key vector. and linear projection matrix Then, jump to S3 to continue training.

[0025] In some embodiments, the receiving end includes an inverse quantization module, a codebook knowledge base, and a decoder; the receiving end uses the trained codebook knowledge base and the inverse quantization module to convert the received codeword index into quantized codewords, and inputs the quantized codewords into the decoder to decode and restore the image.

[0026] In some embodiments, the decoder consists of two convolutional operation modules, a residual stack module, and a ReLU activation function in sequence; the residual stack module is located in the first layer of the decoder; the feature map is upsampled stepwise through two levels of 4×4 convolutional operations with a stride of 2, and based on the decoder, the input... The quantized feature map is decoded and reconstructed as The output image.

[0027] In some embodiments, the residual stack module in the decoder consists of two residual blocks, each containing a 3×3 kernel convolutional layer and a ReLU activation function.

[0028] The present invention provides an image adaptive quantization communication system, including an encoder, an adaptive quantization module, a codebook knowledge base, an inverse quantization module, and a decoder for executing the image adaptive quantization communication method.

[0029] The present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the image adaptive quantization communication method.

[0030] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the image adaptive quantization communication method.

[0031] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image adaptive quantization communication method.

[0032] This invention provides an image adaptive quantization communication method and system. Through multi-codebook collaboration, learnable linear transformation, and attention-based dynamic selection mechanisms, it adaptively maps continuous deep semantic features into discrete quantized representations, achieving adaptive matching and dynamic updating between codebook elements and semantic features. By quantizing the image, transmitting it, and then decoding and restoring it, the efficiency of image transmission is greatly improved. Compared with existing schemes that rely on external pre-trained models or fixed codebook configurations, this invention can automatically optimize the semantic latent space structure during end-to-end training, avoiding codebook representation collapse and significantly enhancing the model's generalization ability and task adaptability. Simultaneously, this method can adaptively select the codebook size and embedding dimension based on the semantic content of the input image, thereby achieving optimal allocation of codebook resources while ensuring semantic expression accuracy. In summary, this invention exhibits lower reconstruction errors and stronger robustness in image semantic compression and reconstruction tasks, providing an efficient and intelligent infrastructure for next-generation semantic-oriented image communication systems. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0034] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0035] Figure 1 This is a flowchart illustrating an image adaptive quantization communication method provided by the present invention.

[0036] Figure 2This invention provides an encoder device and workflow for an image adaptive quantization communication method.

[0037] Figure 3 This is an adaptive dynamic quantization flowchart of an image adaptive quantization communication method provided by the present invention.

[0038] Figure 4 This invention provides a decoder device and workflow for an image adaptive quantization communication method.

[0039] Figure 5 This is a system structure diagram of an image adaptive quantization communication system provided by the present invention.

[0040] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0041] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0042] It should be noted that the terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps, units, or modules is not necessarily limited to those explicitly listed, but may include other steps, units, or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0043] Figure 1 This is a flowchart illustrating an image adaptive quantization communication method provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 110, 120, 130 and 140.

[0044] Step 110: The sending end extracts features from the input image using an encoder to obtain the deep semantic features of the image, forming an initial codeword representation; the dimensional structure of the input image is as follows: ,in, Indicates the number of channels. Indicates altitude, Indicates width;

[0045] Specifically, the execution subject of the image adaptive quantization communication method provided in this embodiment of the invention is an image adaptive quantization communication system.

[0046] To address the shortcomings of existing VQ-VAE-based technologies in semantic communication applications, a solution is needed that can eliminate external model dependencies and directly optimize the overall structure of the semantic latent space through built-in mechanisms, fundamentally preventing representation collapse and enhancing model generalization ability. Simultaneously, this solution needs to overcome static configuration limitations and achieve adaptive quantization for single images, enabling models to intelligently and dynamically balance codebook size and embedding dimension based on their respective semantic complexity, ultimately achieving optimal codebook resource allocation. This method proposes a learnable linear projection matrix to dynamically optimize the codebook space and utilizes an attention-driven quantization policy network to generate the optimal codebook usage combination for each image instance in real time, ultimately achieving a significant improvement in semantic representation efficiency and reconstruction quality without prior knowledge. Therefore, this invention proposes an image adaptive quantization communication method based on a semantic knowledge base with adaptive dynamic quantization and codebook updates.

[0047] This invention provides an image adaptive quantization communication method, considering a scenario of large-volume image transmission in a low-bandwidth channel environment. In this scenario, the input image... have dimensional structure (where Indicates the number of channels. Indicates altitude, (This refers to the width; taking the standard Red Green Blue (RGB) color mode as an example, the number of channels...) The value is 3.

[0048] The system comprises a complete processing unit consisting of an encoder, an adaptive quantization module, a codebook knowledge base, an inverse quantization module, and a decoder. These modules work collaboratively to achieve efficient image transmission under low bandwidth constraints while maintaining image semantic fidelity.

[0049] The sending end first obtains the deep semantic features of the image through feature extraction to form the initial codeword representation.

[0050] In this embodiment of the invention, the image is encoded by an encoder to obtain semantic features. ,Right now ,in, E It refers to the encoder. For shape Input, This represents the set of parameters for this encoding.

[0051] Step 120: The sending end uses an adaptive dynamic quantization mechanism to quantize the initial codeword representation and generate a codeword index as a quantization feature; wherein, the adaptive dynamic quantization mechanism refers to adaptively mapping continuous deep semantic features into discrete quantized representations through multi-codebook collaboration, learnable linear transformation and attention-based dynamic selection mechanism.

[0052] Specifically, the sending end uses an adaptive dynamic quantization mechanism to quantize the codeword and generate a codeword index as a quantization feature.

[0053] After obtaining continuous deep semantic features (initial codeword representation), these continuous features need to be converted into discrete symbol representations in order to achieve efficient transmission in low-bandwidth channels. This embodiment does not use the traditional fixed codebook quantization method, but instead uses an adaptive dynamic quantization mechanism to achieve this process.

[0054] In this embodiment of the invention, the adaptive dynamic quantization mechanism refers to adaptively mapping continuous deep semantic features into discrete quantized representations through multi-codebook collaboration, learnable linear transformation, and attention-based dynamic selection. The core of this mechanism lies in breaking the limitations of static codebook configuration in traditional schemes, enabling real-time and dynamic adjustment of the quantization strategy based on the semantic complexity of the input image content. Specifically, this mechanism mainly includes the collaborative work of the following key layers:

[0055] First, a multi-codebook collaborative and learnable linear transformation architecture is introduced. Since a single codebook often struggles to cover the complex and diverse semantic space of images, the system pre-defines multiple parallel codebook resources. However, the semantic features output by the encoder are uniform. To enable the same feature to adapt to different codebook spaces, this embodiment utilizes a learnable linear transformation matrix to perform a projection transformation on the initial codeword representation. This transformation maps shared semantic features to independent embedding subspaces that match each specific codebook, thereby decoupling the dimensional limitations of feature extraction and codebook representation, allowing each codebook to perform quantization matching on its most proficient feature dimension.

[0056] Secondly, to determine which codebook combinations to use to represent the current image features, this embodiment of the invention introduces an attention-based dynamic selection mechanism. This mechanism treats the deep semantic features extracted by the encoder as query vectors. Through the key vector within the system Interactive computation is performed to measure the correlation or matching degree between the current image content and various codebooks.

[0057] Based on the results of the dynamic selection process described above, the linearly transformed features are subjected to a nearest neighbor search within the selected codebook to find the best-matching codeword, and the index of that codeword in the codebook is output. These codeword indices constitute the quantized features that need to be transmitted in the end.

[0058] Step 130: The sending end sends the codeword index to the receiving end.

[0059] Specifically, after the aforementioned adaptive dynamic quantization process, the original high-dimensional, continuous image data has been transformed into a set of highly compressed, discrete codeword indices. The transmitting end sends these codeword indices to the receiving end via a physical communication channel.

[0060] Step 140: After obtaining the codeword index, the receiving end performs inverse quantization using the codebook knowledge base and reconstructs the restored image using the decoder.

[0061] Specifically, after acquiring the quantization features, the receiving end uses a codebook knowledge base for inverse quantization and reconstructs a restored image that is semantically consistent with the original image. .

[0062] In this embodiment of the invention, after the receiving end receives the codeword index data from the sending end through the communication interface, it initiates the image reconstruction process; the process mainly includes two stages: inverse quantization and decoding reconstruction.

[0063] This invention provides an image adaptive quantization communication method that adaptively maps continuous deep semantic features into discrete quantized representations through multi-codebook collaboration, learnable linear transformation, and attention-based dynamic selection mechanisms. This achieves adaptive matching and dynamic updating between codebook elements and semantic features. By quantizing the image, transmitting it, and then decoding and restoring it, the efficiency of image transmission is greatly improved. Compared with existing schemes that rely on external pre-trained models or fixed codebook configurations, this invention can automatically optimize the semantic latent space structure during end-to-end training, avoiding codebook representation collapse and significantly enhancing the model's generalization ability and task adaptability. Furthermore, this method can adaptively select the codebook size and embedding dimension based on the semantic content of the input image, thereby achieving optimal allocation of codebook resources while ensuring semantic expression accuracy. In summary, this invention exhibits lower reconstruction errors and stronger robustness in image semantic compression and reconstruction tasks, providing an efficient and intelligent infrastructure for next-generation semantic-oriented image communication systems.

[0064] In some embodiments, the encoder consists of a convolution operation module, a residual stack, and a ReLU activation function; during each encoding, the encoder configures the number of input channels, the number of output channels, and the feature extraction density of the convolutional layer for the input data.

[0065] Specifically, Figure 2This invention provides an encoder device and workflow for an image adaptive quantization communication method, such as... Figure 2 As shown, the encoder consists of a convolution operation module, a residual stack, and a ReLU activation function. During each encoding process, the encoder sets different parameters for the input data, including the number of input channels, the number of output channels, and the feature extraction density of the convolutional layers, but its underlying logic and basic operation remain the same.

[0066] The present invention provides an image adaptive quantization communication method that, by setting different parameters for the input data through an encoder, can flexibly adjust the granularity and depth of feature extraction according to the complexity of the image content and the different feature levels. While effectively capturing deep semantic details of the image, it avoids redundant calculations, thereby significantly improving coding efficiency and system resource utilization while ensuring high-quality semantic representation.

[0067] In some embodiments, the main feature convolutional layer of the convolution operation module uses a 3×3 convolutional kernel; each convolutional layer is followed by a ReLU activation function.

[0068] Specifically, in this embodiment of the invention, the defined convolution operation module uses a 3×3 convolution kernel in its main feature convolution layer to achieve dense extraction of spatial features, significantly reducing the structural complexity of the model while maintaining the accuracy of feature extraction. Furthermore, each convolution layer is followed by a ReLU activation function to effectively suppress the gradient vanishing phenomenon, together forming a complete nonlinear transformation unit.

[0069] The image adaptive quantization communication method provided in this embodiment of the invention forms a highly optimized basic computing structure through the collaborative design of convolution operation modules, laying a key technical foundation for improving the computing efficiency of the entire network while ensuring high-performance output.

[0070] In some embodiments, the residual stack consists of two sequentially connected normalized residual blocks; each residual block contains three convolutional layers with kernel sizes of 1×1, 3×3 and 1×1 respectively, and a ReLU activation function is connected after each convolutional layer.

[0071] Specifically, in this embodiment of the invention, the residual stack is defined as consisting of two sequentially connected normalized residual blocks. Each residual block contains three convolutional layers with kernel sizes of 1×1, 3×3, and 1×1, respectively, and a ReLU activation function is connected after each convolutional layer.

[0072] The present invention provides an image adaptive quantization communication method that constructs a residual stack based on two sequentially connected standardized residual blocks and connects a ReLU activation function after each convolutional layer. While keeping the feature map space size unchanged, it realizes the enhanced extraction and optimization of deep features, effectively ensuring the stability of gradient propagation and the richness of feature representation.

[0073] In some embodiments, the encoder is composed of three convolutional operation modules, two residual stacks, and a ReLU activation function in sequence, wherein the two residual stacks are embedded sequentially between the first two convolutional operation modules; the encoder adopts a hierarchical downsampling architecture, where the first two convolutional layers of the encoder progressively compress the spatial resolution of the input data to 1 / 4 of the original through convolutional operations with a stride of 2, while increasing the channel dimension to 128 dimensions; the last convolutional layer keeps the feature map size unchanged, and the output shape is... .

[0074] Specifically, in this embodiment of the invention, the encoder is defined as consisting of three convolutional operation modules, two residual stacks, and a ReLU activation function arranged sequentially, with the two residual stacks embedded between the first two convolutional operation modules. Simultaneously, the encoder employs a hierarchical downsampling architecture. The first two convolutional layers progressively compress the spatial resolution of the input data to one-quarter of its original size through convolutional operations with a stride of 2, while simultaneously increasing the channel dimension to 128 dimensions. The last convolutional layer maintains the feature map size unchanged, focusing on the extraction and refinement of deep semantic features, with an output shape of... .

[0075] This invention provides an image adaptive quantization communication method. Based on three convolution operation modules, two residual stacks, and a ReLU activation function, an encoder is constructed. Through the collaborative design of hierarchical downsampling convolution and residual connections, the spatial resolution of the image is gradually compressed to 1 / 4 of the original to reduce data redundancy. At the same time, the residual stack is used to enhance the nonlinear expression and refinement of deep semantic features, and the ReLU function is used to effectively suppress the gradient vanishing problem in deep networks. Thus, while ensuring the extremely high compression rate required for low-bandwidth transmission, high-precision and robust extraction of the core semantic information of the image is achieved.

[0076] In some embodiments, the specific steps of the adaptive dynamic quantization mechanism include:

[0077] S1: Feature Projection and Dimensional Adaptation: Semantic Features Extracted by the Encoder Perform a set of parallel two-dimensional convolution operations The semantic features The channel dimension from a unified dimension Projected onto the independent embedding dimensions corresponding to each codebook. Generate a set of feature vectors And perform a dimension flattening operation on the projected feature vectors. Merge spatial dimensions, keeping the last dimension as 1. ;

[0078] S2: Parameter Initialization: Randomly initialize the network parameter key vector Initialize the linear projection matrix with a normal distribution ;

[0079] S3: Dynamic codebook selection: Selecting the semantic features extracted by the encoder. as query vector Input multi-head attention module, based on learnable key vectors A set of codebook selection weights is calculated; these weights are then input into the Gumbel-Softmax function for processing, generating an approximate one-hot distribution. Three-dimensional codebook selection score;

[0080] S4: Nearest Neighbor Quantization and Straight-Through Estimation: For each codebook branch The projected feature vector and the code vector after online linear transformation Perform a nearest neighbor search among them to obtain the code vector index matrix with the minimum Euclidean distance. ,in, The pass-through estimator technique is used to output the quantization results during forward propagation. In backpropagation, the gradient of the original feature is directly transmitted, i.e. ;

[0081] S5: Quantization Dimension Alignment: Align the quantization results of each codebook branch. Transform into dimensional reshaping A three-dimensional matrix is ​​formed, and the channel dimensions are restored to the unified dimension through a convolutional projection layer. ,Right now ;in, Indicates the first i The quantization result of each codebook branch through dimension reshaping and channel dimension alignment after 1×1 convolutional projection;

[0082] S6: Quantization result stitching: ... One recovery to dimension The subsequent codebook features are concatenated along the feature dimensions to form dimensional comprehensive feature matrix ;

[0083] S7: Quantization Result Fusion: Using the codebook, select scores and perform batch matrix multiplication (BMM) on the comprehensive feature matrix. Perform weighted fusion to generate the final result. Quantization characteristics of dimension ;

[0084] S8: Quantization Loss Calculation: During training, the arithmetic mean of the vector quantization losses based on all codebooks is calculated as an additional loss term. Image reconstruction loss Determine the total loss The model is optimized; the vector quantization loss includes codebook loss and commitment loss; the codebook loss drives the vectors in the codebook to converge towards the encoder output features; the commitment loss constrains the encoder output features to converge towards their corresponding nearest neighbor code vectors, i.e. ;in, To reconstruct the image; MSE refers to mean squared error; Weighting parameters for adjusting the quantization loss ratio;

[0085] S9: Convergence Judgment and Update: If the total loss converges, output the reconstructed image and codeword index, and transmit the codeword index to the receiving end; if the total loss does not converge, update the key vector. and linear projection matrix Then, jump to S3 to continue training.

[0086] Specifically, in this embodiment of the invention, the main function of the adaptive dynamic quantization module at the transmitting end is to adaptively map the continuous semantic features output by the encoder into discrete quantized representations through multi-codebook collaboration, learnable linear transformation, and attention-based dynamic selection mechanism. Figure 3 This is an adaptive dynamic quantization flowchart of an image adaptive quantization communication method provided by the present invention, as shown below. Figure 3 As shown, the workflow of this module includes the following steps:

[0087] S1: Feature Projection and Dimensional Adaptation: Semantic Features Extracted by the Encoder First, we enter the feature projection stage.

[0088] Through a set of parallel two-dimensional convolution operations Shared semantic features The channel dimension from a unified dimension Projected onto the respective independent embedding dimensions of each codebook. Generate a set of feature vectors Subsequently, a dimension flattening operation is performed on each projected feature vector. This merges the spatial dimensions while keeping the last dimension unchanged. This prepares two-dimensional structured data for subsequent codebook nearest neighbor search. Among them, This refers to two-dimensional convolution operations; This refers to the dimensional flattening operation.

[0089] S2: Parameter Initialization: Set the number of training rounds Randomly initialize network parameter key vectors Initialize the linear projection matrix with a normal distribution ,in, and These are all network parameters.

[0090] S3: Dynamic codebook selection: Employing a multi-head attention mechanism to select semantic features extracted by the encoder. as query vector Input a multi-head attention module, and learnable key vectors. A set of codebook selection weights were calculated. ,in, Depend on indivual 3D matrix The weights are structured and correspond to their respective codebooks; these weights are then processed by the Gumbel-Softmax function to obtain an approximate one-hot distribution. A three-dimensional codebook selection score is used to enable dynamic codebook configuration decisions based on the semantic content of the input image.

[0091] S4: Nearest Neighbor Quantization and Straight-Through Estimation: For each codebook branch The projected feature vector and the code vector after online linear transformation Perform a nearest neighbor search among them to find the code vector index matrix with the smallest Euclidean distance. ,in, Subsequently, a pass-through estimator technique is employed to output the quantization results during forward propagation. In backpropagation, the quantization operation is bypassed, and the original features are directly transmitted. The gradient, i.e. This ensures that the entire module can undergo end-to-end gradient training.

[0092] S5: Quantization Dimension Alignment: First, quantize the codebook results... Transformed through the view function The three-dimensional matrix is ​​then processed by a convolutional projection layer to reduce the channel dimensions. Restore to the unified dimension ,Right now The `view` function is used to reshape the tensor dimensions while keeping the data content unchanged. This represents the quantization result of the i-th codebook branch after dimensional reshaping and 1×1 convolutional projection to align the channel dimensions.

[0093] S6: Quantization result stitching: ... indivual The codebook after By splicing along the feature dimension, a sequence is formed. dimensional comprehensive feature matrix This establishes a structured relationship between the codebook and spatial dimensions, providing an optimized data foundation for feature fusion.

[0094] S7: Quantization Result Fusion: Then, the score is selected using the codebook generated in step S3. The comprehensive feature matrix is ​​obtained by batch matrix multiplication (BMM). Weighted fusion is performed to generate the final product that incorporates adaptive weights. Quantization characteristics of dimension ;

[0095] S8: Quantization Loss Calculation: During training, the vector quantization loss function for each codebook consists of the codebook loss and the commitment loss. The codebook loss drives the vectors in the codebook to converge towards the encoder's output features, while the commitment loss constrains the encoder's output features to converge towards their corresponding nearest-neighbor code vectors. The arithmetic mean of the quantization losses (vector quantization losses) of all codebooks is used as an additional loss term. Image reconstruction loss The overall optimization objective of the jointly constituted model This joint optimization mechanism effectively improves the utilization efficiency of the codebook space and the representation quality of discretized features. Here, MSE is an abbreviation for Mean Squared Error, representing the mean squared error function. To reconstruct the image. Weighting parameters for adjusting the quantization loss ratio; This indicates that gradient propagation has stopped.

[0096] S9: Convergence Judgment and Update: If the total loss... If convergence is achieved, output the codeword index and transmit it to the receiving end; otherwise, update the key vector. and linear projection matrix Then, jump to S3 to continue training.

[0097] The present invention provides an image adaptive quantization communication method, which, based on an adaptive dynamic quantization module, achieves adaptive matching and dynamic updating between codebook elements and semantic features by introducing a multi-head attention mechanism, a Gumbel-Softmax selection strategy, and a trainable parameter matrix.

[0098] In some embodiments, the receiving end includes an inverse quantization module, a codebook knowledge base, and a decoder; the receiving end uses the trained codebook knowledge base and the inverse quantization module to convert the received codeword index into quantized codewords, and inputs the quantized codewords into the decoder to decode and restore the image.

[0099] Specifically, in this embodiment of the invention, the processing at the receiving end involves an inverse quantization module, a codebook knowledge base, and a decoder. After the codeword index transmitted through the channel is sent to the receiving end, the inverse quantization module uses the trained codebook knowledge base to convert the codeword index into quantized codewords, and then inputs the quantized codewords into the decoder for final decoding and image restoration.

[0100] This invention provides an image adaptive quantization communication method. Based on an inverse quantization module, a codebook knowledge base, and a decoder, the receiving end is constructed. After receiving low-bitrate discrete index data, it can accurately restore deep semantic features using a shared codebook. Through nonlinear decoding mapping and upsampling operations that are inverses of the encoding process, it efficiently reconstructs low-resolution features into a high-fidelity restored image. This significantly reduces communication bandwidth usage while achieving high-quality reconstruction of image semantic content.

[0101] In some embodiments, the decoder consists of two convolutional operation modules, a residual stack module, and a ReLU activation function in sequence; the residual stack module is located in the first layer of the decoder; the feature map is upsampled stepwise through two levels of 4×4 convolutional operations with a stride of 2, and based on the decoder, the input... The quantized feature map is decoded and reconstructed as The output image.

[0102] Specifically, Figure 4 This invention provides a decoder device and workflow for an image adaptive quantization communication method, such as... Figure 4 As shown, in this embodiment of the invention, the decoder consists of two convolution operation modules, a residual stack module, and a ReLU activation function, arranged sequentially. The residual stack module is located in the first layer of the decoder and is used to perform deep feature extraction and nonlinear mapping on the quantized features in the early stages of decoding to improve feature representation and decoding accuracy. Subsequently, the feature map is upsampled step by step through two levels of 4×4 convolution operations with a stride of 2, achieving layer-by-layer restoration of spatial resolution. This structure effectively reduces the channel dimension while maintaining feature continuity, thus improving the input... The quantized feature map is decoded and reconstructed as The output image is then used to obtain an RGB restoration result with the same resolution as the original input.

[0103] This invention provides an image adaptive quantization communication method that constructs a decoder based on two convolution operation modules, a residual stack module, and a ReLU activation function. The residual stack performs deep nonlinear mapping and detail analysis on the quantized features in the early stage of decoding. Combined with two-level upsampling convolution operations, the spatial resolution is gradually restored. This not only effectively reduces the channel dimension but also alleviates the gradient vanishing problem while maintaining feature continuity. Thus, high-precision image semantic reconstruction and pixel-level restoration are achieved after low-bandwidth transmission.

[0104] In some embodiments, the residual stack module in the decoder consists of two residual blocks, each containing a 3×3 kernel convolutional layer and a ReLU activation function.

[0105] Specifically, in this embodiment of the invention, the residual stack module consists of two residual blocks, each containing a 3×3 kernel convolutional layer and a ReLU activation function. The specific structure is as follows: after the input feature map is processed by the first residual block, the output is added to the input to form a residual connection, and then processed by the second residual block to finally output the enhanced feature map.

[0106] The present invention provides an image adaptive quantization communication method, which effectively improves the network's ability to express deep features and alleviates the gradient vanishing problem by configuring a 3×3 kernel convolutional layer and a ReLU activation function for each residual block.

[0107] The system provided by the embodiments of the present invention will be described below. The system described below can be referred to in correspondence with the method described above.

[0108] Figure 5 This is a system architecture diagram of the image adaptive quantization communication system provided by the present invention, as shown below. Figure 5 As shown, the system includes an encoder, an adaptive quantization module, a codebook knowledge base, an inverse quantization module, and a decoder for implementing an image adaptive quantization communication method. The adaptive quantization module is the same as the quantization module.

[0109] The image adaptive quantization communication system provided in this invention adaptively maps continuous deep semantic features into discrete quantized representations through multi-codebook collaboration, learnable linear transformation, and attention-based dynamic selection mechanisms. This achieves adaptive matching and dynamic updating between codebook elements and semantic features. By quantizing the image, transmitting it, and then decoding and restoring it, the efficiency of image transmission is greatly improved. Compared with existing schemes that rely on external pre-trained models or fixed codebook configurations, this invention can automatically optimize the semantic latent space structure during end-to-end training, avoiding codebook representation collapse and significantly enhancing the model's generalization ability and task adaptability. Furthermore, this method can adaptively select the codebook size and embedding dimension based on the semantic content of the input image, thereby achieving optimal allocation of codebook resources while ensuring semantic expression accuracy. In summary, this invention exhibits lower reconstruction errors and stronger robustness in image semantic compression and reconstruction tasks, providing an efficient and intelligent infrastructure for next-generation semantic-oriented image communication systems.

[0110] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device may include a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical commands stored in the memory 630 to execute the methods described in the above embodiments, for example:

[0111] The sending end extracts features from the input image using an encoder to obtain the image's deep semantic features, forming an initial codeword representation; the dimensional structure of the input image is as follows: ,in, Indicates the number of channels. Indicates altitude, The width is represented by the following: The sending end uses an adaptive dynamic quantization mechanism to quantize the initial codeword representation and generates a codeword index as a quantization feature; The adaptive dynamic quantization mechanism refers to adaptively mapping continuous deep semantic features into discrete quantized representations through multi-codebook collaboration, learnable linear transformation, and attention-based dynamic selection mechanism; The sending end sends the codeword index to the receiving end; After obtaining the codeword index, the receiving end performs inverse quantization using a codebook knowledge base and reconstructs the restored image using a decoder.

[0112] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] The processor in the electronic device provided in this embodiment of the invention can call logical instructions in the memory to implement the above method. Its specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, which will not be repeated here.

[0114] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0115] The specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, so it will not be repeated here.

[0116] This invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0117] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image adaptive quantization communication method, characterized in that, include: The sending end extracts features from the input image using an encoder to obtain the image's deep semantic features, forming an initial codeword representation; the dimensional structure of the input image is as follows: ,in, Indicates the number of channels. Indicates altitude, Indicates width; The transmitting end uses an adaptive dynamic quantization mechanism to quantize the initial codeword representation and generate a codeword index as a quantization feature. The adaptive dynamic quantization mechanism refers to the adaptive mapping of continuous deep semantic features into discrete quantization representations through multi-codebook collaboration, learnable linear transformation, and attention-based dynamic selection mechanism. The adaptive dynamic quantization mechanism includes: The channel dimensions of the initial codeword representation are projected onto the embedding dimension size corresponding to multiple codebooks using convolution operations to obtain the corresponding projected feature vectors; The correlation between the initial codeword representation and the learnable key vector is calculated using a multi-head attention mechanism to generate codebook selection weights. The codebook selection weights are then processed by probability distribution to obtain codebook selection scores. A nearest neighbor search is performed between the projected feature vector and the code vectors learned linearly online by each codebook to obtain the codeword index with the smallest distance; Based on the selected scores from the codebooks, the quantization results from different codebooks are convolved, concatenated, and weighted to generate the final quantization features. The sending end sends the codeword index to the receiving end; After obtaining the codeword index, the receiving end performs inverse quantization using the codebook knowledge base and reconstructs the restored image using the decoder.

2. The image adaptive quantization communication method according to claim 1, characterized in that, The encoder consists of a convolution operation module, a residual stack, and a ReLU activation function; during each encoding, the encoder configures the number of input channels, the number of output channels, and the feature extraction density of the convolutional layer for the input data.

3. The image adaptive quantization communication method according to claim 2, characterized in that, The main feature convolutional layer of the convolutional operation module uses a 3×3 convolutional kernel; a ReLU activation function is integrated after each convolutional layer.

4. The image adaptive quantization communication method according to claim 2, characterized in that, The residual stack consists of two sequentially connected normalized residual blocks; each residual block contains three convolutional layers with kernel sizes of 1×1, 3×3 and 1×1 respectively, and a ReLU activation function is connected after each convolutional layer.

5. The image adaptive quantization communication method according to claim 2, characterized in that, The encoder consists of three convolutional operation modules, two residual stacks, and a ReLU activation function arranged sequentially, with the two residual stacks embedded between the first two convolutional operation modules. The encoder employs a hierarchical downsampling architecture. The first two convolutional layers progressively compress the spatial resolution of the input data to one-quarter of its original value through convolution operations with a stride of 2, while simultaneously increasing the channel dimension to 128 dimensions. The last convolutional layer maintains the feature map size unchanged, and the output shape is... .

6. The image adaptive quantization communication method according to claim 1, characterized in that, The specific steps of the adaptive dynamic quantization mechanism include: S1: Feature Projection and Dimensional Adaptation: Semantic Features Extracted by the Encoder Perform a set of parallel 2D convolution operations The semantic features The channel dimension from a unified dimension Projected onto the independent embedding dimensions corresponding to each codebook. Generate a set of feature vectors And perform a dimension flattening operation on the projected feature vectors. The length and width dimensions are merged, while the channel dimension remains unchanged. ; S2: Parameter Initialization: Randomly initialize the network parameter key vector Initialize the linear projection matrix with a normal distribution ; S3: Dynamic codebook selection: Selecting the semantic features extracted by the encoder. as query vector The input is fed into the multi-head attention module and based on the learnable key vectors. A set of codebook selection weights is calculated; these weights are then input into the Gumbel-Softmax function for processing, generating an approximate one-hot distribution. Three-dimensional codebook selection score; S4: Nearest Neighbor Quantization and Straight-Through Estimation: For each codebook branch The projected feature vector and the code vector after online linear transformation Perform a nearest neighbor search among them to obtain the code vector index matrix with the minimum Euclidean distance. ,in, The pass-through estimator technique is used to output the quantization results during forward propagation. In backpropagation, the gradient of the original feature is directly transmitted, i.e. ; S5: Quantization Dimension Alignment: Align the quantization results of each codebook branch. Transform into dimensional reshaping A three-dimensional matrix is ​​formed, and the channel dimensions are restored to the unified dimension through a convolutional projection layer. ,Right now ;in, Indicates the first i The quantization result of each codebook branch through dimension reshaping and channel dimension alignment after 1×1 convolutional projection; S6: Quantization result stitching: ... Each recovery to a dimension The subsequent codebook features are concatenated along the feature dimensions to form dimensional comprehensive feature matrix ; S7: Quantization Result Fusion: Using the codebook, select scores and perform batch matrix multiplication (BMM) on the comprehensive feature matrix. Perform weighted fusion to generate the final result. Quantization characteristics of dimension ; S8: Quantization Loss Calculation: During training, the arithmetic mean of the vector quantization losses based on all codebooks is calculated as an additional loss term. Image reconstruction loss Determine the total loss The model is optimized; the vector quantization loss includes codebook loss and commitment loss, i.e. The codebook loss is used to drive the vectors in the codebook to converge towards the encoder output features; the commitment loss is used to constrain the encoder output features to converge towards their corresponding nearest neighbor code vectors; wherein, To reconstruct the image; MSE refers to mean squared error; Weighting parameters for adjusting the quantization loss ratio; S9: Convergence Judgment and Update: If the total loss converges, output the codeword index and transmit the codeword index to the receiving end; if the total loss does not converge, update the key vector. and linear projection matrix Then, jump to S3 to continue training.

7. The image adaptive quantization communication method according to claim 1, characterized in that, The receiving end includes an inverse quantization module, a codebook knowledge base, and a decoder; the receiving end uses the trained codebook knowledge base and the inverse quantization module to convert the received codeword index into quantized codewords, and inputs the quantized codewords into the decoder to decode and restore the image.

8. The image adaptive quantization communication method according to claim 7, characterized in that, The decoder consists of two convolutional operation modules, a residual stack module, and a ReLU activation function, arranged sequentially. The residual stack module is located in the first layer of the decoder. The feature map is upsampled progressively through two levels of 4×4 convolutional operations with a stride of 2. Based on the decoder, the input... Quantized features are decoded and reconstructed as The output image.

9. The image adaptive quantization communication method according to claim 7, characterized in that, The residual stack module in the decoder consists of two residual blocks, each containing a 3×3 kernel convolutional layer and a ReLU activation function.

10. An image adaptive quantization communication system, characterized in that, It includes an encoder, an adaptive quantization module, a codebook knowledge base, an inverse quantization module, and a decoder for performing an image adaptive quantization communication method according to any one of claims 1 to 9.

Images