A joint source-channel coding image transmission method based on layered adaptive codebook

Abstract

The application discloses a kind of joint source channel coding image transmission methods based on layered adaptive codebook, it is related to wireless communication field.The method constructs image level semantic codebook and feature level quantization codebook, reduces the semantic information quantity to be transmitted by residual coding, and semantic feature is quantized as discrete index and is transmitted by digital modulation.Relevance evaluation and ordering are carried out to code word, and the number of active code word is adaptively selected according to signal-to-noise ratio: a small amount of high importance code word is activated to reduce error code risk when signal-to-noise ratio is low, and all code words are activated to improve quantization accuracy when signal-to-noise ratio is high.The image is reconstructed by index lookup table and residual compensation at the receiving end.The joint optimization of reconstruction loss, codebook entropy regularization, discrete KL divergence loss and structured channel loss is introduced, and the codebook is updated using exponential moving average mechanism to prevent codebook collapse.The application method realizes the dynamic balance of reconstruction quality and robustness in different channel environments through perception, adaptive selection and double codebook cooperation.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and specifically to a joint source-channel coded image transmission method based on a hierarchical adaptive codebook. Background Technology

[0002] In modern wireless communication environments, the rapid growth in demand for wireless image transmission places higher demands on spectral efficiency and channel robustness. Enhancing channel-adaptive image transmission and semantic communication has become a top priority. Currently, wireless image transmission largely relies on traditional discrete architectures, i.e., image compression followed by channel coding. However, under conditions of poor channel quality or limited bandwidth, the transmission efficiency of traditional methods significantly decreases.

[0003] In complex wireless communication environments, robust transmission of high-quality images is of significant research value. Current image transmission methods largely employ traditional architectures based on Shannon's separation theorem, using serially concatenated independent source coding (such as Joint Picture Experts Group (JPEG) and Improved Image Compression Standard (BPG)) and channel coding (such as Low-Density Parity-Check (LDPC) and Turbo codes) to address the transmission task. However, these methods fail to perceive the semantic importance of image content in scenarios with drastically fluctuating wireless channels, leading to significant cliff effects and insufficient adaptability. In recent years, with the development of deep learning, end-to-end image transmission methods based on neural networks have demonstrated powerful nonlinear mapping capabilities and semantic preservation characteristics in complex channel environments, providing a new technical path to overcome the bottlenecks in wireless image transmission.

[0004] Image transmission schemes based on Vector Quantization Variational Autoencoders (VQ-VAE) are a current research direction in the field of semantic communication. The core logic lies in mapping the input image to a discrete codebook index space, using discrete indices instead of continuous features for transmission, simplifying the integration process of quantization and encoding. In the VQ-VAE architecture, after the image's feature representation is extracted by the encoder, it is mapped to a codebook index through nearest neighbor search. The receiver only needs to look up the index and process it through the decoder to reconstruct the image. Compared to traditional separate architectures such as JPEG and H.264, VQ-VAE achieves end-to-end joint optimization, resulting in superior semantic representation performance in low bitrate environments.

[0005] However, existing VQ-VAE-based image transmission schemes face the following technical problems in wireless channel environments: First, fixed codebook size leads to insufficient channel adaptability. Existing schemes use a static codebook structure, which struggles to balance transmission robustness and representation accuracy under time-varying signal-to-noise ratio conditions. When channel quality fluctuates, the fixed-size codebook cannot dynamically adjust the quantization strategy according to the instantaneous channel capacity, resulting in limited rate-distortion performance. Simultaneously, codebook collapse effects prevent a large number of codewords from being effectively activated, further reducing the codebook's representation efficiency. Second, discrete indexing lacks robustness under channel noise. While discrete indexing reduces the amount of transmitted data, the single codebook lookup mechanism lacks fault tolerance when subjected to bit-flipping errors; even small index deviations can lead to severe image distortion. Existing methods do not consider digital modulation bit error patterns during codebook design, and there is no explicit correlation between the codebook's geometric structure and channel error probability. Third, there is a lack of a codeword-level semantic importance perception mechanism. Existing schemes perform equal quantization and transmission on all codewords, failing to establish a hierarchical evaluation system for codeword importance. Under bandwidth-limited or low signal-to-noise ratio conditions, the system cannot identify and prioritize the protection of highly important codewords that contribute significantly to the reconstruction quality, resulting in critical semantic information and non-critical information bearing the same risk of bit errors.

[0006] There is a lack of systematic solutions to the above problems, and there is a lack of optimization methods and deployment schemes that unify channel awareness, semantic evaluation, adaptive quantization and hierarchical transmission into an end-to-end framework. Summary of the Invention

[0007] The problem to be solved by this invention is to provide a joint source-channel coded image transmission method based on hierarchical adaptive codebooks. By introducing multi-dimensional codeword importance assessment, channel adaptive codebook selection, and dual codebook cooperative transmission mechanism into the VQ-VAE architecture, robust representation and reliable transmission of image semantic information are achieved in complex wireless channel environments.

[0008] This invention adopts the following technical solution: a joint source-channel coded image transmission method based on hierarchical adaptive codebook, comprising the following steps:

[0009] S1. Construct a coarse-grained semantic codebook and a fine-grained feature codebook. The coarse-grained semantic codebook is used to represent the global semantic information of the image, and the fine-grained feature codebook is used to represent the detailed information of the image.

[0010] S2. Perform multi-dimensional importance evaluation on each codeword in the fine-grained feature codebook to obtain the importance ranking of the codewords. The multi-dimensional importance includes semantic importance and codeword usage frequency.

[0011] S3. Based on the current channel signal-to-noise ratio, adaptively select the top K codewords with the highest importance scores from the fine-grained feature codebook after importance ranking to form an activation codebook subset, and use an exponential moving average mechanism to update the codebook online to prevent codebook collapse during training.

[0012] S4. Map the semantic features of the input image to codeword indices in the activation codebook subset, and simultaneously obtain the matching index of the input image in the coarse-grained semantic codebook.

[0013] S5. Map the codeword index and the matching index into a bit stream and send it into an additive white Gaussian noise channel for transmission.

[0014] S6. At the receiving end, the corresponding image-level codeword is obtained by looking up the table in the coarse-grained semantic codebook using the matching index; the quantized semantic features are obtained by looking up the table in the fine-grained feature codebook using the codeword index, and the residual image is reconstructed by the semantic decoder; the residual image is added to the image-level codeword to obtain the final reconstructed image.

[0015] Furthermore, the coarse-grained semantic codebook in step S1 is obtained by clustering in the semantic space based on a pre-trained visual-semantic model; the fine-grained feature codebook is obtained by clustering token-level features extracted by the encoder.

[0016] Furthermore, in the multi-dimensional importance evaluation described in step S2, semantic importance is the additional distortion introduced when the features originally quantified by the codeword are forced to be mapped to the next nearest codeword after the codeword is removed; the greater the increase, the greater the contribution of the codeword to maintaining the reconstruction quality, and the higher its semantic importance.

[0017] The frequency of codeword usage is measured by the number of times the codeword is activated on the training set. The more times it is activated, the more versatile the codeword is.

[0018] After normalizing the two-dimensional indicators to the [0,1] interval, a weighted combination is used to obtain the comprehensive importance score of each codeword. :

[0019] ;

[0020] in, For the semantic importance of code words, For the frequency of use of code words, , The weighting coefficients are used to sort the M codewords in the codebook in descending order based on their overall importance scores, thus obtaining the importance ranking mapping. .

[0021] Furthermore, step S3, which involves adaptively selecting the top K codewords with the highest importance scores, is achieved through a monotonically increasing function, such that the number of activated codewords increases as the signal-to-noise ratio increases.

[0022] To maintain compatibility with digital modulation formats, the number of activation codewords K is limited to a power of 2, and the size K of the activation codeword set is determined through a two-step mapping based on the signal-to-noise ratio (SNR):

[0023] The first step of mapping will affect the signal-to-noise ratio. The parameterized Sigmoid function is mapped to a continuous desired bit length. :

[0024] ;

[0025] in, For the Sigmoid function, This is the bit length offset. For the range of bit length variation, This is the signal-to-noise ratio scaling factor, used to control the sensitivity of the number of activation codewords to changes in signal-to-noise ratio; The bias coefficient controls the signal-to-noise ratio range during which the number of active codewords switches in a stepwise manner. Both are preset mapping parameters that can be determined based on index error rate analysis and rate-distortion performance experimental results.

[0026] The second step of mapping discretizes the continuous expected bit length into integer index bit lengths:

[0027] ;

[0028] in, It is a rounding function. Used to limit the integer index bit length to Within the range, Minimum number of bits Given the maximum number of bits, the final number of activated codewords is: .

[0029] This mechanism ensures that the number of activated codewords increases in a stepwise manner as the channel quality improves: when the signal-to-noise ratio is low, K takes a small value, activating only a small number of codewords with high importance to shorten the index bit length and reduce the bit error rate; when the signal-to-noise ratio is high, K gradually increases until the entire codebook is activated to improve quantization accuracy and reconstruction fidelity.

[0030] Furthermore, to prevent codebook collapse during training, an exponential moving average (EMA) mechanism is used to update codebook parameters online during the training phase.

[0031] Specifically, the EMA statistics for codeword cluster size and embedding vector sum are maintained. After each training batch, the statistics are updated based on the usage of the current batch, thereby updating the codebook vector. This mechanism achieves stable codebook learning through the smooth accumulation of historical information, avoiding codebook collapse, and does not require the introduction of an additional commitment loss term.

[0032] Specifically, the EMA attenuation factor is set to 0.99.

[0033] Further, the codeword index described in step S4 is obtained by calculating the Euclidean distance between the feature vector and the codeword in the activation codebook subset using the nearest neighbor search algorithm, thus obtaining the local index (value range) corresponding to each token. to );

[0034] The matching index uses the contrastive language-image pre-trained model (CLIP) to calculate the semantic similarity between the input image and each image in the coarse-grained semantic codebook, and then matches them to obtain the image index with the highest similarity.

[0035] Furthermore, the channel transmission of the bit stream in step S5 adopts binary phase shift keying (BPSK) modulation and additive white Gaussian noise (AWGN) channel model.

[0036] The sending end maps the index to a binary bit stream, each bit The signal is generated by binary phase shift keying (BPSK). The received signal after transmission through the channel is ;in, It is additive white Gaussian noise that follows a normal distribution. The receiving end uses hard decision... Recover the bit sequence; the theoretical bit error rate is .

[0037] Here, erfc is the complementary error function, which serves as a dynamic weight to automatically enhance the structured loss at low signal-to-noise ratios. For linear signal-to-noise ratio, The signal-to-noise ratio is expressed in decibels.

[0038] Furthermore, the final reconstructed image described in step S6 is obtained through residual compensation. The reconstructed residual image output by the semantic decoder is directly added to the image-level codeword to obtain:

[0039] ;

[0040] in, The reconstructed residual image output by the semantic decoder. The image-level codewords are obtained by the receiving end from the coarse-grained semantic codebook based on the matching index. The final reconstructed image obtained.

[0041] Furthermore, the training of the method employs end-to-end optimization using a multi-objective joint loss function. Represented as:

[0042] ;

[0043] in, To assess the distortion loss during reconstruction, mean squared error (MSE) is used to measure the reconstructed image. With the original image The pixel-level difference is calculated using the following formula: This ensures that the reconstructed image is close in quality to the original image.

[0044] To structure the channel loss, the Euclidean distance in the feature space of codeword pairs with a Hamming distance of 1 is constrained to align the codebook geometry with the bit-flipping error pattern of the binary phase-shift keying channel; the calculation formula is as follows: , This is the set of codeword indices that are activated (selected as nearest neighbor quantization results) in the current batch. The number of activation codewords, Represented as the first The index of each codeword, The number of binary bits for each index, where M is the codebook size. For typing The Hamming neighbor set, i.e., the binary representation of AND All indices that differ by only 1 bit The set, Represented as Hamming distance; For typing and The corresponding D-dimensional codebook vector in the feature space; It is the squared Euclidean distance between the two codebook vectors.

[0045] These are the weighting coefficients for the structured loss. This represents the theoretical bit error rate corresponding to the current signal-to-noise ratio. This represents the theoretical bit error rate (BER) of BPSK corresponding to the current signal-to-noise ratio.

[0046] The discrete KL divergence loss is used, and the KL divergence term aims to guide the quantization index distribution to approximate a pre-defined prior distribution. The calculation formula is: , For the first The quantization index of each tag, where N is the total number of quantization indexes in the current batch; is the KL divergence coefficient.

[0047] The prior distribution By assigning a comprehensive importance score to each codeword Normalization is performed to obtain high-importance codewords, which are assigned higher prior probabilities. This reduces their encoding cost in the loss function, guiding the model to prioritize the activation of high-quality codewords during the quantization phase.

[0048] The codebook entropy regularization loss is used to prevent codebook collapse by maximizing the entropy value of the codeword usage probability distribution in the current batch. The calculation formula is as follows: ,in, For typing Frequency of use in the current batch is the entropy regularization weighting coefficient.

[0049] Furthermore, the channel transmission employs BPSK modulation and an AWGN channel model for equivalent modeling. During the training phase, the method calculates the theoretical bit error rate based on channel state information. ,in, It is a Gaussian Q-function. Signal-to-noise ratio;

[0050] A bit-level flip perturbation simulation is performed on the binary bit stream obtained by quantization index mapping. This process enables the codec to perceive physical layer noise during the training phase by explicitly simulating the transmission of index errors in the digital link, thereby learning a semantic representation with error adaptive capability.

[0051] Furthermore, during training, the reconstruction quality metrics monitored include peak signal-to-noise ratio (PSNR) and multi-scale structural similarity (SSIM), while the channel robustness metrics include bit error rate (BER) and index error rate (IER).

[0052] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0053] 1. This invention introduces multi-dimensional codeword importance assessment, channel-adaptive codebook selection, and dual-codebook cooperative transmission mechanism into the VQ-VAE architecture, thereby achieving robust representation and reliable transmission of image semantic information in complex wireless channel environments.

[0054] 2. This invention can dynamically adjust the codebook activation strategy according to the real-time channel quality, and significantly improve the image reconstruction quality in low-noise environments while ensuring the integrity of the basic semantics. Attached Figure Description

[0055] Figure 1 This is a flowchart of the joint source-channel coding image transmission method of the present invention;

[0056] Figure 2 This is a schematic diagram of the improved structure based on the VQ-VAE method in an example of the present invention;

[0057] Figure 3 This is a performance comparison chart of the method of this invention with other image transmission methods under varying channel conditions.

[0058] Figure 4 This is a schematic diagram showing the ablation experiment results of the method of the present invention and other image transmission methods under variable channel conditions. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on these embodiments by other researchers in the art are within the protection scope of this invention. Furthermore, the step numbers in the embodiments of this invention are only set for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0060] In one embodiment of the present invention, a joint source-channel coded image transmission method based on a hierarchical adaptive codebook is provided, the process of which is as follows: Figure 1 As shown, it includes:

[0061] Construct a coarse-grained semantic codebook and a fine-grained feature codebook. The coarse-grained semantic codebook is used to represent the global semantic information of the image, and the fine-grained feature codebook is used to represent the detailed information of the image.

[0062] A multi-dimensional importance evaluation is performed on each codeword in the fine-grained feature codebook to obtain the importance ranking of the codewords. The multi-dimensional importance includes semantic importance and codeword usage frequency.

[0063] Based on the current channel signal-to-noise ratio, the top K codewords with the highest importance scores are adaptively selected from the fine-grained feature codebook after importance sorting to form a subset of the activation codebook;

[0064] The semantic features of the input image are mapped to codeword indices in the activation codebook subset, and the matching index of the input image in the coarse-grained semantic codebook is obtained simultaneously.

[0065] The codeword index and matching index are mapped into a bit stream and sent into an additive white Gaussian noise channel for transmission;

[0066] At the receiving end, the corresponding image-level codewords are obtained by looking up a table in the coarse-grained semantic codebook using the matching index, and the basic reconstructed image is obtained; the quantized semantic features are obtained by looking up a table in the fine-grained feature codebook using the codeword index, and the residual image is reconstructed by the semantic decoder.

[0067] The residual image is added to the image-level codeword to obtain the final reconstructed image.

[0068] Specifically, the experiment in this embodiment was conducted on a single NVIDIA GTX 5060Ti graphics card, implemented using the Python language and the PyTorch deep learning framework. The experiment used the STL-10 dataset, with all images uniformly scaled to 3×256×256. The encoder and decoder adopted a four-stage Swin Transformer structure, with a bottleneck dimension C=32. After four downsampling operations, the encoder output L=256 semantic tokens.

[0069] In this embodiment, the image transmission system includes a transmitter and a receiver, such as... Figure 2 As shown, the two share a set of two-layer codebooks as a semantic knowledge base.

[0070] First, a two-layer codebook is constructed, consisting of an image-level semantic codebook and a feature-level quantization codebook. The image-level semantic codebook comprises N representative images. Specifically, a pre-trained visual-semantic model is used to extract global semantic features from each image in the training set. Then, K-means clustering is performed in the global semantic feature space. For each cluster center, the training image closest to that cluster center or the image-level template generated corresponding to that cluster center is selected as the image-level codeword. Each image-level codeword is a full-size image (3×H×W) used to represent the global semantic category and scene layout, and is used as the base reconstructed image for residual compensation at the receiving end. The value of N ranges from 10 to 50. The feature-level quantization codebook consists of M D-dimensional vectors, obtained by performing MiniBatch K-means clustering on all token features output by the semantic encoder. The cluster centers serve as the initial values ​​for the codewords and are registered as learnable parameters during the training phase to participate in end-to-end optimization. The value of M ranges from 128 to 256, and the value of D ranges from 16 to 64.

[0071] After the codebook is constructed, the system performs a multi-dimensional importance evaluation of each codeword in the feature-level codebook. Semantic importance is measured by calculating the increase in reconstruction loss after removing the codeword; a larger increase indicates a greater contribution of the codeword to the reconstruction quality. Usage frequency is obtained by counting the cumulative number of times the codeword is selected as a nearest neighbor during training; a higher frequency indicates a wider data distribution covered by the codeword.

[0072] After performing Min-Max normalization on the two dimensions respectively, a weighted combination is obtained to obtain the comprehensive importance score. Based on this, the M codewords are sorted in descending order to form an importance ranking map. .

[0073] Then, during the transmission phase, the transmitter determines the number of activation codewords K based on the current channel signal-to-noise ratio (SNR). Specifically, to maintain compatibility with the digital modulation format, this embodiment limits the number of activation codewords to a power of 2 and restricts the integer index bit length to between 6 and 8 bits. The transmitter first determines the current channel SNR... Mapping to a continuous expected bit length using a parameterized Sigmoid function:

[0074] ;

[0075] Then, the continuous expected bit length is discretized into integer index bit length:

[0076] ;

[0077] in, It is a rounding function. Used to limit the integer index bit length to Within the range, the final number of activated codewords is: .

[0078] The sending end selects the top K codewords from the sorted feature codebook to form an activation subset. It then performs nearest-neighbor quantization on the L semantic tokens output by the encoder within this subset, obtaining L discrete indices. Simultaneously, the sending end matches the input image with the image-level codebook to obtain the index of the nearest semantic template. These feature indices and template indices together constitute all the semantic information to be transmitted.

[0079] The indexes, after being binary mapped, are transmitted in an AWGN channel using BPSK modulation. The transmitting end converts each index into... Bits of binary data, mapped to The signal is transmitted afterward.

[0080] The received signal after transmission through the channel is ,in, The receiving end uses hard decision The bit sequence is recovered and restored to a discrete index. The current activation level is notified to the receiver via a control field in the frame header, the overhead of which is negligible relative to the data payload.

[0081] At the receiver, the decoding process comprises two parallel branches. The coarse-grained branch uses the recovered template index to directly look up a table in the pre-shared image-level codebook to obtain the underlying reconstructed image containing global semantic information. The fine-grained branch uses the recovered feature index sequence to find the corresponding codeword in the feature-level codebook, reassembles it into quantized semantic features, and then reconstructs the residual through the SwinTransformer decoder. The final reconstructed image is obtained through residual compensation: ,in, Reconstructing images based on data, To reconstruct the residual.

[0082] Furthermore, the training of the system in this embodiment employs end-to-end multi-objective joint optimization. The total loss function is defined as:

[0083] ;

[0084] The meanings of each item are as follows: To reconstruct distortion loss, mean squared error is used to measure the pixel-level difference between the reconstructed image and the original image;

[0085] For the discrete KL divergence loss, calculate the negative logarithmic prior probability of selecting a specific codeword using the quantization index:

[0086] ;

[0087] in, For the first The quantization index of each token is obtained by performing Min-Max normalization on the comprehensive importance score of each codeword. High-importance codewords are assigned higher prior probabilities, thereby guiding the model to activate high-importance codewords first. is the KL divergence coefficient.

[0088] To reduce codebook entropy regularization loss, the encoder is encouraged to use codewords evenly, preventing some codewords from being used too infrequently.

[0089] ;

[0090] in, For typing Frequency of use in the current batch; is the entropy regularization weighting coefficient.

[0091] To address the structured channel loss, for each codeword appearing in the current batch, calculate the squared Euclidean distances in the feature space between it and its neighboring codewords with a Hamming distance of 1, and then take the average:

[0092] ;

[0093] in, This is the set of codeword indices that are activated (selected as nearest neighbor quantization results) in the current batch. The number of activation codewords, Represented as the first The index of each codeword, The number of binary bits for each index, where M is the codebook size; For typing The Hamming neighbor set, i.e., the binary representation of AND All indices that differ by only 1 bit The set, Represented as Hamming distance; For typing and The corresponding D-dimensional codebook vector in the feature space; It is the squared Euclidean distance between the two codebook vectors.

[0094] In this loss, the inner summation traversal codewords All Hamming neighbors and the number of neighbors Normalization involves averaging the outer summation over all activation codewords within the batch. These are the weighting coefficients for the structured loss; The theoretical bit error rate (BER) of BPSK corresponding to the current signal-to-noise ratio is... It is a complementary error function, which acts as a dynamic weight to automatically enhance the structured loss at low signal-to-noise ratios.

[0095] During the training phase, the feature-level codebook is updated online using an exponential moving average (EMA) mechanism with a decay coefficient set to 0.99. In this embodiment, the system monitors the frequency of codeword usage. When the cumulative activation count of a codeword falls below one-tenth of the average, it is reset to a randomly selected encoder output feature from the current batch to prevent codebook collapse. Training uses the Adam optimizer with an initial learning rate of 3×10⁻⁻⁶. 4 The training signal-to-noise ratio is randomly sampled from a predefined set. After training, the model with the best overall performance on the validation set is retained.

[0096] Experiments were conducted on the STL-10 dataset, with images scaled to 256×256 resolution. The encoder and decoder employed a four-stage Swin Transformer architecture with a bottleneck dimension of 32, and the encoder output 256 semantic tokens. The image-level codebook contained 32 representative images, and the feature-level codebook contained 256 32-dimensional codewords, initialized using MiniBatch K-means and updated online during training using an exponential moving average.

[0097] Training was performed using the Adam optimizer with an initial learning rate of 3 × 10⁻ 4 The batch size is 12, and the training signal-to-noise ratio (SNR) is randomly sampled from -2, 1, 4, 7, 10, and 13 dB. The channel uses an AWGN model with BPSK modulation, and the receiver performs hard-decision demodulation. The weights for semantic importance and codeword usage frequency in the codeword importance evaluation are set to 0.3 and 0.7, respectively. The adaptive strategy employs a codebook activation mechanism based on Sigmoid continuous mapping, floor-out discretization, and range pruning. In this embodiment, the integer index bit length B is limited to between 6 and 8 bits, corresponding to the number of activated codewords K=64, K=128, and K=256. When the SNR is low, the system tends to select smaller B and K to reduce the index bit length and error risk; when the SNR is high, the system selects larger B and K to improve quantization accuracy and reconstruction quality. Table 1 below shows the specific mapping logic and performance indicators of the system performing adaptive codebook selection under different channel conditions, listing several representative SNRs and their assigned number of codewords, BER, IER, CBR, and CR indicators.

[0098] Table 1: Channel-Aware Codebook Adaptive Selection Strategy and Parameter Mapping Table

[0099]

[0100] It is important to note that, apart from the system structure parameters which are static, all other metrics in Table 1 are dynamic and change with the SNR of each batch. The table shows the deterministic mapping relationship corresponding to each SNR.

[0101] As the signal-to-noise ratio (SNR) decreases from 10 dB to -2 dB, the system automatically reduces the number of activation codewords from 256 to 64, corresponding to a reduction in the number of index bits from 8 bits to 6 bits. Compared to a scheme that uses all 256 codewords (8 bits), the adaptive strategy reduces the index error rate by 15.4% at SNR=-2 dB and by 11.3% at SNR=1 dB. Simultaneously, the channel bandwidth ratio decreases with the reduction in the number of activation codewords, and the compression ratio increases from 766:1 to 1021:1, indicating that the system achieves more reliable index transmission with less transmission overhead under low SNR conditions. When the SNR is above 4 dB, the index error rate drops below 10%, and the system activates all codewords to fully utilize channel capacity and improve quantization accuracy.

[0102] Experiments conducted in a single GTX 5060ti hardware environment and a Python software environment demonstrate the test results of the joint source-channel coding image transmission method based on hierarchical adaptive codebooks. Figure 3 and Figure 4 As shown.

[0103] from Figure 3 As can be seen, this method achieves superior performance compared to traditional methods in environments with low signal-to-noise ratios. Furthermore, ablation experiments also validate the effectiveness of the channel-adaptive method, demonstrating more flexible codeword and bandwidth usage and avoiding unnecessary bit error rates. Figure 4 The results of comparative experiments on the SSIM (structural similarity) index are presented.

[0104] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A joint source-channel coded image transmission method based on hierarchical adaptive codebook, characterized in that, Includes the following steps: S1. Construct a coarse-grained semantic codebook and a fine-grained feature codebook. The coarse-grained semantic codebook is used to represent the global semantic information of the image, and the fine-grained feature codebook is used to represent the detailed information of the image. S2. Perform multi-dimensional importance evaluation on each codeword in the fine-grained feature codebook to obtain the importance ranking of the codewords. The multi-dimensional importance includes semantic importance and codeword usage frequency. S3. Based on the current channel signal-to-noise ratio, adaptively select the top K codewords with the highest importance scores from the fine-grained feature codebook after importance ranking to form an activation codebook subset, and use an exponential moving average mechanism to update the codebook online to prevent codebook collapse during training. S4. Map the semantic features of the input image to codeword indices in the activation codebook subset, and simultaneously obtain the matching index of the input image in the coarse-grained semantic codebook. S5. Map the codeword index and the matching index into a bit stream and send it into an additive white Gaussian noise channel for transmission. S6. At the receiving end, the corresponding image-level codeword is obtained by looking up the table in the coarse-grained semantic codebook using the matching index; the quantized semantic features are obtained by looking up the table in the fine-grained feature codebook using the codeword index, and the residual image is reconstructed by the semantic decoder. The residual image is added to the image-level codeword to obtain the final reconstructed image.

2. The joint source-channel coded image transmission method according to claim 1, characterized in that, The coarse-grained semantic codebook described in step S1 is obtained by clustering in the semantic space based on a pre-trained visual-semantic model. The fine-grained feature codebook is obtained by clustering token-level features extracted by the encoder.

3. The joint source-channel coded image transmission method according to claim 1, characterized in that, In the multi-dimensional importance evaluation described in step S2, semantic importance is the additional distortion introduced when the features originally quantified by the codeword are forced to be mapped to the next nearest codeword after the codeword is removed; the codeword usage frequency is measured by the number of activations of the codeword on the training set, and the more activations, the stronger the generality of the codeword.

4. The joint source-channel coded image transmission method according to claim 1, characterized in that, Step S2, which involves obtaining the importance ranking of codewords, includes: The semantic importance and codeword usage frequency indices are normalized, and a weighted combination is used to obtain the comprehensive importance score for each codeword. : ； in, For the semantic importance of code words, For the frequency of use of code words, , These are the weighting coefficients; The M codewords in the codebook are sorted in descending order based on their overall importance scores to obtain the importance ranking map. .

5. The joint source-channel coded image transmission method according to claim 1, characterized in that, Step S3 describes the adaptive selection of the top K codewords with the highest importance scores, which is achieved through a monotonically increasing function, such that the number of activated codewords increases as the signal-to-noise ratio increases. The size K of the activation code character set is determined through a two-step mapping based on the signal-to-noise ratio: The first step of mapping will affect the signal-to-noise ratio. The parameterized Sigmoid function is mapped to a continuous desired bit length. : ； in, For the Sigmoid function, This is the bit length offset. For the range of bit length variation, This is the signal-to-noise ratio scaling factor, used to control the sensitivity of the number of activation codewords to changes in signal-to-noise ratio; The bias coefficient is used to control the signal-to-noise ratio range in which the number of active codewords switches in a stepwise manner; The second step of mapping discretizes the continuous expected bit length into integer indexed bit lengths. : ； in, It is a rounding function. Used to limit the integer index bit length to Within the range, Minimum number of bits The maximum number of bits, and the final number of activated codewords is .

6. The codeword index mentioned in step S4 is obtained by calculating the Euclidean distance between the feature vector and the codeword in the activation codebook subset using the nearest neighbor search algorithm, and the local index corresponding to each token is obtained, with a value range of 0 to K-1; The matching index uses a contrastive language-image pre-trained model to calculate the semantic similarity between the input image and each image in the coarse-grained semantic codebook, and then matches them to obtain the image index with the highest similarity.

7. The joint source-channel coded image transmission method according to claim 1, characterized in that, The channel transmission of the bit stream in step S5 adopts a binary phase shift keying modulation and additive white Gaussian noise channel model: The sending end maps the index to a binary bit stream, each bit Signal modulated by binary phase shift keying The received signal after transmission through the channel is ;in, It is additive white Gaussian noise that follows a normal distribution. ; The receiving end uses hard decision Recover the bit sequence; the theoretical bit error rate is ; Among them, erfc is the complementary error function, which serves as a dynamic weight to automatically enhance the structured loss at low signal-to-noise ratios. For linear signal-to-noise ratio, The signal-to-noise ratio is expressed in decibels.

8. The joint source-channel coded image transmission method according to claim 1, characterized in that, The final reconstructed image described in step S6 is obtained through residual compensation. The reconstructed residual image output by the semantic decoder is directly added to the image-level codeword to obtain: ； in, The reconstructed residual image output by the semantic decoder. The image-level codewords are obtained by the receiving end from the coarse-grained semantic codebook based on the matching index. The final reconstructed image obtained.

9. The joint source-channel coded image transmission method according to claim 1, characterized in that, The training method employs a multi-objective loss function, expressed as: ； in, To reconstruct distortion loss, mean squared error is used to measure the pixel-level difference between the reconstructed image and the original image; To structure the channel loss, the codebook geometry is aligned with the bit-flipping error pattern of the binary phase shift keying channel by constraining the Euclidean distance of codeword pairs with a Hamming distance of 1 in the feature space. These are the weighting coefficients for the structured loss; This represents the theoretical bit error rate corresponding to the current signal-to-noise ratio. For discrete KL divergence loss, calculate the negative logarithmic prior probability of selecting a specific codeword by the quantization index; The KL divergence coefficient; To reduce codebook entropy regularization loss, the encoder is encouraged to use each codeword evenly, preventing some codewords from being used too infrequently. is the entropy regularization weighting coefficient.

10. The joint source-channel coded image transmission method according to claim 9, characterized in that, The method calculates the theoretical bit error rate based on channel state information during the training phase. ,in, It is a Gaussian Q-function. Signal-to-noise ratio; Bit-level flip perturbation simulation is performed on the binary bit stream obtained by quantization index mapping; the reconstructed quality indicators monitored include peak signal-to-noise ratio and multi-scale structural similarity, and the channel robustness indicators include bit error rate (BER) and index error rate (IER).

Images