Image transmission method of joint source channel coding based on hybrid retransmission

Abstract

The invention relates to the technical field of wireless communication, and discloses an image transmission method for joint source channel coding based on hybrid retransmission, which comprises the following steps: acquiring an original image to generate an initial feature representation and transmitting the initial feature representation; decoding to generate a preliminary reconstructed image; calculating distortion measure to judge whether retransmission is triggered or not; after triggering retransmission, generating an activation feature group number by using a policy network; modulating the intermediate characteristics of the encoder through a signal-to-noise ratio adaptive module; generating supplementary transmission feature representation according to the activation feature group number; and fusing the preliminary reconstruction information and the supplementary transmission features to generate a final reconstruction image. According to the invention, through channel adaptive modulation, intelligent rate decision and semantic increment retransmission, the technical problems of coding strategy solidification, lack of dynamic adjustment of transmission rate and unintelligent retransmission strategy are solved.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and more specifically, to an image transmission method based on joint source-channel coding with hybrid retransmission. Background Technology

[0002] In the field of wireless communication, image transmission faces multiple challenges, including highly time-varying channel conditions, limited transmission resources, and unstable reconstruction quality. Traditional joint source-channel coding techniques employ fixed coding strategies and transmission rates, making it difficult to adapt to dynamically changing channel environments.

[0003] Existing technologies primarily employ joint source-channel coding and hybrid automatic repeat request (ARPC) mechanisms for image transmission. While joint source-channel coding can optimize end-to-end image transmission, its fixed coding strategy makes it unable to adjust the robustness of feature representations in a timely manner when channel quality deteriorates, leading to a significant drop in reconstruction quality. Conversely, when channel quality is good, the fixed high-redundancy coding wastes bandwidth resources. Although the hybrid APC mechanism introduces feedback and retransmission mechanisms, its feedback process incurs additional latency overhead, and data loss may still occur after exceeding the maximum number of retransmissions, failing to effectively guarantee the bit error rate under extreme channel conditions.

[0004] The main technical problems with existing technologies are: insufficient adaptability of encoders to channel conditions, resulting in significant differences in the anti-interference performance of feature representations under different signal-to-noise ratios; lack of dynamic adjustment mechanism for transmission rate, making it impossible to achieve coordinated optimization of resources and quality based on image content complexity and channel quality; and insufficient intelligence in retransmission strategies, leading to redundancy in retransmission content and failure to fully utilize the semantic information acquired in the initial transmission. These problems result in wireless image transmission systems facing difficulties in practical applications, including unstable reconstruction quality, low spectral efficiency, and insufficient robustness under multipath fading channels. Summary of the Invention

[0005] This invention provides an image transmission method based on joint source-channel coding with hybrid retransmission, which solves the technical problems of fixed coding strategies, lack of dynamic adjustment of transmission rate, and unintelligent retransmission strategies in related technologies.

[0006] This invention provides an image transmission method based on joint source-channel coding with hybrid retransmission, comprising the following steps: The original image is acquired and input into the joint source-channel encoder to generate an initial feature representation; The initial feature representation is transmitted through a physical channel, and the receiving end obtains the received signal with noise. The receiving end inputs the noisy received signal into the decoder to generate a preliminary reconstructed image; Calculate the distortion measure between the initially reconstructed image and the original image, and determine whether the distortion measure is greater than a preset distortion threshold. If it is greater than the preset distortion threshold, trigger the retransmission mechanism. After the retransmission mechanism is triggered, the sending end extracts the content features of the original image, obtains the signal-to-noise ratio parameter of the current channel, concatenates the content features and the signal-to-noise ratio parameter to generate a joint state vector, inputs the joint state vector into the policy network, the policy network outputs the probability distribution of the number of activated feature groups, and samples the probability distribution to generate a discrete number of activated feature groups. A signal-to-noise ratio (SNR) adaptive module is inserted between the residual network layers of the joint source-channel encoder. The SNR adaptive module obtains the feature map of the intermediate layer of the encoder, performs global average pooling on each channel of the feature map to obtain a channel-level feature vector, concatenates the channel-level feature vector with the current channel SNR to obtain a fusion vector, inputs the fusion vector into a fully connected network to generate a channel-level scaling factor and a channel-level offset, and performs channel-level modulation on the feature map to generate an adaptive feature representation. The encoder encodes the original image to generate a complete feature representation, selectively activates the complete feature representation according to the number of activated feature groups, generates a supplementary feature representation, and the transmitter transmits the supplementary feature representation to the receiver through a physical channel. The receiving end acquires the supplementary feature representation, fuses the intermediate feature representation corresponding to the preliminary reconstructed image with the supplementary feature representation, and the decoder processes the fused features to generate the final reconstructed image. The policy network outputs the joint state vector by inputting it into a two-layer fully connected neural network. An activation function is applied to the output of the first fully connected layer, and a normalized activation function is applied to the output of the second fully connected layer to generate the probability distribution. The sampling of the probability distribution adopts a differentiable sampling technique. In the forward propagation stage, a discrete vector is obtained by selecting the maximum value. In the backpropagation stage, a continuous approximation is used to achieve gradient backpropagation, and the discrete vector is converted into a thermometer-coded mask. The thermometer-coded mask ensures that the activated feature groups are selected continuously starting from index zero.

[0007] Furthermore, the distortion measurement is calculated using mean square error, and the formula for calculating the distortion measurement is the sum of the squares of the differences between the corresponding pixel values ​​of the original image and the preliminary reconstructed image divided by the total number of pixels in the image.

[0008] Furthermore, the input of the policy network also includes distortion information of the preliminary reconstructed image, which is defined as the mean square error between the preliminary reconstructed image and the original image. The policy network learns a mapping relationship from the joint state vector to the number of activated feature groups, and the mapping relationship satisfies a dynamic adjustment strategy that the greater the distortion, the more activated feature groups there are.

[0009] Furthermore, the generation process of the channel-level scaling factor and the channel-level offset is as follows: The fusion vector is input into the first fully connected layer, and the first fully connected layer performs dimensionality reduction output according to the dimensionality reduction ratio. Apply an activation function to the output of the first fully connected layer; The activated output is input into two parallel fully connected layers to generate the channel-level scaling factor and the channel-level offset, respectively. A limit activation function is applied to the channel-level scaling factor to ensure that its value is between zero and one.

[0010] Furthermore, the number of activated feature groups is directly proportional to the actual number of transmitted bits. When the channel quality is good, the strategy network selects a smaller number of activated feature groups to reduce the number of transmitted bits, and when the channel quality is poor, the strategy network selects a larger number of activated feature groups to increase the number of transmitted bits.

[0011] Furthermore, the receiving end does not require additional control information for the recognition process of the supplementary feature representation. The receiving end automatically determines the number of activated feature groups by detecting the distribution pattern of non-zero elements in the feature vector.

[0012] Furthermore, the feature fusion process includes: The decoder passes the initially received signal through a partial decoding layer to generate intermediate feature representations; Decode the non-zero feature groups in the supplementary feature representation to generate a supplementary feature representation; The intermediate feature representation and the supplementary feature representation are fused together by feature concatenation to generate a fused feature; The fused features are input into subsequent layers of the decoder to generate the final reconstructed image.

[0013] Furthermore, it also includes a balance factor prediction network. The input of the balance factor prediction network includes the content features of the original image, the current channel signal-to-noise ratio, and the historical transmission state vector. The historical transmission state vector contains the peak signal-to-noise ratio and bit rate information in the previous transmission rounds. The balance factor prediction network outputs a dynamic balance factor. During training, an overall loss function containing the dynamic balance factor is used. The overall loss function includes the mean squared error term of image reconstruction and the bandwidth regularization term of the activated feature group. The dynamic balance factor is dynamically adjusted according to the current image content, channel state, and historical transmission conditions.

[0014] Furthermore, an attention mechanism is introduced into the decoder, which assigns different weights to different spatial locations of the preliminary reconstructed features and the supplementary features. The decoder calculates a spatial attention map of the preliminary reconstructed features and the supplementary features, and weights the supplementary features based on the spatial attention map. The weighted supplementary features are then fused with the preliminary reconstructed features to generate the final fused features, which are then input into the subsequent decoding layer.

[0015] This invention provides an image transmission system based on hybrid retransmission joint source channel coding, for performing the aforementioned image transmission method based on hybrid retransmission joint source channel coding, comprising: The encoding module is used to acquire the original image, input the original image into the joint source-channel encoder, and generate an initial feature representation; A channel transmission module is used to transmit the initial feature representation through a physical channel; The preliminary decoding module is used to input the noisy received signal into the decoder to generate a preliminary reconstructed image; The distortion assessment module is used to calculate the distortion measure between the preliminary reconstructed image and the original image, and to determine whether to trigger the retransmission mechanism. The strategy decision module is used to extract the content features of the original image and the signal-to-noise ratio parameter of the current channel after the retransmission mechanism is triggered, generate a joint state vector, and output the probability distribution of the number of activated feature groups through the policy network to generate a discrete number of activated feature groups. The adaptive modulation module is used to perform channel-level modulation on the feature map of the encoder's intermediate layer through the signal-to-noise ratio adaptive module to generate an adaptive feature representation. The supplementary transmission generation module is used to selectively activate the complete feature representation according to the number of activated feature groups, generate a supplementary feature representation, and transmit it. The feature fusion module is used to fuse the intermediate feature representations corresponding to the preliminary reconstructed image with the supplementary feature representations to generate the final reconstructed image.

[0016] The beneficial effects of this invention are as follows: By introducing a signal-to-noise ratio adaptive module, a policy network-driven rate control method, and a semantic incremental retransmission strategy, this invention solves the technical problems of fixed coding strategies, lack of dynamic adjustment of transmission rates, and unintelligent retransmission strategies in wireless image transmission, achieving the following technical effects: First, the signal-to-noise ratio adaptive module performs channel-level scaling and offset modulation on the intermediate features of the encoder, enabling the feature distribution to be dynamically adjusted with the signal-to-noise ratio. This adaptively enhances robustness under different channel types, such as additive white Gaussian noise channels and Rayleigh fading channels, significantly reduces the bit error rate, and improves the quality of the initial reconstructed image. Second, the policy network-driven rate control method dynamically adjusts the number of activated feature groups based on image content features, channel signal-to-noise ratio, and distortion information from the initial reconstruction, achieving adaptive control of the transmitted bit volume. This reduces the amount of transmitted data for images with simple textures or good channel quality, while providing richer semantic information for images with complex textures or poor channel quality, effectively reducing average channel usage while ensuring reconstruction quality. Third, the semantic incremental retransmission strategy dynamically determines the number of incremental semantic feature groups that need to be retransmitted based on the degree of distortion of the initial reconstruction. The retransmitted content only contains incremental semantic information, avoiding the retransmission of successfully received parts. Compared with the traditional method of retransmitting the entire data packet, it significantly reduces retransmission overhead, reduces the number of retransmissions, and improves the retransmission success rate. Fourth, the balance factor prediction network dynamically adjusts the trade-off coefficient between reconstruction quality and bandwidth overhead based on image content characteristics, channel status, and historical transmission conditions, achieving scene-adaptive multi-objective optimization. This enables the system to flexibly adjust performance indicators under different application requirements, enhancing the practicality and generalization ability of the method. In summary, this invention constructs a closed-loop system of channel adaptive feature modulation, differentiable rate decision-making, semantic incremental retransmission, and dynamic balance optimization, enabling wireless image transmission systems to maintain high reconstruction quality and low resource consumption under various channel conditions and different signal-to-noise ratios. It has significant technological advancements and practical value. Attached Figure Description

[0017] Figure 1 This is a flowchart of the image transmission method based on joint source-channel coding with hybrid retransmission according to the present invention; Figure 2 This is a structural diagram of the channel encoder, decoder, and signal-to-noise ratio adaptive module of the present invention; Figure 3 This is a structural diagram of the rate control network of the present invention; Figure 4 This is a comparison chart of the average channel usage of the present invention with other methods for image transmission and reconstruction under different channel types and different signal-to-noise ratios; Figure 5 This is a technical flowchart of the present invention; Figure 6 This is a schematic diagram of the hybrid automatic repeater mechanism model constructed in this invention. Detailed Implementation

[0018] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, some features described in the examples may be combined in other examples.

[0019] This embodiment provides an image transmission method based on joint source-channel coding with hybrid retransmission, such as... Figure 1 As shown, it includes the following steps: Step 1: Acquire the original image and generate initial feature representations using a joint source-channel encoder; The sending end obtains the raw image to be transmitted. ,in Indicates the image height. Indicates the image width. Indicates the number of channels. The raw image is input into the joint source-channel encoder. In this process, the encoder uses a deep neural network to achieve an end-to-end mapping from image to channel features, generating an initial feature representation. : in, Represents the network parameters of the encoder. The dimension representing the feature. This is a continuous feature vector suitable for channel transmission. This feature represents both the semantic information of the image and a certain degree of anti-interference capability.

[0020] The aforementioned joint source-channel encoder comprises multiple layers of convolutional neural network and residual connection structures. The input layer receives the original image, extracts multi-scale features from the image through several convolutional layers, and the residual connection structure transmits feature information between different layers. The output layer maps the extracted features into continuous feature vectors suitable for channel transmission. This encoder and decoder are jointly optimized within an end-to-end training framework. The training process employs supervised learning, updating network parameters by minimizing the distortion between the reconstructed image and the original image. The optimization strategy uses the Adam optimization algorithm.

[0021] Step 2: Obtain the noisy received signal based on the physical channel transmission characteristic representation; Represent the initial features Transmission is performed via a physical channel. When the communication channel is an additive white Gaussian noise channel, the receiver acquires a noisy received signal. for: in, This represents additive white Gaussian noise. Indicates noise power. Represents the identity matrix.

[0022] When the communication channel is a Rayleigh fading channel, the receiver acquires a noisy received signal. for: in, This represents the channel fading coefficient according to the Rayleigh distribution.

[0023] Step 3: Use a decoder to process the received signal and generate a preliminary reconstructed image; The receiver will receive the signal with noise. Input to decoder In this process, the decoder learns to recover semantic information from noisy features of the original image based on a deep neural network, generating a preliminary reconstructed image. : in, Indicates the network parameters of the decoder. This indicates the preliminary reconstruction results under the current channel conditions.

[0024] The aforementioned decoder comprises multiple layers of convolutional neural network and residual connection structures. The input layer receives the noisy received signal, and the spatial resolution of the image is gradually restored through deconvolutional layers. The residual connection structures transfer feature information between different layers, and the output layer generates the reconstructed image. This decoder and encoder are jointly optimized in an end-to-end training framework. The training process adopts a supervised learning mode, and the optimization strategy uses the Adam optimization algorithm.

[0025] Step 4: Calculate the distortion metric of the preliminary reconstructed image and determine whether to trigger the retransmission mechanism; The receiving end calculates the preliminary reconstructed image. With the original image Distortion measurement between Mean squared error is used as the distortion assessment index: in, The first part represents the original image. pixel value, The first image represents the initial reconstructed image. pixel value, This represents the total number of pixels in the image.

[0026] Determine distortion measurement Is it greater than the preset distortion threshold? .like The receiving end then sends a NACK retransmission request signal to the sending end through the feedback link, triggering the hybrid automatic repeat mechanism; if If the receiving end sends an ACK confirmation signal to the sending end, the transmission ends successfully.

[0027] Step 5: Obtain image content features and channel signal-to-noise ratio, and use the policy network to generate the number of activation feature groups; When the retransmission mechanism is triggered, the sending end extracts the original image. Content features. A global average pooling operation is performed on the original image to obtain the image content feature vector. .

[0028] The aforementioned global average pooling operation averages the input image along its spatial dimension, with the input being the original image. The output is an image content feature vector. This vector contains global semantic information about the image.

[0029] Simultaneously obtain the signal-to-noise ratio parameter of the current channel. Image content feature vector and channel signal-to-noise ratio Concatenate the vectors to generate a joint state vector. : Joint state vector The input is fed into a policy network, which consists of a multilayer perceptron (MLP) and a softmax layer. The policy network outputs a probability distribution of the number of activations in feature groups. ,in This represents the maximum number of feature groups. The probability distribution is analyzed using the Gumbel-Softmax sampling technique. Sampling is performed to generate discrete activation feature groups. .

[0030] The aforementioned policy network receives a joint state vector at its input layer, extracts state features through two fully connected layers, and generates a probability distribution by applying a softmax activation function to its output layer. This policy network is jointly optimized with the encoder and decoder within an end-to-end training framework. The training process employs a policy gradient method, updating network parameters by minimizing the overall loss function, which includes reconstruction error and bandwidth regularization terms. The optimization strategy utilizes the Adam optimization algorithm. The overall loss function is a weighted sum of the mean squared error between the reconstructed image and the original image, plus the bandwidth regularization term of the activated feature group. The bandwidth regularization term is the sum of the elements of the thermometer encoding mask, and the weight coefficients are dynamically generated by the balancing factor prediction network.

[0031] It should be noted that the output process of the policy network includes the following steps: converting the joint state vector... The input is a two-layer fully connected neural network. The output dimension of the first fully connected layer is 128, and the output dimension of the second fully connected layer is... Apply the softmax activation function to the output of the second fully connected layer to generate a probability distribution. ,in Indicates activation The probabilities of each feature group satisfy the following condition. .

[0032] It should be noted that the Gumbel-Softmax sampling process includes the following steps: generating probability distributions. Gumbel noise vectors of the same dimension Each element , The variables are uniformly distributed random variables; during the forward propagation phase, discrete one-hot vectors are obtained using Gumbel-Max sampling. The calculation formula is: in, The indicator function is used; during the backpropagation phase, the probability distribution of softening is calculated using the Gumbel-Softmax continuous approximation. : in, Represents temperature parameters, controlling the smoothness of the distribution; converts discrete one-hot vectors Convert to thermometer encoding mask ,in This mask ensures that the activated feature groups are selected consecutively starting from index 0.

[0033] Step 6: Modulate the intermediate features of the encoder using the signal-to-noise ratio adaptive module to generate an adaptive feature representation; In the joint source channel encoder An adaptive signal-to-noise ratio (SNR) module is inserted between the residual network layers. This module acquires the feature maps of the intermediate layers of the encoder. ,in , and These represent the height, width, and number of channels of the intermediate feature, respectively.

[0034] For intermediate feature maps Global average pooling is performed on each channel to obtain channel-level feature vectors. Channel-level feature vectors With the current channel signal-to-noise ratio Concatenate the vectors to obtain the fused vector. .

[0035] fusion vector The input is fed into a two-layer fully connected network, and the first fully connected network outputs a channel-level scaling factor. The second-layer fully connected network outputs channel-level offsets. For intermediate feature maps Channel-level modulation is performed to generate adaptive feature representations. : in, Indicates the channel index. and They represent the first The original and modulated feature maps of each channel are presented. This adaptive feature representation dynamically adjusts the feature distribution according to the signal-to-noise ratio, enhancing robustness under different channel conditions.

[0036] The aforementioned two-layer fully connected network receives the fused vector at its input layer. The first fully connected layer performs dimensionality reduction on the input and applies an activation function, then splits into two parallel branches to generate the scaling factor and offset, respectively. This fully connected network is jointly optimized with the encoder and decoder in an end-to-end training framework. The training process adopts a supervised learning mode, and the optimization strategy uses the Adam optimization algorithm.

[0037] It should be noted that the channel-level scaling factor and channel-level offset The generation process is as follows: merging the vector The input is the first fully connected layer, and the output dimension of this layer is... ,in To achieve the dimensionality reduction ratio, the ReLU activation function is applied to the output of the first fully connected layer. The activated outputs are then input into two parallel fully connected layers, each with an output dimension of [missing value]. Generate scaling factors respectively and offset ; for scaling factor Apply the sigmoid activation function to ensure that its value range is within the specified range. between.

[0038] Step 7: Selectively activate the encoded features based on the number of activated feature groups to generate supplementary feature representations; The encoder encodes the original image to generate a complete feature representation. ,in This represents the dimension of each feature group. The number of activated feature groups is based on the policy network generated in step 5. and thermometer encoding mask Selective activation of encoded features.

[0039] thermometer encoding mask Applied to complete feature representation Generate the activated retransmission feature representation : in, This represents element-wise multiplication. In the activated feature representation, the first... One feature group retains its original value, while the remaining feature groups are set to zero. The sending end will transmit the supplementary feature representation. Transmitted to the receiving end via physical channels.

[0040] It should be noted that the number of activated feature groups The value of is related to the actual number of bits transmitted. They are directly proportional: in, This represents the average channel capacity for each feature group. When channel quality is good or image content is simple, the policy network tends to choose a smaller value. The value is increased to reduce the number of transmitted bits and decrease channel resource usage; when the channel quality is poor or the image content is complex, the policy network tends to select a larger value. This increases the number of transmitted bits and improves the reconstruction quality.

[0041] It should be noted that the receiving end does not require additional control information for recognizing the supplementary feature representation. Because a thermometer encoding method is used, the activated feature groups are arranged consecutively starting from index 0. The receiving end automatically determines the number of activated feature groups by detecting the distribution pattern of non-zero elements in the feature vector. This allows for the correct parsing of the feature set.

[0042] Step 8: Use the decoder to fuse the preliminary reconstruction information and the supplementary features to generate the final reconstructed image; The receiving end obtains the retransmission feature representation transmitted through the physical channel, denoted as... The initial image reconstruction will be performed. Corresponding intermediate feature representation and supplementary feature representation The features are then fused. The decoder processes the fused features to generate the final reconstructed image. : in, This represents a decoder network that supports feature fusion. The final reconstructed image... Compared to the initial reconstructed image It has higher reconstruction quality and significantly reduced distortion.

[0043] It should be noted that the feature fusion process includes the following steps: the decoder will initially receive the signal. Intermediate feature representations are generated through a partial decoding layer. ; representation of supplementary transmission features Decode the non-zero feature groups in the data to generate supplementary feature representations. The intermediate features are represented by feature concatenation or weighted summation. and supplementary feature representation Perform fusion to generate fusion features or ,in and For fusion weights; fusion features The input decoder is then fed into subsequent layers to generate the final reconstructed image. .

[0044] In this embodiment, to further enhance the intelligence of the retransmission strategy, distortion information of the initially reconstructed image is introduced into the input of the strategy network. Distorted information Defined as preliminary reconstructed image With the original image Mean square error between: The joint state vector of the policy network is modified as follows: Policy networks learn from joint state vectors Number of activated feature groups Mapping relationship: This mapping relationship satisfies a dynamic adjustment strategy that increases the number of activated feature groups as distortion increases, i.e.: When the initial reconstruction of the image is distorted When the distortion is large, the policy network tends to activate more feature groups, enhancing the ability to repair the retransmitted content; when the distortion is large... When the size is small, the policy network tends to activate fewer feature groups to avoid unnecessary resource consumption.

[0045] In this embodiment, a balance factor prediction network is introduced to achieve a dynamic trade-off between reconstruction quality and bandwidth overhead. The input to this prediction network includes image content features. Current channel signal-to-noise ratio and historical transmission state vector Historical transmission state vector Including peak signal-to-noise ratio in preceding transmission rounds And the bitrate information used. The balance factor prediction network outputs a dynamic balance factor. : in, This represents the parameters of the prediction network. It is a non-negative scalar. During training, an overall loss function that includes a dynamic rebalancing factor is used: Among them, the first item The mean square error of image reconstruction, the second term To activate the bandwidth regularization term of the feature group, This represents the training dataset. Balance factor. The system dynamically adjusts based on current image content, channel status, and historical transmission data to achieve scene-adaptive quality and resource balance.

[0046] The aforementioned balance factor prediction network is implemented based on a multilayer perceptron. The input layer receives a concatenated vector of image content features, channel signal-to-noise ratio, and historical transmission state vectors. This vector is then processed through several fully connected layers to extract joint features, and the output layer generates a non-negative dynamic balance factor. This prediction network is jointly optimized with the encoder, decoder, and policy network within an end-to-end training framework. The training process employs supervised learning, updating network parameters by minimizing the overall loss function, and using the Adam optimization algorithm as the optimization strategy.

[0047] In this embodiment, to enhance the efficiency of the decoder in utilizing the reconstructed features, an attention mechanism is introduced into the decoder. The attention mechanism assigns different weights to different spatial locations of the preliminary reconstructed features and the reconstructed features, focusing on image regions with significant distortion. Specifically, the decoder calculates the preliminary reconstructed features... and supplementary transmission features Spatial attention map : in, This represents the convolution operation. Represents feature concatenation. Based on attention map. For supplementary transmission features Weighting: Weighted overtransmission features Features of the initial reconstruction The system fuses the data to generate the final fused features, which are then input into subsequent decoding layers. This attention mechanism enables the decoder to adaptively allocate computational resources to regions of the image that require focused restoration, further improving the quality of the final reconstructed image.

[0048] It is understood that data preprocessing methods known to those skilled in the art include data cleaning, data transformation, and data reduction. Data transformation includes type conversion and normalization and standardization. Although the dimensions and types of data were omitted in the description of the preceding embodiments, data preprocessing is a technical knowledge known to those skilled in the art and a prerequisite step in data processing. Therefore, the previously described well-known data preprocessing steps were not described independently.

[0049] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. An image transmission method based on joint source-channel coding with hybrid retransmission, characterized in that, Includes the following steps: The original image is acquired and input into the joint source-channel encoder to generate an initial feature representation; The initial feature representation is transmitted through a physical channel, and the receiving end obtains the received signal with noise. The receiving end inputs the noisy received signal into the decoder to generate a preliminary reconstructed image; Calculate the distortion measure between the initially reconstructed image and the original image, and determine whether the distortion measure is greater than a preset distortion threshold. If it is greater than the preset distortion threshold, trigger the retransmission mechanism. After the retransmission mechanism is triggered, the sending end extracts the content features of the original image, obtains the signal-to-noise ratio parameter of the current channel, concatenates the content features and the signal-to-noise ratio parameter to generate a joint state vector, inputs the joint state vector into the policy network, the policy network outputs the probability distribution of the number of activated feature groups, and samples the probability distribution to generate a discrete number of activated feature groups. A signal-to-noise ratio (SNR) adaptive module is inserted between the residual network layers of the joint source-channel encoder. The SNR adaptive module obtains the feature map of the intermediate layer of the encoder, performs global average pooling on each channel of the feature map to obtain a channel-level feature vector, concatenates the channel-level feature vector with the current channel SNR to obtain a fusion vector, inputs the fusion vector into a fully connected network to generate a channel-level scaling factor and a channel-level offset, and performs channel-level modulation on the feature map to generate an adaptive feature representation. The encoder encodes the original image to generate a complete feature representation, selectively activates the complete feature representation according to the number of activated feature groups, generates a supplementary feature representation, and the transmitter transmits the supplementary feature representation to the receiver through a physical channel. The receiving end acquires the supplementary feature representation, fuses the intermediate feature representation corresponding to the preliminary reconstructed image with the supplementary feature representation, and the decoder processes the fused features to generate the final reconstructed image. The policy network outputs the joint state vector by inputting it into a two-layer fully connected neural network. An activation function is applied to the output of the first fully connected layer, and a normalized activation function is applied to the output of the second fully connected layer to generate the probability distribution. The sampling of the probability distribution adopts a differentiable sampling technique. In the forward propagation stage, a discrete vector is obtained by selecting the maximum value. In the backpropagation stage, a continuous approximation is used to achieve gradient backpropagation, and the discrete vector is converted into a thermometer-coded mask. The thermometer-coded mask ensures that the activated feature groups are selected continuously starting from index zero.

2. The image transmission method based on joint source-channel coding with hybrid retransmission according to claim 1, characterized in that, The distortion metric is calculated using mean squared error based on the image semantic reconstruction quality, which evaluates the semantic similarity between the original image and the preliminary reconstructed image in the pixel space, rather than just evaluating the accuracy of bit-level transmission. The distortion metric is calculated by dividing the sum of squares of the differences between corresponding pixel values ​​in the original image and the preliminary reconstructed image by the total number of pixels in the image.

3. The image transmission method based on joint source-channel coding with hybrid retransmission according to claim 1, characterized in that, The input to the policy network also includes distortion information of the preliminary reconstructed image, which is defined as the mean square error between the preliminary reconstructed image and the original image. The policy network learns a mapping relationship from the joint state vector to the number of activated feature groups, and the mapping relationship satisfies a dynamic adjustment strategy that the greater the distortion, the more activated feature groups there are.

4. The image transmission method based on joint source-channel coding with hybrid retransmission according to claim 1, characterized in that, The process for generating the channel-level scaling factor and the channel-level offset is as follows: The fusion vector is input into the first fully connected layer, and the first fully connected layer performs dimensionality reduction output according to the dimensionality reduction ratio. Apply an activation function to the output of the first fully connected layer; The activated output is input into two parallel fully connected layers to generate the channel-level scaling factor and the channel-level offset, respectively. A limit activation function is applied to the channel-level scaling factor to ensure that its value is between zero and one.

5. The image transmission method based on joint source-channel coding with hybrid retransmission according to claim 1, characterized in that, The number of activated feature groups is directly proportional to the actual number of transmitted bits. When the channel quality is good, the strategy network selects a smaller number of activated feature groups to reduce the number of transmitted bits. When the channel quality is poor, the strategy network selects a larger number of activated feature groups to increase the number of transmitted bits.

6. The image transmission method based on joint source-channel coding with hybrid retransmission according to claim 1, characterized in that, The receiving end does not require additional control information for the recognition process of the supplementary feature representation. The receiving end automatically determines the number of active feature groups by detecting the distribution pattern of non-zero elements in the feature vector.

7. The image transmission method based on joint source-channel coding with hybrid retransmission according to claim 1, characterized in that, The feature fusion process includes: The decoder passes the initially received signal through a partial decoding layer to generate intermediate feature representations; Decode the non-zero feature groups in the supplementary feature representation to generate a supplementary feature representation; The intermediate feature representation and the supplementary feature representation are fused together by feature concatenation to generate a fused feature; The fused features are input into subsequent layers of the decoder to generate the final reconstructed image.

8. The image transmission method based on joint source-channel coding with hybrid retransmission according to claim 1, characterized in that, It also includes a balance factor prediction network. The input of the balance factor prediction network includes the content features of the original image, the current channel signal-to-noise ratio, and the historical transmission state vector. The historical transmission state vector contains the peak signal-to-noise ratio and bit rate information in the previous transmission rounds. The balance factor prediction network outputs a dynamic balance factor. During training, an overall loss function including the dynamic balance factor is used. The overall loss function includes the mean square error term of image reconstruction and the bandwidth regularization term of the activated feature group. The dynamic balance factor is dynamically adjusted according to the current image content, channel state, and historical transmission conditions.

9. The image transmission method based on joint source-channel coding with hybrid retransmission according to claim 1, characterized in that, An attention mechanism is introduced into the decoder, which assigns different weights to different spatial locations of the preliminary reconstructed features and the supplementary features. The decoder calculates a spatial attention map of the preliminary reconstructed features and the supplementary features, and weights the supplementary features based on the spatial attention map. The weighted supplementary features are then fused with the preliminary reconstructed features to generate the final fused features, which are then input into the subsequent decoding layer.

10. An image transmission system based on hybrid retransmission joint source channel coding, used to execute the image transmission method based on hybrid retransmission joint source channel coding as described in any one of claims 1 to 9, characterized in that, include: The encoding module is used to acquire the original image, input the original image into the joint source-channel encoder, and generate an initial feature representation; A channel transmission module is used to transmit the initial feature representation through a physical channel; The preliminary decoding module is used to input the noisy received signal into the decoder to generate a preliminary reconstructed image; The distortion assessment module is used to calculate the distortion measure between the preliminary reconstructed image and the original image, and to determine whether to trigger the retransmission mechanism. The strategy decision module is used to extract the content features of the original image and the signal-to-noise ratio parameter of the current channel after the retransmission mechanism is triggered, generate a joint state vector, and output the probability distribution of the number of activated feature groups through the policy network to generate a discrete number of activated feature groups. The adaptive modulation module is used to perform channel-level modulation on the feature map of the encoder's intermediate layer through the signal-to-noise ratio adaptive module to generate an adaptive feature representation. The supplementary transmission generation module is used to selectively activate the complete feature representation according to the number of activated feature groups, generate a supplementary feature representation, and transmit it. The feature fusion module is used to fuse the intermediate feature representations corresponding to the preliminary reconstructed image with the supplementary feature representations to generate the final reconstructed image.

Images