Source-channel joint coding method for joint digital communication and analog communication

Abstract

The invention discloses an information source channel joint coding method for joint digital communication and analog communication. The method comprises the following steps: step 1, in a digital branch, inputting a picture, and firstly performing down-sampling and image coding to obtain a code stream; 2, in the analog branch, subtracting the predicted value from the input picture to obtain a residual error, and performing learning-based transformation on the residual error to obtain a characteristic of the residual error; step 3, bandwidth allocation is realized; 4, sending the characteristics subjected to bandwidth allocation to a receiving end through a wireless channel; 5, at a receiving end, the signals received by the digital branches are restored into predicted values through demodulation, channel decoding, image decoding and up-sampling in sequence; and step 6, performing demodulation and learning-based inverse transformation on the signals of the analog branches, reconstructing a residual error, and adding the residual error and a predicted value to obtain a reconstructed picture. According to the invention, source and channel coding is carried out by using a traditional separated coding method through a digital branch, and transmission is carried out by using a residual error through an analog branch, so that better performance is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of transmission efficiency of LJSCC (LJSCC), and in particular relates to a source channel joint coding method for joint digital communication and analog communication. Background Art

[0002] Modern communication systems use a two-step coding process to transmit image / video data: (i) the image / video data is compressed using a source coding algorithm to remove inherent redundancy and reduce the amount of information transmitted; (ii) the compressed bit stream is first encoded with an error correction code that allows for resilient transmission against errors and then modulated. Shannon's separation theorem proves that the two-step source channel coding approach is theoretically optimal in the asymptotic limit of infinitely long source channel blocks. Although in practice, joint source channel coding (JSCC) is known to outperform the separation approach, it has advantages for practical communication systems due to the modularity provided by the separation architecture.

[0003] Furthermore, efficient compression algorithms (e.g., JPEG, WebP, BPG) and near-optimal channel codes (e.g., LDPC, Turbo codes) have been employed in practice to approach the theoretical limit. However, many emerging applications ranging from the Internet of Things to autonomous driving and the Tactile Internet require the transmission of image / video data under extreme latency, bandwidth, and / or energy constraints, which rules out computationally demanding long block length source and channel coding techniques.

[0004] In recent years, end-to-end optimized neural or deep learning based data transmission LJSCC has become an active research area in semantic communication. For image transmission tasks, the use of convolutional neural network (CNN) backbones and their variants can currently produce end-to-end image transmission performance that exceeds the classic separation-based methods (JPEG / JPEG2000 / BPG combined with advanced channel codes).

[0005] Bourtsoulatze et al. proposed the first convolutional neural network (CNN) based scheme that outperforms separation-based digital transmission schemes under low signal-to-noise ratio (SNR) and channel bandwidth conditions, especially for small-dimensional sources such as the tiny CIFAR10 images (32×32 resolution) dataset, highlighting the effectiveness of their approach.

[0006] Later, Xu et al. proposed the attention-based LJSCC, which constructed a single network capable of handling a range of SNR values ​​for better model adaptability.

[0007] Yuan et al. further improved the method and achieved excellent performance across various signal-to-noise ratio levels during transmission without relying on channel prior information. Yang et al. implemented adaptive rate according to different channel signal-to-noise ratios and image content.

[0008] Several existing learning-based joint source channel coding (LJSCC) methods mentioned above have limitations in transmission efficiency. Although LJSCC surpasses traditional digital methods in performance, its design as an analog transmission framework still results in lower efficiency than digital transmission. This is because LJSCC relies on mapping the source to a dense constellation, which may cause a large performance degradation when the channel quality is poor. Moreover, although LJSCC can achieve SNR scalability, so that the reconstruction quality can decay gracefully when the channel SNR decreases, this design may not perform well under different channel conditions. In addition, existing methods lack flexibility in handling bandwidth allocation and cannot be optimized according to different transmission requirements, which affects the overall transmission performance. Summary of the invention

[0009] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a source-channel joint coding method for joint digital communication and analog communication, which combines the advantages of digital transmission and analog transmission and uses neural networks to improve transmission efficiency while maintaining the scalability of signal-to-noise ratio. The framework adopts a dual-branch structure, in which the digital branch uses the traditional separate coding method for source and channel coding, while the analog branch uses residuals for transmission to achieve better performance.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is:

[0011] A source channel joint coding method for joint digital communication and analog communication comprises the following steps:

[0012] Step 1: In the digital branch, the input image is first downsampled and image encoded, and the resulting code stream acts on three parts;

[0013] The first part performs image decoding and upsampling to generate prediction values ​​for use by the simulation branch;

[0014] The second part performs channel coding and modulation, transmits through the wireless channel, and sends it to the receiving end;

[0015] The third part,plays a role in bandwidth allocation of the simulation branches;

[0016] Step 2: In the simulation branch, the input image is subtracted from the predicted value generated in step 1 to obtain the residual, and the residual is transformed based on learning to obtain the residual feature;

[0017] Step 3: According to the proposed bandwidth allocation, using the bitstream in Step 1 and the given bandwidth, generate a mask, and mask the features of the residual in Step 2 to achieve bandwidth allocation;

[0018] Step 4: Perform power allocation and dense modulation on the features with bandwidth allocation in Step 3, and send them to the receiving end through a wireless channel;

[0019] Step 5: At the receiving end, the signal received by the digital branch is successively demodulated, channel decoded, image decoded, and upsampled to restore the predicted value;

[0020] Step 6: The signal of the analog branch is demodulated and subjected to a learning-based inverse transform to reconstruct the residual. Finally, the residual is added to the predicted value obtained in Step 5 to obtain the reconstructed image.

[0021] In Step 1, the image coding in the digital branch adopts a deep learning-based method;

[0022] The image coding adopts JPEG, BPG, or VVC;

[0023] In Step 1, the input image I 3×H×W , is downsampled at a given ratio r, 0 < r < 1, and then the thumbnail is compressed using an image compression algorithm, i.e., image coding, with a given quantization parameter (QP) to generate a compressed bitstream s bin .

[0024] In the first part of Step 1, after the compressed bitstream s bin is image decoded and upsampled, the prediction of the analog branch is obtained

[0025] In the second part of Step 1, the compressed bitstream s bin is channel encoded with a convolutional code at a rate of 1 / 2 and QPSK modulated.

[0026] In Step 2, the input image is subtracted by the prediction to obtain the residual, and the obtained residual is sent to a learning-based transform to extract the feature F C×H′×W′ , expressed as:

[0027]

[0028] where f enc (·) represents the learning-based transform.

[0029] In Step 3, the bandwidth allocation dynamically adjusts the bandwidth ratio of the digital and analog branches according to the signal-to-noise ratio (SNR);

[0030] The bandwidth allocation adaptively allocates bandwidth resources according to the image complexity;

[0031] When the channel condition reaches a preset limit (such as the signal-to-noise ratio is lower than a certain threshold), the bandwidth allocation automatically switches to the single-branch transmission mode. In this mode, data is transmitted in either the digital or analog branch.

[0032] Specifically, step 3 is as follows:

[0033] Under the constraint of the available bandwidth, the feature is passed to the bandwidth allocation module to generate a binary mask MASK C×H′×W′ ;

[0034] The feature F' to be transmitted is determined by the following formula:

[0035] F' = F C×H′×W′ ⊙ MASK C×H′×W′ ,

[0036] where ⊙ represents element-wise multiplication. After that, F' is flattened into a one-dimensional vector F v ' and the elements corresponding to zero in MASK are deleted. Then, for each feature element of F C×H′×W′ ' v a power scaling g is performed, where i = 1, 2,..., N and N ≤ C × H' × W'; i The scaled element

[0037] can be obtained through the following formula:

[0038]

[0039] where P is the total transmission power, and ∈ is set to 1 × 10 -10 . Then, each pair of scaled elements is mapped to a complex symbol in the 64K-QAM constellation and transmitted through the original OFDM channel.

[0040] The bandwidth allocation in step 3 is to adapt to different bandwidths through the binary mask MASK C×H′×W′ . The mask size is constrained by C × H' × W' = 3 × H × W to achieve full-bandwidth transmission.

[0041] The generated mask depends on the number and position of non-zero elements. The number of non-zero elements depends on the available bandwidth, which is defined by the channel bandwidth ratio CR (0 < CR < 1) as:

[0042]

[0043] B is the total consumed bandwidth, that is, the number of transmitted symbols. B ana and B dig represent the bandwidths consumed by the analog and digital branches respectively.

[0044] Using analog bandwidth B ana , MASK C×H′×W′ Each element mask i,j,k It can be obtained through the following methods:

[0045]

[0046] In step 6, at the receiving end of the analog branch, the demodulated receiving vector Interference by noise e:

[0047]

[0048] in

[0049] The step 6 is specifically as follows:

[0050] Perform learning-based inverse transformation and reconstruction of the residual, generated by the following formula:

[0051]

[0052] where f dec (·) is the inverse transformation based on learning, thus obtaining the reconstructed image.

[0053] Beneficial effects of the present invention:

[0054] First, the hybrid digital-analog framework (Hybrid-LJSCC) proposed in the present invention improves the transmission efficiency by combining the advantages of digital transmission and analog transmission. By introducing digital branches and using existing digital compression methods (such as BPG) for preliminary compression and prediction, the encoding of the source signal is made more effective. This method significantly reduces the redundant information transmitted in the channel, thereby improving the overall transmission efficiency.

[0055] Secondly, the present invention optimizes the bandwidth utilization of the channel while maintaining the scalability of the signal-to-noise ratio (SNR). By introducing the processing of the residual signal in the analog branch, the invention effectively utilizes the prediction generated by the digital branch to ensure the integrity and accuracy of the signal during transmission. This design enables the system to achieve high-quality image reconstruction even in a low signal-to-noise ratio environment. Compared with the "cliff effect" of traditional methods, the present invention shows superior robustness.

[0056] Furthermore, experimental results show that Hybrid-LJSCC achieves a peak signal-to-noise ratio (PSNR) improvement of up to 4dB over the latest LJSCC baselines (such as SwinJSCC and DeepJSCC) under multiple data sets and different channel conditions. This not only demonstrates the obvious advantages of the present invention in image reconstruction quality, but also shows its applicability and effectiveness in end-to-end communication systems.

[0057] Finally, the digital-analog hybrid design of the present invention provides greater room for scalability and flexibility. With the continuous advancement of technology, the specific implementation of the digital and analog parts can be further optimized according to actual needs in the future to meet the needs of different application scenarios. This feature makes the present invention have better adaptability and prospects, and can meet the challenges of future communication technology development. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the process framework of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0060] like Figure 1 As shown, Figure 1 The framework of the proposed method is shown, which mainly includes two branches: a digital transmission branch and an analog transmission branch; both branches include sending and receiving respectively, and both branches have a complete transmission process.

[0061] The digital branch generates predictions for the analog branch using a separation-based source and channel coding approach;

[0062] The prediction produced by the digital branch codec is then subtracted from the input image to simplify subsequent power allocation. The obtained residual is then transmitted through the analog branch;

[0063] The latest LJSCC schemes, such as DeepJSCC and SwinJSCC, can be used for analog transmission to benefit from the powerful nonlinear representation of neural networks. Although higher quality prediction will lead to more improvements in the analog branch, it will also increase the bandwidth of digital transmission and allocate less bandwidth to the analog branch.

[0064] In order to make a better trade-off, a bandwidth allocation module is introduced.

[0065] Step (a): Digital transmission: The efficiency of analog transmission alone is insufficient. By introducing a digital transmission branch, a predicted value is provided for analog transmission, thereby improving the transmission efficiency.

[0066] In the digital branch, the input image I 3×H×WFirst, downsample at a given ratio r (0 < r < 1) to save bandwidth;

[0067] Then, use an image compression algorithm, i.e., image encoding (such as BPG, VVC, or advanced end-to-end image compression), to compress the thumbnail with a given quantization parameter (QP) to generate a compressed stream s bin ; QP is the quantization parameter when encoding with existing schemes such as BPG or VVC, generally ranging from 0 to 51

[0068] After decoding and upsampling, obtain the prediction of the analog branch A smaller ratio r and a larger QP will consume less bandwidth but result in a lower-quality prediction of the analog branch; conversely, a higher-quality prediction will be obtained.

[0069] To transmit the compressed stream s without errors bin , adopt conservative protection, i.e., use a convolutional code with a rate of 1 / 2 for channel encoding and QPSK modulation to be compatible with the IEEE802.11a standard.

[0070] Although BPSK is the most robust modulation, QPSK can achieve the exact same bit error rate when the energy per symbol is doubled, while the bandwidth can be reduced by half compared to BPSK.

[0071] At the receiving end, demodulate the received signal, perform corresponding channel decoding and image decoding, and then use bilinear interpolation for upsampling to restore the predicted value to the original image size.

[0072] Step (2): Analog transmission

[0073] The digital branch is difficult to maintain good transmission performance under various channel conditions, while using the analog branch to achieve SNR scalability enables the system to maintain good transmission performance under various conditions. Before performing analog transmission, first subtract the predicted value obtained by decoding the digital branch generated by the digital branch from the input image to obtain the residual; then send the obtained residual to a learning-based transform to extract the feature F C×H′×W′ , expressed as:

[0074]

[0075] where f enc (·) represents the learning-based transform; CNN, Transformer, or even more advanced architectures can be used as such transforms;

[0076] Then, with the available bandwidth as a constraint, pass the feature to the bandwidth allocation module to generate a binary mask MASK C×H′×W′ ; Then, the feature F′ to be transmitted is determined by the following formula:

[0077] F′=F C×H′×W′ ⊙MASK C×H′×W′ ,

[0078] Where ⊙ represents the element-wise product. After that, F′ is flattened into a one-dimensional vector F v ′ and delete MASK C×H′×W′ The elements corresponding to zero in F v Each characteristic element of ′ Perform power scaling g i , where i = 1, 2, ..., N and N ≤ C × H′ × W′;

[0079] The scaled element It can be obtained by the following formula:

[0080]

[0081] Where P is the total transmission power, ∈ is set to 1×10 -10 Then, each pair of scaling elements is mapped to a complex symbol in the 64K-QAM constellation and transmitted through the original OFDM channel, thus avoiding the traditional channel coding and modulation used in digital communication; at the receiving end of the analog branch, due to the error generated by the signal during the transmission process, the demodulated receiving vector Interference by noise e:

[0082]

[0083] in Then, the learning-based inverse transformation and reconstruction of the residual is performed, which is generated by the following formula:

[0084]

[0085] where f dec (·) is the inverse transformation based on learning, thus obtaining the reconstructed image.

[0086] In general, at the analog branch receiving end, the received signal is demodulated and inversely transformed based on learning to obtain a reconstructed residual, which is added to the predicted value obtained by the digital branch to obtain a reconstructed image.

[0087] Step (three): Bandwidth allocation: In order to achieve balanced bandwidth for digital transmission and analog transmission, and enable the system to efficiently utilize bandwidth under different channel conditions, a bandwidth allocation module is designed.

[0088] This module aims to adapt the number of extracted features to a given bandwidth constraint. Considering that the digital bandwidth varies with the image content, a binary mask MASK similar to DeepJSCC is introduced C×H′×W′To adapt to different bandwidths. The mask size is constrained by C×H′×W′ = 3×H×W to achieve full bandwidth transmission.

[0089] The generated mask depends on two key parameters: the number and position of non-zero elements. The number of non-zero elements depends on the available bandwidth, which is defined by the channel bandwidth ratio CR (0 < CR < 1) as:

[0090]

[0091] B is the total consumed bandwidth, that is, the number of transmitted symbols, B ana and B dig represent the bandwidths consumed by the analog and digital branches respectively. It can be seen that given B, B ana and B dig is a trade-off. A higher B dig will result in a higher quality prediction but will also cause B ana to decrease.

[0092] In the implementation, BPG is adopted as the image codec. The downsampling rate r and the quantization parameter QP are designed through trial and error to control B dig . Set r = 1 / 64 and QP = 27. The remaining bandwidth B - B dig is allocated to the analog branch B ana . Using the analog bandwidth B ana , each element mask of MASK C×H′×W′ can be obtained as follows: i,j,k

[0093]

[0094] The present invention proposes a digital-analog hybrid transmission framework, which jointly completes source coding, channel coding, modulation, and transmission through the digital branch and the analog branch. The digital branch can generate a prediction signal based on any compression algorithm, while the analog branch improves the reconstruction quality through residual transmission, enabling the system to efficiently transmit under different channel conditions.

[0095] A joint coding mechanism for the digital and analog branches is proposed. The framework forms a residual by subtracting the predicted value of the digital branch from the image for analog transmission. This residual transmission method can effectively improve the accuracy of image reconstruction and ensure high-quality reconstruction even under low signal-to-noise ratio conditions.

[0096] A bandwidth adaptive allocation mechanism is proposed, and a bandwidth allocation strategy is designed for the digital and analog branches, enabling the system to efficiently utilize the bandwidth under different channel conditions, thereby optimizing the transmission performance.

[0097] Compatible with various deep learning and traditional coding methods: The framework allows the use of various deep learning coding methods or traditional coding methods as the encoding method of digital or analog branches, and is not limited to a specific model architecture or coding algorithm. Any coding method combined with this framework and using a digital-analog hybrid strategy falls within the scope of protection of this invention.

[0098] The digital-analog hybrid transmission architecture alternative of the present invention: The present invention currently adopts a two-branch framework (digital and analog branches). An alternative solution is to use other encoding methods in the digital branch, such as traditional image encoding schemes and deep learning-based image encoding schemes, while the analog branch can adopt a more flexible transmission strategy based on the residual image. In addition, Figure 1 The "upsampling" and "downsampling" in the above are not necessary. They are used to match the encoding method in the digital branch and should be considered as a whole with the image encoding method.

[0099] Alternative residual calculation methods: In this solution, the difference between the prediction value of the digital branch and the original image is used as the residual for transmission. Other prediction value calculation methods can be used, such as the result obtained by applying adaptive filtering to the original image as the prediction value, or dynamically selecting different generation methods based on content characteristics. These alternative calculation methods can ensure the adaptability of the analog branch to channel changes, while improving transmission efficiency and reconstruction effect.

[0100] Adaptive strategy replacement for bandwidth allocation: The bandwidth allocation strategy can adopt other adaptive mechanisms, such as dynamically adjusting the bandwidth ratio of digital and analog branches based on the signal-to-noise ratio (SNR), or adaptively allocating bandwidth resources according to the complexity of the image. Under extreme channel conditions, the bandwidth allocation strategy can also completely switch to a single-branch (digital or analog) transmission mode to adapt to instantaneous transmission requirements. This strategy expands the flexibility of bandwidth allocation and can further improve transmission performance under different channel conditions.

[0101] Compatible deep learning model alternatives: In addition to the existing convolutional neural network (CNN), other deep learning models such as Transformer or self-attention mechanism can be used in digital and analog branches to better capture image features and optimize encoding. These alternative models can also complete source-channel joint encoding and are suitable for different channel conditions and image characteristics.

Claims

1. A source channel joint coding method for joint digital communication and analog communication, characterized in that: Including the following steps; Step 1: In the digital branch, the input image is first downsampled and image-encoded, and the resulting bitstream acts on three parts; The first part is to perform image decoding and upsampling to generate a prediction value for use in the analog branch; The second part is to perform channel encoding and modulation, transmit through a wireless channel, and send to the receiving end; The third part plays a role in the bandwidth allocation of the analog branch. Step 2: In the analog branch, subtract the prediction value generated in Step 1 from the input image to obtain a residual, and pass the residual through a learning-based transformation to obtain the features of the residual; Step 3: According to the proposed bandwidth allocation, use the bitstream in Step 1 and the given bandwidth to generate a mask, and mask the features of the residual in Step 2 to achieve bandwidth allocation; Step 4: Perform power allocation and dense modulation on the features after bandwidth allocation in Step 3, and send them to the receiving end through a wireless channel; Step 5: At the receiving end, the signal received by the digital branch is successively demodulated, channel decoded, image decoded, and upsampled to restore the prediction value; Step 6: Demodulate the signal of the analog branch and perform a learning-based inverse transformation to reconstruct the residual. Finally, add the residual to the prediction value obtained in Step 5 to finally obtain the reconstructed image.

2. The method for joint source channel coding of joint digital communication and analog communication according to claim 1, characterized in that: The image encoding in the digital branch in Step 1 adopts a deep learning-based method; The image encoding adopts JPEG, BPG or VVC.

3. The source channel joint coding method for joint digital communication and analog communication according to claim 1, characterized in that: In the said step 1, input image I 3×H×W is downsampled at a given ratio r, where 0 < r < 1, and then the thumbnail is compressed using an image compression algorithm, i.e., image coding, with a given quantization parameter (QP) to generate a compressed stream s bin ; In the first part of step 1, the compressed stream s bin After image decoding and upsampling, the prediction of the simulated branch is obtained 4. The source channel joint coding method for joint digital communication and analog communication according to claim 3, characterized in that: In the second part of step 1, the compressed stream s bin Channel coding and QPSK modulation are performed with a rate 1 / 2 convolutional code.

5. The source channel joint coding method for joint digital communication and analog communication according to claim 3, characterized in that: In step 2, the input image is subtracted from the predicted Get the residual, and send the obtained residual to the learning-based transformation to extract the feature F C×H′×W′ , expressed as: where f enc (·) denotes a learning-based transformation.

6. The source channel joint coding method for joint digital communication and analog communication according to claim 5, characterized in that: In Step 3, the bandwidth allocation dynamically adjusts the bandwidth ratio of the digital and analog branches according to the signal-to-noise ratio (SNR); The bandwidth allocation adaptively allocates bandwidth resources according to the image complexity; When the channel condition reaches a preset limit, the bandwidth allocation automatically switches to the single-branch transmission mode. In this mode, data is transmitted in any one of the digital or analog branches.

7. The method for joint source channel coding of joint digital communication and analog communication according to claim 5, characterized in that: Step 3 is specifically as follows: With the available bandwidth as a constraint, the features are passed to the bandwidth allocation module to generate a binary mask MASK C ×h′×W′ ; The feature F′ to be transmitted is determined by the following formula: F′=F C×H′×W′ ⊙MASK C×H′×W′ , Where ⊙ represents the element-wise product. After that, F′ is flattened into a one-dimensional vector F′ v And delete MASK C×H′×w′ The elements corresponding to zero in F′ v Each characteristic element Perform power scaling g i , where i = 1, 2, ..., N and N ≤ C × H′ × W′; The scaled element It can be obtained by the following formula: Where P is the total transmission power, ∈ is set to 1×10 -10 ,Then, each pair of scaled elements is mapped to a complex symbol in the 64K-QAM constellation and transmitted through the original OFDM channel; The bandwidth allocation in step 3 is as follows: C×H′×W′ To adapt to different bandwidths, the mask size is constrained by C×H′×W′=3×H×W to achieve full bandwidth transmission.

8. The method for joint source channel coding of joint digital communication and analog communication according to claim 7, characterized in that: The generated mask depends on the number and position of non-zero elements. The number of non-zero elements depends on the available bandwidth, which is defined by the channel bandwidth ratio CR (0 < CR < 1) as: B is the total bandwidth consumed, that is, the number of transmitted symbols, ana and B dig Represents the bandwidth consumed by analog and digital branches respectively; Using analog bandwidth B ana , MASK C×H′×W′ Each element mask i,j,k Obtained through:

9. The method for joint source channel coding of joint digital communication and analog communication according to claim 6, characterized in that: In step 6, at the receiving end of the analog branch, the demodulated receiving vector Interference by noise e: in 10. The method for joint source channel coding of joint digital communication and analog communication according to claim 9, characterized in that: Step 6 is specifically as follows: Perform a learning-based inverse transformation and reconstruction of the residual, which is generated by the following formula: where f dec (·) is the inverse transformation based on learning, thus obtaining the reconstructed image.

Images