A joint source-channel coding method for digital and analog communications

Abstract

A source-channel joint coding method for combined digital and analog communications includes the following steps: Step 1: In the digital branch, an input image is first downsampled and image-encoded to obtain a code stream; Step 2: In the analog branch, a predicted value is subtracted from the input image to obtain a residual, and the residual is subjected to a learning-based transformation to obtain a residual feature; Step 3: Bandwidth allocation is implemented; Step 4: The bandwidth-allocated features are transmitted to a receiving end via a wireless channel; Step 5: At the receiving end, the signal received by the digital branch is sequentially demodulated, channel-decoded, image-decoded, and upsampled to restore it to a predicted value; Step 6: The signal of the analog branch is demodulated and subjected to a learning-based inverse transformation to reconstruct the residual, and the residual is added to the predicted value to obtain a reconstructed image. The present invention uses a traditional separate coding method for source and channel coding in the digital branch, while the analog branch uses the residual for transmission to achieve better performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of transmission efficiency of long-distance combined source channel coding (LJSCC), and in particular relates to a source channel joint coding method for combined digital communication and analog communication. Background Art

[0002] Modern communication systems use a two-step coding process to transmit image / video data: (i) the image / video data is compressed using a source coding algorithm to remove inherent redundancy and reduce the amount of information transmitted; (ii) the compressed bit stream is first encoded with an error-correcting code that provides resiliency against errors and then modulated. Shannon's separation theorem proves that the two-step source-channel coding approach is theoretically optimal in the asymptotic limit of infinitely long source-channel blocks. While joint source-channel coding (JSCC) is known to outperform the separation approach in practical applications, it has advantages for practical communication systems due to the modularity provided by the separation architecture.

[0003] Furthermore, in practice, efficient compression algorithms (e.g., JPEG, WebP, BPG) and near-optimal channel codes (e.g., LDPC, Turbo codes) have been employed to approach the theoretical limit. However, many emerging applications, from the Internet of Things to autonomous driving and the tactile Internet, require the transmission of image / video data under extreme latency, bandwidth, and / or energy constraints, which precludes computationally demanding long block length source and channel coding techniques.

[0004] In recent years, end-to-end optimized neural or deep learning-based data transmission (LJSCC) has become an active research area in semantic communication. For image transmission tasks, the use of convolutional neural network (CNN) backbones and their variants can currently produce end-to-end image transmission performance that exceeds classic separation-based methods (JPEG / JPEG2000 / BPG combined with advanced channel codes).

[0005] Bourtsoulatze et al. proposed the first convolutional neural network (CNN)-based scheme that outperforms separation-based digital transmission schemes under low signal-to-noise ratio (SNR) and channel bandwidth conditions, especially for small-dimensional sources such as the tiny CIFAR10 image (32×32 resolution) dataset, highlighting the effectiveness of their approach.

[0006] Later, Xu et al. proposed the attention-based LJSCC, which constructed a single network capable of handling a range of signal-to-noise ratio values ​​to achieve better model adaptability.

[0007] Yuan et al. further improved the method and achieved excellent performance across various signal-to-noise ratio levels during transmission without relying on prior information of the channel. Yang et al. implemented adaptive rate according to different channel signal-to-noise ratios and image content.

[0008] The aforementioned existing learning-based joint source channel coding (LJSCC) methods have limitations in transmission efficiency. Although LJSCC surpasses traditional digital methods in performance, its design as an analog transmission framework still results in lower efficiency than digital transmission. This is because LJSCC relies on mapping the source onto a dense constellation, which can lead to significant performance degradation when the channel quality is poor. Moreover, although LJSCC can achieve SNR scalability, allowing the reconstruction quality to decay gracefully as the channel SNR decreases, this design may not perform well under different channel conditions. In addition, existing methods lack flexibility in handling bandwidth allocation and cannot be optimized according to different transmission requirements, thus affecting overall transmission performance. Summary of the Invention

[0009] To overcome the shortcomings of the aforementioned prior art, the present invention aims to provide a joint source-channel coding method for digital and analog communications. By combining the advantages of digital and analog transmission, this method leverages neural networks to improve transmission efficiency while maintaining scalability in signal-to-noise ratio. This framework employs a dual-branch structure: the digital branch uses traditional separate coding methods for source and channel coding, while the analog branch utilizes residuals for transmission, achieving improved performance.

[0010] In order to achieve the above object, the technical solution adopted by the present invention is:

[0011] A source-channel joint coding method for combined digital communication and analog communication comprises the following steps:

[0012] Step 1: In the digital branch, the input image is first downsampled and encoded, and the resulting code stream is applied to three parts;

[0013] The first part performs image decoding and upsampling to generate prediction values ​​for use by the simulation branch;

[0014] The second part performs channel coding and modulation, transmits through the wireless channel, and sends it to the receiving end;

[0015] The third part plays a role in bandwidth allocation of the simulation branches;

[0016] Step 2: In the simulation branch, the predicted value generated in step 1 is subtracted from the input image to obtain the residual, and the residual is transformed based on learning to obtain the residual feature;

[0017] Step 3: According to the proposed bandwidth allocation, use the bitstream in Step 1 and the given bandwidth to generate a mask, and mask the features of the residuals in Step 2 to achieve bandwidth allocation;

[0018] Step 4: Perform power allocation and dense modulation on the features with bandwidth allocation in Step 3, and send them to the receiving end through a wireless channel;

[0019] Step 5: At the receiving end, the signal received by the digital branch is successively demodulated, channel decoded, image decoded, and upsampled to restore the predicted value;

[0020] Step 6: The signal of the analog branch is demodulated and subjected to a learning-based inverse transform to reconstruct the residuals. Finally, the residuals are added to the predicted value obtained in Step 5 to obtain the reconstructed picture.

[0021] In Step 1, the image encoding in the digital branch adopts a deep learning-based method;

[0022] The image encoding adopts JPEG, BPG, or VVC;

[0023] In Step 1, the input image I

[0028] , , , enc ,

[0030] , bin ,

[0026] ,

[0029] ,

[0025] , ,

[0027] , , C×H′×W′ , , , , , is downsampled at a given ratio r, 0 < r < 1, and then the thumbnail is compressed using an image compression algorithm, i.e., image encoding, with a given quantization parameter (QP) to generate a compressed bitstream s bin .

[0024] In the first part of Step 1, after the compressed bitstream s bin is image decoded and upsampled, the prediction of the analog branch is obtained [[ID=2,8]]

[0025] In the second part of Step 1, the compressed bitstream s bin is channel encoded with a convolutional code at a rate of 1 / 2 and QPSK modulated.

[0026] In Step 2, the input image is subtracted from the prediction to obtain the residuals, and the obtained residuals are sent to a learning-based transform to extract the features F C×H′×W′ , which is expressed as:

[0027]

[0028] where f enc (·) represents the learning-based transform.

[0029] In Step 3, the bandwidth allocation dynamically adjusts the bandwidth ratio of the digital and analog branches according to the signal-to-noise ratio (SNR);

[0030] The bandwidth allocation adaptively allocates bandwidth resources according to the image complexity;

[0031] When the channel condition reaches a preset limit (such as the signal-to-noise ratio being lower than a certain threshold), the bandwidth allocation automatically switches to the single-branch transmission mode. In this mode, data is transmitted using either the digital or analog branch.

[0032] Step 3 is specifically as follows:

[0033] Under the constraint of the available bandwidth, the feature is passed to the bandwidth allocation module to generate the binary mask MASK C×H′×W′ ;

[0034] The feature F' to be transmitted is determined by the following formula:

[0035] F' = F C×H′×W′ ⊙ MASK C×H′×W′ ,

[0036] where ⊙ represents element-wise multiplication. After that, F' is flattened into a one-dimensional vector F v ' and the elements corresponding to zero in MASK are deleted. Then, for each feature element of F C×H′×W′ ' v a power scaling g is performed, where i = 1, 2,..., N and N ≤ C × H' × W'; i The scaled element

[0037] can be obtained through the following formula:

[0038]

[0039] where P is the total transmission power and ∈ is set to 1 × 10 -10 . Then, each pair of scaled elements is mapped to a complex symbol in the 64K-QAM constellation and transmitted through the original OFDM channel.

[0040] The bandwidth allocation in step 3 is to adapt to different bandwidths through the binary mask MASK C×H′×W′ whose size is constrained by C × H' × W' = 3 × H × W to achieve full-bandwidth transmission.

[0041] The generated mask depends on the number and position of non-zero elements. The number of non-zero elements depends on the available bandwidth, which is defined by the channel bandwidth ratio CR (0 < CR < 1) as:

[0042]

[0043] B is the total consumed bandwidth, that is, the number of transmitted symbols, B ana and B dig respectively represent the bandwidths consumed by the analog and digital branches.

[0044] Using analog bandwidth B ana , MASK C×H′×W′ Each element mask i,j,k It can be obtained through the following methods:

[0045]

[0046] In step 6, at the receiving end of the analog branch, the demodulated receiving vector Interference by noise e:

[0047]

[0048] in

[0049] The step 6 is specifically as follows:

[0050] Perform learning-based inverse transformation and reconstruction of the residual, generated by the following formula:

[0051]

[0052] where f dec (·) is the inverse transformation based on learning, which results in the reconstructed image.

[0053] Beneficial effects of the present invention:

[0054] First, the hybrid digital-analog framework (Hybrid-LJSCC) proposed in this paper improves transmission efficiency by combining the advantages of digital and analog transmission. By introducing a digital branch and utilizing existing digital compression methods (such as BPG) for preliminary compression and prediction, the encoding of the source signal is more efficient. This method significantly reduces the redundant information transmitted in the channel, thereby improving overall transmission efficiency.

[0055] Secondly, the present invention optimizes channel bandwidth utilization while maintaining signal-to-noise ratio (SNR) scalability. By introducing residual signal processing in the analog branch, the invention effectively leverages the prediction generated by the digital branch, ensuring signal integrity and accuracy during transmission. This design enables high-quality image reconstruction even in low SNR environments, demonstrating superior robustness compared to the "cliff effect" of traditional methods.

[0056] Furthermore, experimental results show that Hybrid-LJSCC achieves up to 4dB improvement in peak signal-to-noise ratio (PSNR) over state-of-the-art LJSCC baselines (such as SwinJSCC and DeepJSCC) across multiple datasets and different channel conditions. This not only demonstrates the significant advantages of the present invention in image reconstruction quality, but also demonstrates its applicability and effectiveness in end-to-end communication systems.

[0057] Finally, the hybrid digital-analog design of the present invention offers greater scalability and flexibility. With continued technological advancement, the specific implementations of the digital and analog components can be further optimized based on actual needs to adapt to the demands of diverse application scenarios. This characteristic gives the present invention greater adaptability and potential, enabling it to meet the challenges of future communications technology development. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the process framework of the present invention. DETAILED DESCRIPTION

[0059] The present invention will be described in further detail below with reference to the accompanying drawings.

[0060] like Figure 1 As shown, Figure 1 The framework of the proposed method is shown, which mainly includes two branches: digital transmission branch and analog transmission branch; both branches include sending and receiving respectively, and both branches have a complete transmission process.

[0061] The digital branch uses a separation-based source and channel coding method to generate predictions for the analog branch;

[0062] The prediction generated by the digital branch codec is then subtracted from the input image to simplify subsequent power allocation. The resulting residual is then transmitted through the analog branch;

[0063] The latest LJSCC schemes, such as DeepJSCC and SwinJSCC, can be used for analog transmission to benefit from the powerful nonlinear representation of neural networks. Although higher quality prediction will lead to more improvements in the analog branch, it will also increase the bandwidth of digital transmission and allocate less bandwidth to the analog branch.

[0064] In order to make a better trade-off, a bandwidth allocation module is introduced.

[0065] Step (1): Digital transmission: Using analog transmission alone is not efficient enough. By introducing a digital transmission branch, a predicted value is provided for analog transmission, thereby improving transmission efficiency.

[0066] In the digital branch, the input image I 3×H×WFirst, downsample at a given ratio r (0 < r < 1) to save bandwidth;

[0067] Then, use an image compression algorithm, i.e., image coding (such as BPG, VVC, or advanced end-to-end image compression), to compress the thumbnail with a given quantization parameter (QP) to generate a compressed stream s bin ; QP is the quantization parameter when encoding with existing schemes such as BPG or VVC, generally ranging from 0 to 51

[0068] After decoding and upsampling, obtain the prediction of the analog branch A smaller ratio r and a larger QP will consume less bandwidth but result in a lower quality prediction for the analog branch; conversely, a higher quality prediction will be obtained.

[0069] To transmit the compressed stream s without errors bin , adopt conservative protection, i.e., use a convolutional code with a rate of 1 / 2 for channel coding and QPSK modulation to be compatible with the IEEE802.11a standard.

[0070] Although BPSK is the most robust modulation, QPSK can achieve the exact same bit error rate when the energy of each symbol is doubled, and the bandwidth can be reduced by half compared to BPSK.

[0071] At the receiving end, demodulate the received signal, perform corresponding channel decoding and image decoding, and then use bilinear interpolation for upsampling to restore the predicted value to the original image size.

[0072] Step (2): Analog transmission

[0073] It is difficult for the digital branch to maintain good transmission performance under various channel conditions. By using the analog branch to achieve SNR scalability, the system can maintain good transmission performance under various conditions. Before performing analog transmission, first subtract the predicted value obtained by decoding the digital branch generated by the digital branch from the input image to obtain the residual; then send the obtained residual to a learning-based transform to extract the feature F C×H′×W′ , expressed as:

[0074]

[0075] where f enc (·) represents a learning-based transform; CNN, Transformer, or even more advanced architectures can be used as such transforms;

[0076] Then, with the available bandwidth as a constraint, pass the feature to the bandwidth allocation module to generate a binary mask MASK C×H′×W′ ; Then, the feature F′ to be transmitted is determined by the following formula:

[0077] F′=F C×H′×W′ ⊙MASK C×H′×W′ ,

[0078] Where ⊙ represents the element-wise product. After that, F′ is flattened into a one-dimensional vector F v ' and delete the MASK C×H′×W′ The elements corresponding to zero in F v Each characteristic element of ′ Perform power scaling g i , where i = 1, 2, ..., N and N ≤ C × H′ × W′;

[0079] Scaled elements It can be obtained by the following formula:

[0080]

[0081] Where P is the total transmission power, ∈ is set to 1×10 -10 ,Then, each pair of scaling elements is mapped to a complex symbol in the 64K-QAM constellation and transmitted through the original OFDM channel, thus avoiding the traditional channel coding and modulation used in digital communication; at the receiving end of the analog branch, due to the error generated by the signal during transmission, the demodulated receiving vector Interference by noise e:

[0082]

[0083] in Then, the learning-based inverse transformation and reconstruction of the residual are performed, which is generated by the following formula:

[0084]

[0085] where f dec (·) is the inverse transformation based on learning, which results in the reconstructed image.

[0086] In general, at the analog branch receiving end, the received signal is demodulated and subjected to a learning-based inverse transform to obtain a reconstructed residual, which is added to the predicted value obtained by the digital branch to obtain a reconstructed image.

[0087] Step (3): Bandwidth allocation: In order to achieve a balanced bandwidth for digital transmission and analog transmission, so that the system can efficiently utilize the bandwidth under different channel conditions, a bandwidth allocation module is designed.

[0088] This module aims to adapt the number of extracted features to a given bandwidth constraint. Considering that the digital bandwidth varies with the image content, a binary mask similar to DeepJSCC is introduced. C×H′×W′To adapt to different bandwidths. The mask size is constrained by C×H′×W′ = 3×H×W to achieve full-bandwidth transmission.

[0089] The generated mask depends on two key parameters: the number and position of non-zero elements. The number of non-zero elements depends on the available bandwidth, which is defined by the channel bandwidth ratio CR (0 < CR < 1) as:

[0090]

[0091] B is the total consumed bandwidth, that is, the number of transmitted symbols, B ana and B dig respectively represent the bandwidths consumed by the analog and digital branches. It can be seen that, given B, B ana and B dig is a trade-off. A higher B dig will result in a higher quality prediction but will also cause B ana to decrease.

[0092] In the implementation, BPG is adopted as the image codec. The downsampling rate r and the quantization parameter QP are designed through trial and error to control B dig . Set r = 1 / 64 and QP = 27. The remaining bandwidth B - B dig is allocated to the analog branch B ana . Using the analog bandwidth B ana , each element mask [[ID=三十三]] C×H′×W′ of MASK i,j,k can be obtained through the following method:

[0093]

[0094] This invention proposes a digital-analog hybrid transmission framework, which jointly completes source coding, channel coding, modulation, and transmission through the digital branch and the analog branch. The digital branch can generate a prediction signal based on any compression algorithm, while the analog branch improves the reconstruction quality through residual transmission, enabling the system to efficiently transmit under different channel conditions.

[0095] A joint coding mechanism for the digital and analog branches is proposed. This framework forms a residual by subtracting the predicted value of the digital branch from the image for analog transmission. This residual transmission method can effectively improve the accuracy of image reconstruction and ensure high-quality reconstruction even under low signal-to-noise ratio conditions.

[0096] A bandwidth adaptive allocation mechanism is proposed, and a bandwidth allocation strategy is designed for the digital and analog branches, enabling the system to efficiently utilize bandwidth under different channel conditions and thus optimizing the transmission performance.

[0097] Compatibility with various deep learning and traditional encoding methods: This framework allows the use of various deep learning encoding methods or traditional encoding methods as encoding methods for digital or analog branches, and is not limited to a specific model architecture or encoding algorithm. Any encoding method that combines with this framework and utilizes a hybrid digital-analog strategy is within the scope of this invention.

[0098] The present invention alternative digital-analog hybrid transmission architecture: The present invention currently adopts a two-branch framework (digital and analog branches). An alternative solution is to use other coding methods in the digital branch, such as traditional image coding schemes and deep learning-based image coding schemes, while the analog branch can adopt a more flexible transmission strategy based on the residual image. In addition, Figure 1 The "up sampling" and "down sampling" in the above are not necessary. They are used to match the encoding method in the digital branch and should be considered as a whole with the image encoding method.

[0099] Alternative residual calculation methods: This solution uses the difference between the digital branch's prediction and the original image as the residual for transmission. Other prediction calculation methods can be used, such as applying adaptive filtering to the original image as the prediction value, or dynamically selecting different generation methods based on content characteristics. These alternative calculation methods ensure the analog branch's adaptability to channel variations while improving transmission efficiency and reconstruction quality.

[0100] Adaptive bandwidth allocation strategies: Bandwidth allocation strategies can employ other adaptive mechanisms, such as dynamically adjusting the bandwidth share of digital and analog branches based on the signal-to-noise ratio (SNR) or adaptively allocating bandwidth resources based on image complexity. Under extreme channel conditions, bandwidth allocation strategies can even fully switch to single-branch (digital or analog) transmission mode to accommodate instantaneous transmission requirements. This strategy expands bandwidth allocation flexibility and further improves transmission performance under varying channel conditions.

[0101] Compatible deep learning model alternatives: In addition to existing convolutional neural networks (CNNs), other deep learning models, such as Transformers or self-attention mechanisms, can be used in both digital and analog branches to better capture image features and optimize encoding. These alternative models can also perform joint source-channel encoding and are applicable to different channel conditions and image characteristics.

Claims

1. A source-channel joint coding method for combined digital communication and analog communication, characterized in that: It includes the following steps; Step 1: In the digital branch, the input image is first downsampled and image-encoded, and the resulting bitstream acts on three parts; The first part is to perform image decoding and upsampling to generate a prediction value for use in the analog branch; The second part is to perform channel encoding and modulation, transmit through a wireless channel, and send it to the receiving end; The third part plays a role in the bandwidth allocation of the analog branch; Step 2: In the analog branch, subtract the prediction value generated in Step 1 from the input image to obtain a residual, and pass the residual through a learning-based transformation to obtain the features of the residual; Step 3: According to the proposed bandwidth allocation, use the bitstream in Step 1 and the given bandwidth to generate a mask, and mask the features of the residual in Step 2 to achieve bandwidth allocation; Step 4: Perform power allocation and dense modulation on the features after bandwidth allocation in Step 3, and send them to the receiving end through a wireless channel; Step 5: At the receiving end, the signal received by the digital branch is successively demodulated, channel decoded, image decoded, and upsampled to restore the prediction value; Step 6: Demodulate the signal of the analog branch and perform a learning-based inverse transformation to reconstruct the residual. Finally, add the residual to the prediction value obtained in Step 5 to obtain the reconstructed image; In step 2, the input image is subtracted from the prediction Get the residual, and send the obtained residual to the learning-based transformation to extract the feature F C×H′×W′ , expressed as: where f enc (·) indicates learning-based transformation; In Step 3, the bandwidth allocation dynamically adjusts the bandwidth ratio of the digital and analog branches according to the signal-to-noise ratio (SNR); The bandwidth allocation adaptively allocates bandwidth resources according to the image complexity; When the channel condition reaches the preset limit, the bandwidth allocation automatically switches to the single-branch transmission mode. In this mode, data is transmitted using any one of the digital or analog branches.

2. The method for joint source-channel coding of combined digital and analog communications according to claim 1, wherein: In Step 1, the image encoding in the digital branch uses a deep learning-based method; The image encoding uses JPEG, BPG, or VVC.

3. The method for joint source-channel coding of combined digital and analog communications according to claim 1, wherein: In the said step 1, input image I 3×H×W is downsampled at a given ratio r, where 0 < r < 1, and then the thumbnail is compressed using an image compression algorithm, i.e., image coding, with a given quantization parameter (QP) to generate a compressed stream s bin ; In the first part of step 1, the compressed stream s bin After image decoding and upsampling, the prediction of the simulated branch is obtained 4. The method for joint source-channel coding of combined digital and analog communications according to claim 3, wherein: In the second part of step 1, the compressed stream s bin Channel coding and QPSK modulation are performed using a rate 1 / 2 convolutional code.

5. The method for joint source-channel coding of combined digital and analog communications according to claim 1, wherein: Step 3 is specifically as follows: With the available bandwidth as a constraint, the features are passed to the bandwidth allocation module to generate a binary mask MASK C ×H′×W′ ; The feature F' to be transmitted is determined by the following formula: F′=F C×H′×W′ ⊙MASK C×H′×W′ , Where ⊙ represents the element-wise product. After that, F′ is flattened into a one-dimensional vector F′ v And delete the MASK C×H′×W′ The elements corresponding to zero in F′ v Each characteristic element Perform power scaling g i , where i = 1, 2, ..., N and N ≤ C × H′ × W′; Scaled elements It can be obtained by the following formula: Where P is the total transmission power, ∈ is set to 1×10 -10 ,Then, each pair of scaled elements is mapped to a complex symbol in the 64K-QAM constellation and transmitted through the original OFDM channel; The bandwidth allocation in step 3 is as follows: C×H′×W′ To adapt to different bandwidths, the mask size is constrained by C×H′×W′=3×H×W to achieve full bandwidth transmission.

6. The method for joint source-channel coding of combined digital and analog communications according to claim 5, characterized in that: The generated mask depends on the number and position of non-zero elements. The number of non-zero elements depends on the available bandwidth, which is defined by the channel bandwidth ratio CR (0 < CR < 1) as: B is the total bandwidth consumed, that is, the number of transmitted symbols, ana and B dig Represents the bandwidth consumed by analog and digital branches respectively; Using analog bandwidth B ana , MASK C×H′×W′ Each element mask i,j,k Obtained through:

7. The method for joint source-channel coding of combined digital and analog communications according to claim 5, characterized in that: In step 6, at the receiving end of the analog branch, the demodulated receiving vector Interference by noise e: in 8. The method for joint source-channel coding of combined digital and analog communications according to claim 7, characterized in that: Step 6 is specifically as follows: Perform a learning-based inverse transformation and reconstruction of the residual, which is generated by the following formula: where f dec (·) is the inverse transformation based on learning, which results in the reconstructed image.

Images