Mutual information based receiver reconstruction model training and application method and related apparatus

Abstract

The application discloses a receiving end reconstruction model training and application method based on mutual information and related devices, and relates to the technical field of wireless communication. The method comprises the following steps: designing an initial reconstruction model, which comprises a multi-scale feature extraction module, a mutual information guided fusion module and a reconstruction module connected in sequence; the multi-scale feature extraction module performs feature extraction on a second image sample to obtain second image feature samples of multiple scales; the mutual information guided fusion module respectively calculates mutual information between first image feature samples and second image feature samples of each scale, and fuses the first image feature samples and the second image feature samples of multiple scales based on the mutual information to obtain fusion features; and the reconstruction module reconstructs the first image sample based on the fusion features to obtain a first reconstructed image sample. The initial reconstruction model is trained to obtain a reconstruction model, and further image reconstruction is performed. The application can make full use of side information and improve the performance of the reconstruction model.

Description

Technical Field

[0001] This application relates to the field of wireless communication technology, and in particular to a method and related apparatus for training and applying a receiver reconstruction model based on mutual information. Background Technology

[0002] With the rapid development of intelligent wireless applications, future 5G / 6G wireless communication systems need to maintain efficient and stable communication capabilities under strict bandwidth and reliability constraints. Joint Source-Channel Coding (JSCC) demonstrates significant advantages over traditional source-channel separation designs under finite code length communication conditions by jointly designing data compression and channel transmission processes. Building upon this, DeepJSCC utilizes deep neural networks to achieve end-to-end feature transmission, enabling wireless communication systems to adaptively represent content and exhibit smooth performance degradation characteristics under different channel conditions. Its distributed version (i.e., the distributed DeepJSCC-WZ system) further utilizes relevant side information from the receiver, following the Wyner-Ziv coding paradigm. Here, the transmitter extracts features from the main information (corresponding to the first image), obtaining main information features (corresponding to the first image features), and sends the main information features and side information (corresponding to the second image) to the receiver. The receiver uses the main information features and side information to reconstruct the main information, completing image reconstruction and improving image reconstruction quality while reducing transmitter complexity.

[0003] However, despite the above advantages, side information is still not fully utilized in the distributed DeepJSCC-WZ system, especially under conditions of correlation variation at different scales (i.e., correlation variation between master information and side information) or complex channel conditions. Summary of the Invention

[0004] The purpose of this application is to provide a method and related apparatus for training and applying a receiver reconstruction model based on mutual information, which can make full use of mutual information and improve the performance of the reconstruction model.

[0005] To achieve the above objectives, this application provides the following solution.

[0006] Firstly, this application provides a method for training a receiver reconstruction model, used to train a reconstruction model of the receiver in a distributed DeepJSCC-WZ system. The receiver reconstruction model training method includes: Obtain the dataset; the dataset includes the first image sample collected by the sending end and the first image feature sample and the second image sample received by the receiving end. The first image feature sample is obtained by encoding the first image sample by the sending end. The first image sample and the second image sample are obtained by the sending end taking pictures of the same area from different perspectives. Using a first image feature sample and a second image sample as input, an initial reconstruction model is used to reconstruct the first image sample to obtain a first reconstructed image sample. The initial reconstruction model includes a multi-scale feature extraction module, a mutual information-guided fusion module, and a reconstruction module connected in sequence. The multi-scale feature extraction module is used to extract features from the second image sample to obtain second image feature samples at multiple scales. The mutual information-guided fusion module is used to calculate the mutual information between the first image feature sample and the second image feature sample at each scale, and to fuse the first image feature sample and the second image feature samples at multiple scales based on the mutual information to obtain fused features. The reconstruction module is used to reconstruct the first image sample based on the fused features to obtain the first reconstructed image sample. The total loss is calculated based on the first image sample, the first reconstructed image sample, the first image feature sample, and the second image feature samples at multiple scales. The initial reconstruction model is updated using the total loss to obtain the updated reconstruction model; Determine if the iteration termination condition has been met; if yes, use the updated reconstruction model as the receiving reconstruction model; if no, use the updated reconstruction model as the initial reconstruction model for the next iteration and return to the "Get Dataset" step; the reconstruction model is used for image reconstruction.

[0007] Secondly, this application provides a method for applying a receiver reconstruction model, the method comprising: Receive first image features and second image; the first image features are obtained by encoding the first image using the transmitting end, and the first image and second image are obtained by the transmitting end taking pictures of the same area from different perspectives; Using the features of the first image and the second image as input, the first image is reconstructed using a reconstruction model to obtain the first reconstructed image; wherein, the reconstruction model is trained using the aforementioned receiver reconstruction model training method.

[0008] Thirdly, this application provides a receiving end for executing the above-described receiving end reconstruction model application method.

[0009] Fourthly, this application provides a distributed communication system, which includes: a transmitter and a receiver; The sending end is used to acquire a first image and a second image, encode the first image using an encoding model to obtain the first image features, and transmit the first image features and the second image to the receiving end; The receiving end is used to execute the above-described receiving end reconstruction model application method.

[0010] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described receiver reconstruction model training method or the above-described receiver reconstruction model application method.

[0011] According to the specific embodiments provided in this application, this application has the following technical effects.

[0012] This application provides a receiver reconstruction model training and application method and related apparatus based on mutual information. Considering that a key limitation of the distributed DeepJSCC-WZ system lies in the insufficient utilization of side information, i.e., side information is not fully utilized, this is because most methods at the receiver, when using main information features and side information to reconstruct the main information, only process at a single scale (i.e., directly concatenating the main information features and side information, and using the concatenated information to reconstruct the main information), or rely on a fixed attention mechanism (i.e., first using an attention mechanism to extract features from the side information, then concatenating the main information features and side information features, and using the concatenated information to reconstruct the main information). When the correlation changes at different scales or under complex channel conditions, the above methods may excessively suppress detailed information or amplify noise in low-correlation regions. Considering that this problem is closely related to key factors in information theory, such as mutual information and rate-distortion tradeoffs, but these key factors are rarely explicitly utilized, this application addresses the network of the reconstruction model. The structure is innovatively designed. The initial reconstruction model includes a multi-scale feature extraction module, a mutual information-guided fusion module, and a reconstruction module connected in sequence. The multi-scale feature extraction module extracts features from the second image sample to obtain second image feature samples at multiple scales. The mutual information-guided fusion module calculates the mutual information between the first image feature sample and the second image feature sample at each scale, and fuses the first image feature sample and the second image feature samples at multiple scales based on the mutual information to obtain fused features. The reconstruction module reconstructs the first image sample based on the fused features to obtain the first reconstructed image sample. The initial reconstruction model is trained to obtain a reconstruction model. This reconstruction model takes the first image features and the second image received by the receiving end as input to reconstruct the first image to obtain the first reconstructed image. Multi-scale feature extraction and mutual information calculation can be performed during the image reconstruction process. Based on the mutual information, features at different scales are fused to further complete the image reconstruction, thereby making full use of mutual information and improving the performance of the reconstruction model. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is an application environment diagram for a receiver reconstruction model training method provided in Embodiment 1 of this application.

[0015] Figure 2 This is a flowchart illustrating a receiver reconstruction model training method provided in Embodiment 1 of this application.

[0016] Figure 3 This is a schematic diagram of the distributed semantic communication system model provided in Embodiment 1 of this application.

[0017] Figure 4 This is a schematic diagram showing the connection between the encoding model, reconstruction model, and task model provided in Embodiment 1 of this application.

[0018] Figure 5 This is a schematic diagram showing a comparison of reconstruction performance on a portion of the CIFAR-10 dataset under different channel bandwidth ratios and signal-to-noise ratios, as provided in Embodiment 1 of this application.

[0019] Figure 6 This is a schematic diagram showing another part of the reconstruction performance comparison on the CIFAR-10 dataset under different channel bandwidth ratios and signal-to-noise ratios provided in Embodiment 1 of this application.

[0020] Figure 7 This diagram illustrates the comparison of reconstruction performance on the CIFAR-10 dataset under the ablation experiment provided in Embodiment 1 of this application.

[0021] Figure 8 This diagram illustrates the comparison of reconstruction performance on the ImageNet32 dataset under the ablation experiment provided in Embodiment 1 of this application.

[0022] Figure 9 This is a flowchart illustrating a receiving end reconstruction model application method provided in Embodiment 2 of this application.

[0023] Figure 10 This is a schematic diagram of the structure of a computer device provided in Embodiment 5 of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Example 1 The receiver reconstruction model training method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown is as follows. The terminal communicates with the server via a network. A data storage system stores the data the server needs to process. This data storage system can be set up independently, integrated into the server, or placed in the cloud or on another server. The terminal can send a training request to the server. Upon receiving the request, the server acquires a dataset, which includes a first image sample collected by the sending end and a first image feature sample and a second image sample received by the receiving end. Using the first image feature sample and the second image sample as input, an initial reconstruction model is used to reconstruct the first image sample, resulting in a first reconstructed image sample. The initial reconstruction model includes a multi-scale feature extraction module, a mutual information-guided fusion module, and a reconstruction module connected in sequence. The multi-scale feature extraction module extracts features from the second image sample to obtain second image feature samples at multiple scales. The mutual information-guided fusion module calculates the first image feature sample and each scale separately. The mutual information between the second image feature samples is used to fuse the first image feature sample and the second image feature samples at multiple scales based on the mutual information, resulting in fused features. The reconstruction module reconstructs the first image sample based on the fused features, resulting in a first reconstructed image sample. Based on the first image sample, the first reconstructed image sample, the first image feature sample, and the second image feature samples at multiple scales, the total loss is calculated. The initial reconstruction model is updated using the total loss, resulting in an updated reconstruction model. The iteration termination condition is then determined. If yes, the updated reconstruction model is used as the reconstruction model for the receiving end. If no, the updated reconstruction model is used as the initial reconstruction model for the next iteration, and the process returns to the "Get Dataset" step. The reconstruction model is then used for image reconstruction. The server can feed back the training result—the reconstruction model obtained for the training request—to the terminal.

[0026] In addition, in some embodiments, the receiving end reconstruction model training method can also be implemented by the server or the terminal alone. For example, the terminal can directly process the training request to be processed, or the server can obtain the training request to be processed from the data storage system and process it.

[0027] The terminals can be, but are not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices, while portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Servers can be implemented using independent servers, server clusters composed of multiple servers, or cloud servers.

[0028] In one exemplary embodiment, such as Figure 2 As shown, a method for training a receiver reconstruction model is provided to train a reconstruction model for the receiver in a distributed DeepJSCC-WZ system. This method is executed by a computer device, specifically by a terminal or server alone, or by both. In this embodiment, the method is applied to... Figure 1 The following steps, S1-S5, are used as an example to illustrate the process of using a server in the example.

[0029] Step S1: Obtain the dataset; The dataset includes a first image sample collected by the transmitting end and a first image feature sample and a second image sample received by the receiving end. The first image feature sample is obtained by encoding the first image sample by the transmitting end. The first image sample and the second image sample are obtained by the transmitting end taking pictures of the same area from different perspectives.

[0030] Step S2: Using the first image feature sample and the second image sample as input, the first image sample is reconstructed using the initial reconstruction model to obtain the first reconstructed image sample. The initial reconstruction model includes a multi-scale feature extraction module, a mutual information guided fusion module, and a reconstruction module connected in sequence. The multi-scale feature extraction module is used to extract features from the second image sample to obtain second image feature samples at multiple scales. The mutual information guided fusion module is used to calculate the mutual information between the first image feature sample and the second image feature sample at each scale, and to fuse the first image feature sample and the second image feature samples at multiple scales based on the mutual information to obtain fused features. The reconstruction module is used to reconstruct the first image sample based on the fused features to obtain the first reconstructed image sample.

[0031] Step S3: Calculate the total loss based on the first image sample, the first reconstructed image sample, the first image feature sample, and the second image feature samples at multiple scales.

[0032] Step S4: Update the initial reconstruction model using the total loss to obtain the updated reconstruction model.

[0033] Step S5: Determine whether the iteration termination condition has been met; if yes, use the updated reconstruction model as the reconstruction model of the receiving end; if no, use the updated reconstruction model as the initial reconstruction model for the next iteration and return to the "Get Dataset" step; the reconstruction model is used for image reconstruction.

[0034] By implementing steps S1 to S5 above, this embodiment can train a reconstruction model for the receiver in the distributed DeepJSCC-WZ system. This reconstruction model can make full use of side information and improve reconstruction performance.

[0035] Below, we will first take a binocular camera system as an example to construct a model of a distributed semantic communication system (specifically, a distributed DeepJSCC-WZ system), such as... Figure 3 As shown, this distributed semantic communication system mainly consists of a transmitter (including binocular cameras), a wireless channel, and a receiver (which can be a base station). In this distributed semantic communication system, the left camera in the binocular cameras acquires the first image sample. , Main information ， 、 and These represent the number of image channels, width, and height, respectively. The right camera in the binocular camera system captures a second image sample. , For side information, the second image sample is an image that is highly related to the first image sample. For example, the first image sample and the second image sample are two images obtained by taking pictures of the same area from different perspectives. The first image sample and the second image sample partially overlap. The side information is only available at the base station, which is the receiving end. This setting is consistent with the classic Wyner-Ziv distributed source coding scenario.

[0036] At the sending end, the first image sample First, the model is mapped using an encoding model (which can be an encoder, specifically a neural network, for example, consisting of multiple sequentially connected residual blocks, each containing convolutional layers; this is a mature existing technology and will not be elaborated upon here). Victory potential representation , For constants: ; in, For the coding model, These are the parameters of the encoding model.

[0037] To meet the power constraints of wireless communication systems, the encoded... Victory potential representation Normalization is performed: ; in, This represents the average power. The normalization process described above is a mature existing technology and will not be elaborated upon here.

[0038] Encoded Victory potential representation Transmitted via wireless channel to the receiving end.

[0039] The wireless channel can use an additive white Gaussian noise channel (AWGN). In this case, the first image feature sample and the second image sample received by the receiver are: ; ; in, The first image feature sample received by the receiving end; It is independent and identically distributed complex Gaussian noise. , For variance, It is the identity matrix. This corresponds to the number of symbols transmitted, with noise added to each symbol; This is the second image sample received by the receiving end. At this point, the signal-to-noise ratio (SNR) of the wireless channel is defined as... The unit is dB.

[0040] For simplicity, some pixels can be removed directly from the lower right corner of the second image sample to obtain the second image sample received by the receiver, without introducing complex Gaussian noise.

[0041] At the receiving end, the base station utilizes the received first image feature samples. (with noise) and second image sample (With noise) Jointly perform image reconstruction: ; in, This is the first reconstructed image sample; To reconstruct the model, These are the parameters for reconstructing the model.

[0042] Another key limitation of the distributed DeepJSCC-WZ system is that the current training strategy is mainly aimed at image reconstruction, with a low degree of coupling with downstream semantic tasks. Therefore, it is difficult to achieve optimal task performance in harsh channel environments. In order to improve reconstruction performance, most systems use neural networks with larger parameter numbers as encoding models to better extract main information features, but this increases the complexity of the transmitter and the alignment with downstream semantic tasks is still insufficient, resulting in the underutilization of multi-scale semantic correlations in side information.

[0043] To address this issue, the first reconstructed image sample is then input into the task model to perform downstream semantic tasks: ; in, Predict results for task classification; For the task model, These are the parameters of the task model.

[0044] Based on the aforementioned distributed semantic communication system, this embodiment further provides a method for training a receiver reconstruction model. The bandwidth efficiency of the wireless communication system is measured by the channel bandwidth ratio. This metric represents the number of channel symbols consumed per pixel. The goal of this embodiment is to maximize the semantic transmission efficiency of the wireless communication system by jointly optimizing image reconstruction quality and downstream semantic task performance under given bandwidth constraints.

[0045] Based on this, this embodiment proposes the DeepJSCC-LZ-MI framework, a task-oriented distributed DeepJSCC-WZ framework based on mutual information (or mutual information driven). It is a distributed semantic communication framework for the Wyner–Ziv distributed semantic communication scenario. The core idea of ​​this framework is to adaptively fuse side information features across multiple spatial scales based on mutual information correlation, rather than relying on a fixed single-scale fusion method. Simultaneously, this embodiment proposes an end-to-end training strategy with a progressive signal-to-noise ratio scheduling mechanism, enabling the reconstruction model to optimize image reconstruction quality while also considering the performance of downstream semantic tasks, thereby more effectively utilizing side information in complex channel environments.

[0046] Figure 4This embodiment demonstrates the overall structure of the DeepJSCC-LZ-MI framework proposed in this embodiment. This framework is used in the Wyner–Ziv distributed semantic communication scenario and consists of two parts: a main information path and a side information path used only at the receiver. To more effectively utilize side information and improve the robustness of the wireless communication system in complex channel environments, the DeepJSCC-LZ-MI framework introduces two key components: (1) A mutual information-guided multi-scale feature aggregation module (MIEGA) is proposed: This module selectively (or adaptively) fuses side information features based on the mutual information correlation between different resolutions, thereby preserving spatial details while suppressing noise, and with almost no additional computational overhead. Compared with simple correlation or attention mechanisms, mutual information can better characterize statistical dependencies, thus providing a more theoretical basis for fusion. (2) A task-oriented end-to-end training strategy (STEGO) is proposed: the robustness of wireless communication system under different channel conditions is improved by progressive signal-to-noise ratio scheduling mechanism. During the training process, the image reconstruction quality and downstream semantic task performance are optimized at the same time, so as to obtain better reconstruction performance under low signal-to-noise ratio conditions, and at the same time, there is no need to retrain the reconstruction model separately for each signal-to-noise ratio.

[0047] The two designs described above enable the reconstruction model to achieve feature fusion that combines information theory-driven and task-oriented approaches with minimal additional complexity.

[0048] The following provides a detailed introduction to the two designs mentioned above. Before calculating mutual information, all features are channel-normalized (zero mean, unit variance) to avoid bias caused by scale differences. During mutual information calculation, a Gaussian correlation approximation method is used, assuming feature pairs within local regions. It can be approximated by a joint Gaussian distribution. This is the first image feature sample after size transformation. The scale after size transformation The second image feature sample is resized to ensure that the dimensions are the same, facilitating subsequent processing and thus obtaining a closed-form mutual information estimation expression. To ensure numerical stability, a small constant is introduced when division or logarithmic operations are involved. , .

[0049] (I) MIEGA: Mutual Information Enhanced Multi-Scale Feature Aggregation Side information typically contains multiple levels of visual information, such as fine-grained texture (edges, small objects), mid-level structural information, and global semantic information. If feature fusion is performed only at a single scale, it may over-suppress detailed information or amplify noise. Therefore, this embodiment extracts side information features from multiple different spatial scales and uses mutual information to guide the fusion.

[0050] In this embodiment, the initial reconstruction model includes a multi-scale feature extraction module, a mutual information-guided fusion module, and a reconstruction module connected in sequence. The multi-scale feature extraction module is used to extract features from the second image sample to obtain second image feature samples at multiple scales. The mutual information-guided fusion module is used to calculate the mutual information between the first image feature sample and the second image feature sample at each scale, and to fuse the first image feature sample and the second image feature samples at multiple scales based on the mutual information to obtain fused features. The reconstruction module is used to reconstruct the first image sample based on the fused features to obtain the first reconstructed image sample.

[0051] (1) Multi-scale feature extraction Lateral information is used to generate features at different scales through a bottom-up feature extraction path.

[0052] In this embodiment, the multi-scale feature extraction module includes a first feature extraction unit and a plurality of second feature extraction units connected in sequence. The input end of the first feature extraction unit is the input end of the multi-scale feature extraction module, used to input second image samples. The output end of the first feature extraction unit and the output end of each of the second feature extraction units are both output ends of the multi-scale feature extraction module, used to output second image feature samples at multiple scales. The first feature extraction unit and each of the second feature extraction units respectively output second image feature samples at one scale.

[0053] The first feature extraction unit includes a first fully connected layer and a first activation function layer connected in sequence. The input of the first fully connected layer is the input of the first feature extraction unit, and the output of the first activation function layer is the output of the first feature extraction unit.

[0054] The second feature extraction unit includes a max pooling layer, a second fully connected layer, and a second activation function layer connected in sequence. The input of the max pooling layer is the input of the second feature extraction unit, and the output of the second activation function layer is the output of the second feature extraction unit.

[0055] Taking a scale of 3 as an example, the multi-scale feature extraction module includes one first feature extraction unit and two second feature extraction units: ; ; ; Among them, superscript These represent the feature map resolution, signifying different scales. for It mainly contains texture information. for This mainly includes mid-level structural features. for It mainly includes global semantic information; The second image feature sample at scale 1 is output by the first feature extraction unit; Since it is a nonlinear function, activation functions such as ReLU or SiLU can be used; These are the weights of the first fully connected layer in the first feature extraction unit; The bias of the first fully connected layer in the first feature extraction unit; The second image feature sample at scale 2 is output by the first second feature extraction unit; These are the weights of the second fully connected layer in the first second feature extraction unit; This is a 2×2 max pooling operation; The bias of the second fully connected layer in the first second feature extraction unit; The second image feature sample at scale 3 is output by the second second feature extraction unit; These are the weights of the second fully connected layer in the second feature extraction unit. This is the bias of the second fully connected layer in the second feature extraction unit.

[0056] (2) Mutual information-guided integration In this embodiment, the mutual information guided fusion module includes a mutual information calculation unit and a first fusion unit connected in sequence. The input end of the mutual information calculation unit is the input end of the mutual information guided fusion module, and the output end of the first fusion unit is the output end of the mutual information guided fusion module.

[0057] 1) Mutual Information Calculation Unit The mutual information calculation unit is used to calculate the mutual information between the first image feature sample and the second image feature sample at each scale, and obtain the mutual information corresponding to the second image feature sample at each scale.

[0058] First, calculate the correlation coefficient: ; in, For the first image feature sample and scale The correlation coefficient between the second image feature samples, ; These are the first image feature samples after normalization and size transformation; The scale after normalization and size transformation The second image feature sample. and They are the same size, and their pixel values ​​are normalized.

[0059] Under the local Gaussian assumption, the mutual information between two features can be estimated using the following formula: ; in, For scale The mutual information corresponding to the second image feature samples, along with Monotonically increasing; It is a constant, for the sake of data stability.

[0060] It should be noted that before calculating mutual information, the first image feature samples and the second image feature samples at multiple scales need to be normalized and resized to ensure they are of the same size. Normalization and resizing are existing mature techniques and will not be elaborated here. Taking a scale of 3 as an example, the size of the main information feature is [B, C, 4, 4], where B is the batch size. The sizes of the secondary information features are [B, C, 16, 16], [B, C, 8, 8], and [B, C, 4, 4]. After spatial alignment and channel alignment, the final size is [B, C, 4, 4]. At this point, the main information feature is... Side information features are If the dimensions are the same, and normalization is performed, then mutual information can be calculated further.

[0061] 2) First Fusion Unit In this embodiment, the first fusion unit is used to calculate the gating weights corresponding to the second image feature samples at each scale based on the mutual information of the second image feature samples at each scale, and to perform a weighted summation of the first image feature samples and the second image feature samples at multiple scales based on the gating weights to obtain the fused features.

[0062] First, convert mutual information into gated weights: ; in, For scale The gating weights corresponding to the second image feature samples; It is a nonlinear function; For scale The learnable threshold parameter corresponding to the second image feature sample.

[0063] Then, based on the gating weights, the fusion is performed, and the formula for calculating the fused features is: ; in, Features of fusion; This is the first image feature sample after size transformation; The scale after size transformation The second image feature sample. The residual structure in the above formula for calculating the fused features only injects feature differences that have informational value.

[0064] (3) FPN Feature Pyramid Fusion In this embodiment, the reconstruction module includes a second fusion unit and a reconstruction unit connected in sequence. The input of the second fusion unit is the input of the reconstruction module, and the output of the reconstruction unit is the output of the reconstruction module.

[0065] The second fusion unit is used to calculate multi-scale fusion features based on second image feature samples and fusion features at multiple scales.

[0066] The second fusion unit includes multiple feature fusion sub-units connected in sequence. Each feature fusion sub-unit corresponds one-to-one with a second image feature sample at multiple scales. The input of the first feature fusion sub-unit is the input of the second fusion unit, used to input the fused features after size transformation. The input of the s-th feature fusion sub-unit is connected to the output of the (s-1)-th feature fusion sub-unit. The output of the S-th feature fusion sub-unit is the output of the second fusion unit, used to output multi-scale fused features. s = 2, 3, ..., S, where S is the number of scales.

[0067] The feature fusion subunit includes an upsampling layer, an attention layer, and an addition layer. The input of the upsampling layer is the input of the feature fusion subunit. The input of the attention layer is connected to the output of the upsampling layer. The first input of the addition layer is connected to the output of the attention layer. The second input of the addition layer is used to input the sum value feature at the corresponding scale (the scale corresponding to the feature fusion subunit). The sum value feature at the corresponding scale is the sum of the fusion feature after size transformation and the second image feature sample at the corresponding scale after size transformation. The output of the addition layer is the output of the feature fusion subunit.

[0068] Taking a scale of 3 as an example, the second fusion unit includes three feature fusion sub-units: ; ; ; in, The summation features are the output of the first feature fusion subunit; The sum value feature is the input to the first feature fusion subunit. It is the sum of the fusion feature after size transformation and the second image feature sample after size transformation at scale 3. The fusion feature after size transformation and the second image feature sample after size transformation at scale 3 have the same size. This is a lightweight channel attention module, specifically an attention layer. =4, 3, 2 The attention layer for the first feature fusion subunit; This is a 2x upsampling; Fusion features after size transformation The summation feature is the output of the second feature fusion subunit; The sum value feature input to the second feature fusion subunit is the sum of the fusion feature after size transformation and the second image feature sample after size transformation at scale 2. The fusion feature after size transformation and the second image feature sample after size transformation at scale 2 have the same size. The attention layer for the second feature fusion subunit; The summation feature output by the third feature fusion subunit is the multi-scale fusion feature. The sum value feature input to the third feature fusion subunit is the sum of the fusion feature after size transformation and the second image feature sample after size transformation at scale 1. The fusion feature after size transformation and the second image feature sample after size transformation at scale 1 have the same size. This is the attention layer for the third feature fusion subunit.

[0069] In this way, high-resolution features can fuse semantic information from low-resolution features while maintaining clear edges.

[0070] It should be noted that the second reconstruction unit in the above-mentioned multi-scale feature extraction module, mutual information-guided fusion module, and reconstruction module constitutes MIEGA.

[0071] MIEGA's inputs are primary information features and lateral information. The output is a multi-scale fused feature. The workflow is as follows: from side information Extract scale set Side information features on each scale; Perform the following operations (normalization and size transformation first): calculate the correlation coefficient, then calculate the mutual information, and finally calculate the gating weights. Based on the gating weights, perform feature fusion to obtain fused features, then construct pyramid features to obtain and return multi-scale fused features. .

[0072] To complete the image reconstruction, based on MIEGA, the reconstruction units in the reconstruction module are further designed to form the initial reconstruction model.

[0073] (4) Reconstruction Unit The reconstruction unit is used to reconstruct the first image sample based on multi-scale fusion features to obtain the first reconstructed image sample.

[0074] The reconstruction unit includes a decoder, which can be any existing decoder.

[0075] In this embodiment, the receiving end reconstruction model training method includes the following steps: (1) Obtain the dataset. The dataset includes the first image sample collected by the transmitter and the first image feature sample and the second image sample received by the receiver. The first image feature sample is obtained by encoding the first image sample by the transmitter. The first image sample and the second image sample are obtained by the transmitter taking pictures of the same area from different perspectives.

[0076] (2) Using the first image feature sample and the second image sample as input, the first image sample is reconstructed using the initial reconstruction model to obtain the first reconstructed image sample.

[0077] (3) The total loss is calculated based on the first image sample, the first reconstructed image sample, the first image feature sample and the second image feature sample at multiple scales.

[0078] The total loss is calculated based on the first image sample, the first reconstructed image sample, the first image feature sample, and the second image feature samples at multiple scales. Specifically, the image reconstruction loss is calculated based on the first image sample and the first reconstructed image sample, the mutual information loss is calculated based on the first image sample, the first reconstructed image sample, the first image feature sample, and the second image feature samples at multiple scales, and the image reconstruction loss and the mutual information loss are weighted and summed to obtain the total loss.

[0079] (4) Update the initial reconstruction model using the total loss to obtain the updated reconstruction model.

[0080] (5) Determine whether the iteration termination condition has been met; if yes, use the updated reconstruction model as the reconstruction model of the receiving end; if no, use the updated reconstruction model as the initial reconstruction model of the next iteration and return to the "Get Dataset" step.

[0081] The iteration termination condition can be either the current iteration count reaching the maximum iteration count or the total loss being less than a preset value.

[0082] (II) STEGO: Task-Oriented Joint Optimization The STEGO training strategy introduces channel conditions of varying difficulty step by step, allowing the reconstruction model to learn from relatively easy communication environments (or channel environments) and then gradually transition to more challenging ones. This encourages the reconstruction model to rely more heavily on lateral information and mutual information gating mechanisms when the quality of primary information deteriorates. During training, the image reconstruction quality and downstream semantic task performance are optimized simultaneously, and the feature flow is regularized using mutual information constraints.

[0083] In this embodiment, the dataset also includes the actual task classification results of the first image sample. Based on the first image sample, the first reconstructed image sample, the first image feature sample, and second image feature samples at multiple scales, the total loss is calculated, specifically including the following steps: (1) Using the first reconstructed image sample as input, the prediction task classification result of the first image sample is obtained by using the task model.

[0084] The task model can use any existing machine learning model, such as a convolutional neural network, to classify the first reconstructed image sample and determine the predicted category of the first reconstructed image sample. The predicted category is the predicted task classification result, and the true category is the true task classification result. The specific category can be designed by the user, and this embodiment does not impose any restrictions on it.

[0085] (2) Based on the first image sample and the first reconstructed image sample, the image reconstruction loss is calculated. Based on the first image sample, the first reconstructed image sample, the first image feature sample and the second image feature sample at multiple scales, the mutual information loss is calculated. Based on the real task classification result and the predicted task classification result, the semantic task loss is calculated.

[0086] (3) The image reconstruction loss, mutual information loss and semantic task loss are weighted and summed to obtain the total loss.

[0087] Total loss is defined as: ; in, Total loss; The weights for the image reconstruction loss; Image reconstruction loss; The weights are used for the semantic task loss. For semantic task loss; Weights for mutual information loss; The mutual information loss is represented by a regularization term.

[0088] (1) Image reconstruction loss To enable the decoder to simultaneously incorporate both low-frequency structural information and high-frequency detail information, this embodiment combines pixel loss, perceptual loss, and structural similarity loss, defining the image reconstruction loss as: ; in, This is the first image sample; This is the first reconstructed image sample; It is the first weight; Weights for perceived loss; To calculate the perceptual loss on a fixed feature extraction network, a fixed feature extraction network (e.g., VGG) is used to extract features from the first image sample and the first reconstructed image sample, respectively, to obtain the first feature and the second feature. The L1 distance or L2 distance between the first feature and the second feature is then calculated to obtain the perceptual loss. Weights for structural similarity loss The structural similarity between the first image sample and the first reconstructed image sample is calculated using an existing formula, which will not be elaborated here, and is used to avoid overly smoothed reconstruction results under low signal-to-noise ratio conditions.

[0089] (2) Semantic task loss The semantic task loss is used to encourage the reconstruction model to preserve the correct semantic information in the reconstructed image. The semantic task loss is defined as: ; in, The expected value is used to calculate the average value. This represents the predicted probability that the task classification result of the task model for the first reconstructed image sample is the true task classification result. This represents the classification result for a real-world task. This loss function ensures that the reconstructed image not only closely approximates the original image in visual quality but also maintains semantic consistency.

[0090] (3) Mutual information constraints To introduce information-theoretic constraints between different feature flows, mutual information regularization terms are introduced at two levels, and the mutual information loss is defined as: ; in, Used to encourage the first reconstructed image sample Preserve the first image sample The amount of information, thus serving as a prior constraint for information preservation, is used to improve image reconstruction results; This is used to encourage a strong statistical correlation between primary and secondary information features, but reinforcement is only applied when these correlations actually provide useful information. This prevents secondary information from being reinforced in the same way as primary information, in order to maximize the useful difference information provided by secondary information. As the second weight, This refers to various scales The pixel values ​​of the same pixel are first added together, and then the average value is calculated.

[0091] ; ; in, for and Mutual information between them As the first variable, As the second variable, and All are normalized and have the same size; for and The correlation coefficient between them; It is a constant.

[0092] make Take the normalized and size-transformed , Take the normalized and size-transformed The above formula can be used to calculate the result. Similarly, let Take the normalized and size-transformed , Take the normalized and size-transformed The above formula can be used to calculate the result. .

[0093] In practical applications, gradient propagation is stopped in the mutual information estimator, meaning there are no learnable network parameters when calculating mutual information. This prevents the reconstruction model from obtaining false mutual information gains by simply amplifying feature magnitudes. Simultaneously, the parameters are used to... Controlling the information contribution between different scales helps prevent the reconstruction model from over-relying on features at a single scale.

[0094] This embodiment further introduces a progressive signal-to-noise ratio (SNR) training strategy: To improve the robustness of the reconstruction model under different channel conditions, this embodiment adopts a progressive SNR training strategy. During the training process, the average SNR changes as a function. The formula for calculating the target SNR is: ; in, The current iteration number The corresponding target signal-to-noise ratio; This represents the minimum signal-to-noise ratio. This represents the maximum signal-to-noise ratio. These are parameters used to control the decay rate; This represents the current iteration number; This represents the maximum number of iterations.

[0095] This progressive signal-to-noise ratio (SNR) training strategy results in a high SNR in the early stages of training, thus stabilizing the basic reconstruction capability of the reconstruction model. As training progresses, the SNR gradually decreases, enabling the reconstruction model to learn how to utilize side information and mutual information gating mechanisms under more complex channel conditions. In the case of Rayleigh fading channels, it is assumed that channel state information (CSI) can be obtained or estimated for each batch for equalization processing. Specifically, for Rayleigh fading channels, known or estimated CSI can be used for MMSE equalization during training. Alternatively, CSI can be used as an additional input and injected into the decoder through affine modulation, which is a mature existing technology and will not be elaborated here. In the case of AWGN channels, CSI is not required.

[0096] In this embodiment, obtaining the dataset specifically includes the following steps: (1) Obtain the initial dataset, which includes the first image sample and the second image sample collected by the sending end.

[0097] (2) Encode the first image sample using the encoding model of the sending end to obtain the first image feature sample.

[0098] The encoding model uses an encoder, which can be any existing encoder.

[0099] (3) The first image feature sample and the second image sample are transmitted to the receiving end through a wireless channel with a target signal-to-noise ratio by the transmitting end, and the first image feature sample and the second image sample received by the receiving end are obtained to obtain the dataset.

[0100] The target signal-to-noise ratio is determined based on the current iteration number.

[0101] STEGO's inputs are datasets and networks. The output is a network with optimized network parameters. The workflow is as follows: Initialize network parameters and set parameters When the model has not converged, execute the following loop: randomly sample training samples. Calculate the current target signal-to-noise ratio Calculated through channel model , The representative channel, if it is a Rayleigh channel, is used simultaneously for equalization via CSI. The classification results for the prediction task are calculated, and the total loss is calculated. and update network parameters When the model converges, the optimized network parameters are output, and the network with the optimized parameters is returned. Of course, you can also choose not to train. .

[0102] The following stability analysis is performed: when the correlation between the side information and the main information features is weak (i.e. (smaller), gating weight This will automatically decrease, causing the calculation formula for fused features to degenerate into an identity mapping, thereby avoiding negative transfer. When the feature weights at a certain scale are too large, the semantic task loss will be penalized by semantic errors, while the mutual information loss will be reduced by the parameters. This involves distributing attention across multiple scale features. Furthermore, it introduces [a concept / mechanism] into the mutual information calculation formula. Used to avoid when The logarithmic function sometimes exhibits numerical instability. In practical applications, the correlation coefficient is often trimmed to meet certain requirements. ， in It is a very small constant.

[0103] The following embodiment uses experiments to verify the performance of the reconstruction model.

[0104] (1) Experimental setup This embodiment conducts experiments on two datasets: 1) CIFAR-10: containing 50,000 training images and 10,000 test images, with a resolution of [missing information]. 2) ImageNet32: Contains approximately 1.28 million training images and 50,000 test images, covering 1000 categories. Each training image undergoes random augmentation, such as random rotation or cropping, to obtain an augmented image. The training image is used as the primary information, and the augmented image is used as secondary information. These two datasets represent standard-sized and large-scale dataset scenarios, respectively. Top-1 classification accuracy is reported on CIFAR-10, and Top-5 classification accuracy is reported on ImageNet32.

[0105] Side information settings: Follow the communication model where only the receiving end has side information.

[0106] Baseline Method: This embodiment uses DeepJSCC-LZ as a self-designed baseline model. This model does not contain any enhancement modules. Compared with DeepJSCC-WZ, the structure of the coding model is changed. The designed coding model includes multiple residual blocks connected in sequence. It is compared with the following methods: DeepJSCC, DeepJSCC-WZ, and DeepJSCC-LZ-MI proposed in this embodiment.

[0107] Channel Models and Evaluation Metrics: Two channel models were considered in the experiments: AWGN (Additive White Gaussian Noise) and Rayleigh fading channel, to evaluate the robustness of the wireless communication system under static and dynamic wireless channel environments. Evaluation metrics included reconstruction quality metrics, semantic task performance metrics, and metrics for evaluating computational efficiency. Reconstruction quality metrics included PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index). Semantic task performance metrics included Top-1 classification accuracy in CIFAR-10 and Top-5 classification accuracy in ImageNet32; accuracy was used for all subsequent evaluations. Metrics for evaluating computational efficiency included the number of model parameters and floating-point operations (FLOPs).

[0108] Training settings: All models were trained using the AdamW optimizer: Learning rate: Batch size: 256, number of training epochs: 320. Cosine learning rate decay and progressive SNR training strategy were used during training. The default channel was AWGN; Rayleigh fading channel results are presented in the ablation experiments. All methods used the same data loading method, side information settings, and training configuration to ensure experimental fairness.

[0109] (2) Overall performance 1) Image reconstruction quality Figure 5 This paper presents a comparison of PSNR and SSIM on the CIFAR-10 dataset under different channel bandwidth ratios and signal-to-noise ratios (SNRs). The channel model is AWGN. Experimental results show that DeepJSCC-LZ-MI achieves higher PSNR and SSIM in all experimental settings, demonstrating a significant advantage over DeepJSCC and DeepJSCC-WZ. As the SNR decreases or bandwidth becomes further limited, the performance gap between different methods widens, indicating that the mutual information-guided multi-scale feature fusion mechanism can still effectively utilize side information even when received features are severely distorted. ImageNet32 classification results are... Figure 8 The ablation experiments presented confirm that a similar trend was observed in large-scale dataset experiments. Even under low signal-to-noise ratio conditions, DeepJSCC-LZ-MI still maintains better reconstruction performance, indicating that information-aware feature injection strategies still have significant advantages for complex visual data.

[0110] 2) Task-oriented semantic performance Figure 6This paper presents a comparison of accuracy and PSNR on the CIFAR-10 dataset under different channel bandwidth ratios and signal-to-noise ratios, with the channel model being AWGN. Figure 6 The signal-to-noise ratio (SNR) of (c) and (d) is 5 dB. Experimental results show that DeepJSCC-LZ-MI outperforms other baseline methods under all SNR conditions, especially under low SNR conditions, where its advantage is more pronounced. This indicates that the method proposed in this embodiment can not only improve visual reconstruction quality but also enhance semantic consistency. ImageNet32 classification results are... Figure 8 The ablation experiments show that DeepJSCC-LZ-MI outperforms various benchmark methods in Top-5 classification accuracy, which further verifies the effectiveness of information-aware fusion and task-oriented training strategies.

[0111] 3) Ablation experiments and analysis To analyze the contribution of each module, this embodiment designs four ablation models: no MI: removes the mutual information guidance mechanism, no longer introduces mutual information calculation in the reconstruction model, and directly concatenates and fuses the main information features and side information features of multiple scales when calculating the fusion features; no Multiscale: uses only single-scale side information, and only calculates the side information features of one scale; no FPN: removes the feature pyramid structure, and no longer calculates multi-scale fusion features, directly inputting the fusion features into the decoder; DeepJSCC-LZ: does not use any enhancement modules.

[0112] a) Module contribution analysis like Figure 7 As shown, Figure 7 The channel models in (a) and (b) are AWGN. Figure 7 The channel models in (c) and (d) are Rayleigh fading channels. Experimental results show that removing any module leads to performance degradation. Among them, the mutual information guidance mechanism has the greatest impact on semantic task performance because it can prevent irrelevant or weakly correlated side information from being injected into the decoder. Multi-scale side information features have the greatest impact on PSNR, indicating that multi-resolution features are crucial for recovering fine-grained structural information. The FPN structure mainly affects the spatial detail recovery capability, and its role is to enhance the expressive power of high-frequency information.

[0113] b) The role of progressive SNR training strategy This embodiment employs a training strategy that progresses from high SNR to low SNR. In the initial training phase, the reconstruction model primarily learns to perform stable reconstruction under high SNR conditions. In the later training phase, the reconstruction model is gradually exposed to more complex channel environments, thereby learning to better utilize side information under low SNR conditions. This strategy avoids the model merely memorizing pixel-level mapping relationships, thus improving the generalization ability of the reconstruction model under different SNR conditions.

[0114] c) Comprehensive Analysis Ablation experiments show that the three modules have complementary effects: mutual information guidance mechanism: enhances information correlation, multi-scale side information features: enriches feature expression ability, and FPN structure: enhances spatial detail recovery. The combination of the three can achieve a performance improvement that is significantly better than that of the individual modules.

[0115] 4) Complexity and efficiency Table 1 shows the results under the same channel bandwidth ratio. and signal-to-noise ratio Performance comparison results of different methods under the given conditions.

[0116] Table 1 Performance Comparison

[0117] Experimental results show that DeepJSCC-LZ-MI achieves a good balance between computational complexity and performance. Although the number of model parameters is similar to that of the baseline model, the reconstruction model achieves significant performance improvement without significantly increasing computational overhead through mutual information guidance mechanism, multi-scale side information features and FPN structure.

[0118] Experimental results show that the mutual information guidance mechanism provides the most significant performance improvement when channel conditions are poor or semantic complexity is high. This mechanism enables the decoder to prioritize the use of side information features with the highest information content, while suppressing noise from weakly correlated regions. Multi-scale side information features and the FPN structure further ensure the simultaneous recovery of semantic information and spatial details. These results indicate that information-aware and task-oriented feature fusion mechanisms are an important research direction for building robust semantic communication systems, especially suitable for communication scenarios with limited bandwidth or computational resources.

[0119] Experimental results on the CIFAR-10 and ImageNet32 datasets show that, compared with the DeepJSCC-WZ method, DeepJSCC-LZ-MI achieves significant improvements in both PSNR and accuracy, while reducing computational complexity by 59.3%, demonstrating higher efficiency and better generalization ability.

[0120] Distributed semantic communication systems often fail to fully utilize side information because correlation fusion methods either operate only at a single scale or ignore the information-theoretic correlation between the main feature and side information features, resulting in limited robustness to varying channel conditions. To address this issue, this embodiment proposes DeepJSCC-LZ-MI, a task-oriented distributed JSCC framework for Wyner-Ziv scenarios. Its core is a mutual information-guided multi-scale feature fusion module that selectively injects complementary side information features based on the mutual information correlation across feature streams, while preserving spatial details with minimal overhead. Furthermore, this embodiment proposes a task-oriented end-to-end training strategy and a progressive SNR scheduling mechanism, enabling the optimization process to better align with semantic task objectives and improving robustness across signal-to-noise ratio environments. Experimental results on the CIFAR-10 and ImageNet32 datasets show that, compared with DeepJSCC-WZ, DeepJSCC-LZ-MI improves PSNR by 0.5–2.1 dB and accuracy by 3.0–15.8%, while reducing FLOPs by 59.3%. The reconstruction model maintains good performance on large-scale datasets (1000 classes) and under Rayleigh fading channel conditions, verifying its application potential in practical 5G / 6G semantic communication systems.

[0121] This embodiment proposes DeepJSCC-LZ-MI, a task-oriented distributed deep joint source-channel coding (DeepJSCC) framework suitable for Wyner-Ziv semantic communication. By combining MI (mutual information)-guided multi-scale feature aggregation with a progressive signal-to-noise ratio (SNR) training strategy, the proposed method can utilize side information more effectively and robustly. Extensive experiments on CIFAR-10 and ImageNet32 demonstrate continuous progress in both reconstruction quality and task-oriented semantic performance, especially under low SNR and bandwidth-constrained conditions. These results confirm that information-aware fusion based on mutual information correlation produces more reliable semantic reconstructions than heuristic feature aggregation while maintaining comparable computational complexity.

[0122] Example 2 This embodiment provides a method for applying a receiver reconstruction model, such as... Figure 9 As shown, the receiving end reconstruction model application method includes the following steps T1-T2.

[0123] Step T1: Receive the first image feature and the second image; the first image feature is obtained by encoding the first image using the transmitting end, and the first image and the second image are obtained by the transmitting end taking pictures of the same area from different perspectives.

[0124] Step T2: Using the first image features and the second image as input, the first image is reconstructed using the reconstruction model to obtain the first reconstructed image; wherein, the reconstruction model is trained using the receiver reconstruction model training method described in Example 1.

[0125] Example 3 This embodiment provides a receiving end, which is used to execute the receiving end reconstruction model application method described in Embodiment 2.

[0126] Example 4 This embodiment provides a distributed communication system, which includes a transmitter and a receiver.

[0127] The sending end is used to acquire a first image and a second image, encode the first image using an encoding model to obtain the first image features, and transmit the first image features and the second image to the receiving end.

[0128] The receiving end is the receiving end described in Example 3, used to execute the receiving end reconstruction model application method described in Example 2.

[0129] Example 5 In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 10 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a receiver reconstruction model training method.

[0130] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0131] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the receiver reconstruction model training method of embodiment 1.

[0132] Example 6 In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the receiver reconstruction model training method in Embodiment 1 or the receiver reconstruction model application method in Embodiment 2.

[0133] Example 7 In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the receiver reconstruction model training method in Embodiment 1 or the receiver reconstruction model application method in Embodiment 2.

[0134] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of the relevant data are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

[0135] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A receiver reconstruction model training method for training a receiver reconstruction model in a distributed DeepJSCC-WZ system, characterized in that, The receiver reconstruction model training method includes: Obtain the dataset; the dataset includes the first image sample collected by the sending end and the first image feature sample and the second image sample received by the receiving end. The first image feature sample is obtained by encoding the first image sample by the sending end. The first image sample and the second image sample are obtained by the sending end taking pictures of the same area from different perspectives. Using a first image feature sample and a second image sample as input, an initial reconstruction model is used to reconstruct the first image sample to obtain a first reconstructed image sample. The initial reconstruction model includes a multi-scale feature extraction module, a mutual information-guided fusion module, and a reconstruction module connected in sequence. The multi-scale feature extraction module is used to extract features from the second image sample to obtain second image feature samples at multiple scales. The mutual information-guided fusion module is used to calculate the mutual information between the first image feature sample and the second image feature sample at each scale, and to fuse the first image feature sample and the second image feature samples at multiple scales based on the mutual information to obtain fused features. The reconstruction module is used to reconstruct the first image sample based on the fused features to obtain the first reconstructed image sample. The total loss is calculated based on the first image sample, the first reconstructed image sample, the first image feature sample, and the second image feature samples at multiple scales. The initial reconstruction model is updated using the total loss to obtain the updated reconstruction model; Determine if the iteration termination condition has been met; if yes, use the updated reconstruction model as the receiving reconstruction model; if no, use the updated reconstruction model as the initial reconstruction model for the next iteration and return to the "Get Dataset" step; the reconstruction model is used for image reconstruction.

2. The receiver reconstruction model training method according to claim 1, characterized in that, The multi-scale feature extraction module includes a first feature extraction unit and multiple second feature extraction units connected in sequence. The input end of the first feature extraction unit is the input end of the multi-scale feature extraction module, used to input second image samples. The output end of the first feature extraction unit and the output end of each second feature extraction unit are both output ends of the multi-scale feature extraction module, used to output second image feature samples at multiple scales. The first feature extraction unit includes a first fully connected layer and a first activation function layer connected in sequence. The input of the first fully connected layer is the input of the first feature extraction unit, and the output of the first activation function layer is the output of the first feature extraction unit. The second feature extraction unit includes a max pooling layer, a second fully connected layer, and a second activation function layer connected in sequence. The input of the max pooling layer is the input of the second feature extraction unit, and the output of the second activation function layer is the output of the second feature extraction unit.

3. The receiver reconstruction model training method according to claim 1, characterized in that, The mutual information guided fusion module includes a mutual information calculation unit and a first fusion unit connected in sequence. The input terminal of the mutual information calculation unit is the input terminal of the mutual information guided fusion module, and the output terminal of the first fusion unit is the output terminal of the mutual information guided fusion module. The mutual information calculation unit is used to calculate the mutual information between the first image feature sample and the second image feature sample at each scale, and obtain the mutual information corresponding to the second image feature sample at each scale. The first fusion unit is used to calculate the gating weights corresponding to the second image feature samples at each scale based on the mutual information of the second image feature samples at each scale, and to perform a weighted summation of the first image feature samples and the second image feature samples at multiple scales based on the gating weights to obtain the fused features. The formula for calculating mutual information is as follows: ； ； in, For scale The mutual information corresponding to the second image feature samples; For the first image feature sample and scale The correlation coefficient between the second image feature samples; It is a constant; These are the first image feature samples after normalization and size transformation; The scale after normalization and size transformation The second image feature sample; The formula for calculating the gating weight is: ； in, For scale The gating weights corresponding to the second image feature samples; It is a nonlinear function; For scale The learnable threshold parameter corresponding to the second image feature sample; The formula for calculating the fusion feature is: ； in, Features of fusion; This is the first image feature sample after size transformation; The scale after size transformation The second image feature sample.

4. The receiver reconstruction model training method according to claim 1, characterized in that, The reconstruction module includes a second fusion unit and a reconstruction unit connected in sequence. The input terminal of the second fusion unit is the input terminal of the reconstruction module, and the output terminal of the reconstruction unit is the output terminal of the reconstruction module. The second fusion unit is used to calculate multi-scale fusion features based on second image feature samples and fusion features at multiple scales. The reconstruction unit is used to reconstruct the first image sample based on multi-scale fusion features to obtain the first reconstructed image sample; The second fusion unit comprises multiple feature fusion subunits connected in sequence. Each feature fusion subunit corresponds one-to-one with a second image feature sample at multiple scales. The input of the first feature fusion subunit is the input of the second fusion unit, used to input the fused features after size transformation. The input of the s-th feature fusion subunit is connected to the output of the (s-1)-th feature fusion subunit. The output of the S-th feature fusion subunit is the output of the second fusion unit, used to output the multi-scale fused features. s = 2, 3, ..., S, where S is the number of scales. The feature fusion subunit includes an upsampling layer, an attention layer, and an addition layer. The input of the upsampling layer is the input of the feature fusion subunit. The input of the attention layer is connected to the output of the upsampling layer. The first input of the addition layer is connected to the output of the attention layer. The second input of the addition layer is used to input the sum value feature at the corresponding scale. The sum value feature at the corresponding scale is the sum of the fused feature after size transformation and the second image feature sample at the corresponding scale after size transformation. The output of the addition layer is the output of the feature fusion subunit. The reconstruction unit includes a decoder.

5. The receiver reconstruction model training method according to claim 1, characterized in that, The dataset also includes the true task classification results of the first image sample. At this point, based on the first image sample, the first reconstructed image sample, the first image feature sample, and second image feature samples at multiple scales, the total loss is calculated, specifically including: Using the first reconstructed image sample as input, the task model is used to predict the classification result of the first image sample. Based on the first image sample and the first reconstructed image sample, the image reconstruction loss is calculated. Based on the first image sample, the first reconstructed image sample, the first image feature sample, and the second image feature samples at multiple scales, the mutual information loss is calculated. Based on the real task classification result and the predicted task classification result, the semantic task loss is calculated. The total loss is obtained by weighted summation of the image reconstruction loss, mutual information loss, and semantic task loss.

6. The receiver reconstruction model training method according to claim 1, characterized in that, Obtaining the dataset specifically includes: Obtain the initial dataset; the initial dataset includes the first and second image samples collected by the sending end; The first image sample is encoded using the encoding model of the sending end to obtain the first image feature sample; The first image feature sample and the second image sample are transmitted from the transmitter to the receiver via a wireless channel with a target signal-to-noise ratio. The first image feature sample and the second image sample received by the receiver are then obtained to obtain the dataset. The target signal-to-noise ratio is determined based on the current iteration number, and the formula for calculating the target signal-to-noise ratio is: ； in, The current iteration number The corresponding target signal-to-noise ratio; This represents the minimum signal-to-noise ratio. This represents the maximum signal-to-noise ratio. These are parameters used to control the decay rate; This represents the current iteration number; This represents the maximum number of iterations.

7. A method for applying a receiver reconstruction model, characterized in that, The method for applying the receiver reconstruction model includes: Receive first image features and second image; the first image features are obtained by encoding the first image using the transmitting end, and the first image and second image are obtained by the transmitting end taking pictures of the same area from different perspectives; Using the features of the first image and the second image as input, the first image is reconstructed using a reconstruction model to obtain a first reconstructed image; wherein the reconstruction model is trained using the receiver reconstruction model training method according to any one of claims 1-6.

8. A receiving end, characterized in that, The receiving end is used to execute the receiving end reconstruction model application method as described in claim 7.

9. A distributed communication system, characterized in that, The distributed communication system includes: a transmitter and a receiver; The transmitting end is used to acquire a first image and a second image, encode the first image using an encoding model to obtain the first image features, and transmit the first image features and the second image to the receiving end; The receiving end is used to execute the receiving end reconstruction model application method as described in claim 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the receiver reconstruction model training method according to any one of claims 1-6 or the receiver reconstruction model application method according to claim 7.

Images