Joint source-channel coding method for image semantic communication

Abstract

The present application relates to a joint source-channel coding method for image semantic communication, comprising the following steps: S1: establishing a joint source-channel coding system model based on a convolutional neural network and an attention mechanism; S2: inputting an image into a semantic encoder of a sending end of the model to extract semantic information of the image; S3: forwarding the extracted semantic information of the image to a channel encoder of the sending end, and dynamically adjusting semantic information output by the system according to a current channel state and a bandwidth compression ratio; S4: sending the encoded information to a channel decoder of a receiving end through a wireless channel, and dynamically adjusting reconstructed semantic information of the system according to the current channel state and the bandwidth compression ratio; and S5: recovering semantic information of the image according to the dynamically adjusted reconstructed semantic information of the system by a semantic decoder of the receiving end, so as to reconstruct the image. The present application can ensure robust communication in a harsh environment and effectively alleviate bandwidth pressure caused by a large amount of data transmission.

Description

Technical Field

[0001] This invention belongs to the field of semantic communication technology and relates to a joint source-channel coding method for image semantic communication. Background Technology

[0002] In recent years, with the rapid development of 5G technology and edge computing, the communication environment is undergoing unprecedented changes. The amount of data is growing explosively, and the complexity of the communication environment has brought unprecedented challenges to traditional communication methods. This is mainly reflected in two aspects: (1) how to ensure the robustness of communication under low signal-to-noise ratio conditions; (2) how to effectively alleviate the bandwidth pressure in massive data transmission.

[0003] Traditional communication methods often employ separate source-channel coding. This approach primarily considers the accuracy and high fidelity of the physical layer, thus requiring the transmission of the complete raw data without regard to content relevance. This leads to low communication efficiency, especially when dealing with massive data transmissions, where bandwidth pressure becomes even more pronounced. Furthermore, this traditional method is susceptible to the "cliff effect," meaning that under challenging communication conditions, such as low signal-to-noise ratio (SNR) environments, the risk of communication failure increases. This makes meeting high-precision task requirements in low SNR environments extremely challenging.

[0004] To address these issues, deep learning-based semantic communication offers a novel approach. This method employs joint source-channel coding, focusing on conveying semantic information rather than all data. This not only improves communication reliability but also effectively reduces data volume. Preliminary research indicates that semantic communication holds great potential in solving two major challenges in traditional communication.

[0005] Current research shows that while nonlocal attention mechanisms have been shown to adaptively adjust feature representations and improve model performance, they also incur considerable computational overhead. To address this issue, window attention mechanisms offer an effective alternative. They can apply attention within a limited scope, thereby significantly reducing the model's computational requirements.

[0006] This invention addresses the poor semantic communication performance of a single model under varying signal-to-noise ratios by proposing an attention-based adaptive coding module for semantic communication. Furthermore, a novel joint source-channel coding method is designed. This method combines deep learning with traditional communication techniques, aiming to achieve efficient and reliable communication transmission. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a joint source-channel coding method for image semantic communication, which solves the technical problem of poor semantic communication performance of a single model under different signal-to-noise ratio conditions.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A joint source-channel coding method for image semantic communication, comprising the following steps:

[0010] S1: Establish a joint source-channel coding system model based on convolutional neural networks and attention mechanisms;

[0011] S2: Input the image into the semantic encoder at the transmitter of the joint source-channel coding system model, and the semantic encoder extracts the semantic information of the image from the input image;

[0012] S3: The semantic information of the image extracted in S2 is forwarded to the channel encoder at the sending end for encoding. The channel encoder dynamically adjusts the semantic information output by the system according to the current channel state and bandwidth compression ratio.

[0013] S4: The encoded information in S3 is sent to the channel decoder at the receiving end through a power-normalized wireless channel. The channel decoder dynamically adjusts the semantic information reconstructed by the system according to the current channel state and bandwidth compression ratio.

[0014] S5: The semantic decoder at the receiving end recovers the semantic information of the image based on the semantic information reconstructed by the dynamically adjusted system in S4, thereby reconstructing the image.

[0015] Furthermore, in S1, the joint source-channel coding system model based on convolutional neural networks and attention mechanisms is specifically: the system includes a transmitter, a wireless channel with power normalization, and a receiver.

[0016] The sending end records the input image as: in, Let I represent the set of real numbers, where H, W, and C represent the height, width, and number of color channels of the image, respectively. During the transmission phase, I is sequentially processed by the semantic encoder. and channel encoder Mapped into an l-dimensional complex vector in Let be a set of complex numbers; assume that both communicating parties know the signal-to-noise ratio (SNR) of the wireless channel, denoted as μ, and the bandwidth compression ratio is denoted as R, and... A smaller R indicates a greater compression. The wireless channel transmission vector x is used to obtain the output vector y. Under additive white Gaussian noise (AWGN) channel conditions, y = x + z, where z is a vector with a mean of 0 and a variance of... The complex Gaussian noise is expressed as: Based on this Where P is the transmission power of the transmitting end; in the receiving phase, the receiving end is controlled by a semantic decoder. and channel decoder Composition, utilizing μ, and Based on this knowledge, y is decoded into the reconstructed image. Mean squared error (MSE) is used to measure the difference between the original image I and the reconstructed image. The difference between them, the loss function, is expressed as:

[0017]

[0018] Where N is the number of samples, The mean squared error distortion is denoted by n, where n is the total number of pixels in the image.

[0019] Furthermore, in S2, the semantic encoder includes several semantic information extraction modules, using... It means that each The module contains a two-dimensional convolutional layer, a generalized segmentation normalization layer (GDN), and an activation function.

[0020] Two-dimensional convolutional layers have local perceptual capabilities that can extract semantic information, including color, texture, shape, and image content.

[0021] The generalized segmentation normalization layer provides non-linear operations, which can extract more complex semantic information from the input image and ensure spatial adaptive normalization;

[0022] The consecutive use of several SIE modules can effectively extract the semantic information of an image. The semantic information output by the semantic encoder, that is, the output of the Mth SIE module, is denoted as... Represented as:

[0023]

[0024] When M=1

[0025] Furthermore, in S3, the channel encoder includes: an attention-based adaptive coding (ACC) module and a two-dimensional convolutional layer, denoted as...

[0026] The AAC module dynamically adjusts the semantic information output by the model to improve coding quality based on the current channel state; Used to adjust the transmitted information according to the bandwidth compression ratio;

[0027] The AAC module consists of two parts: semantic enhancement and semantic adjustment.

[0028] The semantic enhancement component includes: a window attention module (WAB) and several residual modules (RB). s ;

[0029] The window attention module weights key regions in the input feature map, allowing the model to process specific regions of the image in more detail, rather than processing the entire image uniformly.

[0030] The residual module focuses on the areas of the system model that need improvement.

[0031] The semantic enhancement specifically refers to:

[0032] The input information of the AAC module is marked as F. in Output information is generated using three residual modules with identical structures. Each residual module consists of three convolutional layers, labeled as... The concatenation of several residual modules enhances the features of the input image to combat channel noise. Simultaneously, to generate the weighting factor ξ, a WAB, three residual modules, and a convolutional layer are used, denoted as... Specifically, the feature maps of the image are divided into Q×Q windows in a non-overlapping manner, and an attention map is computed in each window using an attention mechanism with t heads; assuming and These are the pixels in the i-th row and j-th column of the q-th window, and the output of the i-th row of the q-th window is denoted as... Represented as:

[0033]

[0034] Among them, W α and W β It is a cross-channel transformation, W γ and W z It is a weight matrix;

[0035] Three residual modules are used after the window attention module, followed by a convolutional layer, then a sigmoid function to generate weight factors ξ∈[0,1]; finally, more detailed semantic information is obtained through weighted summation and residual operations. and Among them, H s W s and C s These represent the width, height, and number of feature channels of the semantically enhanced feature map, respectively.

[0036] The semantic adjustment improves the channel attention mechanism, enabling it to dynamically adjust more information of interest based on the constantly changing channel state information of the wireless channel, i.e., the signal-to-noise ratio μ.

[0037] The semantic adjustment specifically involves using the sum of the standard deviation and the mean, SDM(·), to capture... Global information within; for the input vector The kth feature channel is defined as z k The output after SDM(·) is represented as:

[0038]

[0039] The obtained global information is concatenated with μ along the feature channel dimension and then processed by two convolutional layers. and To predict weighting factors, by weighting The adjusted output is represented as follows:

[0040]

[0041] Where Concat(·) represents a join operation. Using the ReLU activation function to learn nonlinear relationships Use the Sigmoid activation function to ensure that the weight factors are between 0 and 1.

[0042] Furthermore, the wireless channel in S4 is an additive white Gaussian noise channel.

[0043] Furthermore, in S4, the channel decoder, once it receives the semantic information y transmitted from the wireless channel, [then...]. Convert it into semantic information

[0044] Channel decoder Includes: AAC module and two-dimensional deconvolution layer

[0045] The AAC module can dynamically adjust the semantic information of the model reconstruction and improve the decoding quality according to the current channel state.

[0046] Used to adjust the dimension of received semantic information according to the bandwidth compression rate; the AAC module in the channel decoder and channel encoder has the same structure but different parameters.

[0047] Furthermore, in S5, the semantic decoder The SIR module contains M semantic information reconstruction modules, denoted as M. Each The module contains a two-dimensional deconvolution layer, an inverse generalized division normalized IGDN layer, and an activation function; and Represent and The reverse process, and The module's structure is symmetrical;

[0048] Semantic decoding is specifically described as follows:

[0049]

[0050] in, express The output of the m-th SIR module, and the output of the last SIR module is the reconstructed image. When m = M

[0051] The beneficial effects of this invention are as follows:

[0052] This invention proposes a joint source-channel coding method for image semantic communication, which includes a flexible attention-based adaptive encoding / decoding module. Considering resource constraints, this method can capture the correlation between spatially adjacent elements without sacrificing too much computational resources, dynamically weight key local semantic information, and dynamically adjust the model output based on the current channel state information, enabling a single model to adapt to a wide range of signal-to-noise ratio training. Experimental results show that even under channel mismatch conditions, compared with the comparative scheme 1 (a joint source-channel coding model based on convolutional neural networks), the proposed scheme is more robust, ensuring robust communication in harsh environments and effectively reducing the bandwidth pressure caused by large-scale data transmission.

[0053] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0054] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0055] Figure 1 This is a flowchart illustrating the joint source-channel coding method for image semantic communication according to the present invention.

[0056] Figure 2 This is a system model diagram of the joint source-channel coding method for image semantic communication according to the present invention;

[0057] Figure 3 This is a neural network structure diagram of the joint source-channel coding method for image semantic communication according to the present invention.

[0058] Figure 4 This is a diagram of the attention-based adaptive coding module of the joint source-channel coding method for image semantic communication according to the present invention. Figure 4 (a) is a structural diagram of the AAC module; Figure 4 (b) is a structural diagram of the residual block; Figure 4 (c) is a diagram of attention calculation units;

[0059] Figure 5 The joint source-channel coding method for image semantic communication proposed in this invention... The training and testing images are plots showing the relationship between signal-to-noise ratio and peak signal-to-noise ratio on the CIFAR-10 dataset;

[0060] Figure 6 The joint source-channel coding method for image semantic communication proposed in this invention... The training and testing images are plots showing the relationship between signal-to-noise ratio and peak signal-to-noise ratio on the CIFAR-10 dataset;

[0061] Figure 7 The joint source-channel coding method for image semantic communication proposed in this invention... The training images are from the ImageNet2012 dataset, and the test images are from the Kodak dataset. The graph shows the relationship between signal-to-noise ratio and peak signal-to-noise ratio.

[0062] Figure 8 The joint source-channel coding method for image semantic communication proposed in this invention... The training images are from the ImageNet2012 dataset, and the test images are from the Kodak dataset. The graph shows the relationship between signal-to-noise ratio and peak signal-to-noise ratio. Detailed Implementation

[0063] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0064] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0065] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0066] Please see Figures 1 to 8 This is a joint source-channel coding method for image semantic communication, such as... Figure 1 As shown: This method includes the following steps:

[0067] S1: Establish a joint source-channel coding system model based on convolutional neural networks and attention mechanisms;

[0068] The established joint source-channel coding system model based on convolutional neural networks and attention mechanisms, such as Figure 2 , Figure 3 As shown, the system includes a transmitter and a receiver. The transmitter uses a semantic encoder to extract semantic information from the input image. To ensure the validity of the information, it is forwarded to a channel encoder before transmission. The encoded information is transmitted to the receiver via a power-normalized wireless channel (e.g., an additive white Gaussian noise channel). Upon receiving the information, the receiver processes it sequentially through a channel decoder and a semantic decoder to reconstruct the image.

[0069] Specifically, the input image at the sending end is denoted as... in Let I represent the set of real numbers, where H, W, and C represent the height, width, and number of color channels of the image, respectively. During the transmission phase, I is sequentially processed by the semantic encoder. and channel encoder Mapped to a 1-dimensional complex vector in Let be a set of complex numbers. It is worth noting that in this invention, it is assumed that both communicating parties know the signal-to-noise ratio (SNR) of the wireless channel, denoted as μ, and the bandwidth compression ratio is denoted as R. A smaller R value indicates a greater compression. The wireless channel transmission vector x is used to obtain the output vector y. In an Additive White Gaussian noise (AWGN) channel, y = x + z, where z is a vector with a mean of 0 and a variance of... Complex Gaussian noise, denoted as Based on this Where P is the transmission power of the transmitting end. During the receiving phase, the receiving end uses a semantic decoder. and channel decoder Composition, utilizing μ, and Based on this knowledge, y is decoded into the reconstructed image. We use Mean Square Error (MSE) to measure the difference between the original image I and the reconstructed image. The difference between them. Therefore, the loss function is determined by the following formula:

[0070]

[0071] Where N is the number of samples, The mean squared error distortion is denoted as n, where n is the total number of pixels in the image.

[0072] S2: Input the image into the semantic encoder at the transmitter of the joint source-channel coding system model, and the semantic encoder extracts the semantic information of the image from the input image;

[0073] Its content includes: a semantic encoder at the sending end to effectively extract semantic information from images. It uses the Convolutional Neural Networks (CNNs) architecture. It contains M semantic information extraction (SIE) modules, using Indicates. Each. The module comprises a 2D convolutional layer, a Generalized Division Normalization (GDN) layer, and an activation function. The local perceptual capabilities of the convolutional layer effectively extract semantic information such as color, texture, shape, and image content. The non-linear operation provided by the GDN layer facilitates the extraction of more complex semantic information from the input image, ensuring spatially adaptive normalization. This is crucial for reducing spatial redundancy and more effectively capturing the spatial structure information of the image. The successive use of multiple SIE modules effectively extracts the semantic information of the image. The semantic information output by the semantic encoder (i.e., the output of the Mth SIE module) is denoted as... It can be represented as:

[0074]

[0075] When M=1

[0076] S3: The semantic information of the image extracted in S2 is forwarded to the channel encoder at the sending end for encoding. The channel encoder dynamically adjusts the semantic information output by the system according to the current channel state and bandwidth compression ratio, such as... Figure 4 As shown;

[0077] Its contents include: the channel encoder at the transmitting end. It consists of an attention-based adaptive coding (ACC) module and a two-dimensional convolutional layer (denoted as...). The AAC module is composed of several components. It can dynamically adjust the semantic information of the model output, improving coding quality based on the current channel state. Used to adjust the transmitted information according to the bandwidth compression ratio. For example... Figure 4 As shown in (a), the AAC module consists of two parts: semantic enhancement and semantic adjustment, which will be explained in detail below.

[0078] C1. Semantic Enhancement: The semantic enhancement part consists of a Window Attention Block (WAB) and multiple Residual Blocks (RBs). In this invention, the Window Attention Block weights key regions in the input feature map. This approach allows the model to process specific regions of the image in more detail, such as specific textures or edges, rather than processing the entire image uniformly. This mechanism enhances feature representations, especially in scenes with rich content or varying details. Subsequently, the Residual Blocks focus on regions where the model needs improvement, as these regions are optimized during training to address reconstruction discrepancies. The detailed process is described below.

[0079] First, the input information of the AAC module (labeled as F) in Output information is generated using three residual modules with identical structures. To improve the stability of the generalized segmentation normalization layer, such as Figure 4 As shown in (b), each residual module consists of three convolutional layers (labeled as...). The system consists of multiple residual modules, which enhance the features of the input image to combat channel noise. Simultaneously, to generate the weighting factor ξ, this invention uses a WAB, three residual modules, and a convolutional layer (denoted as...). Specifically, the feature maps of the image are divided into Q×Q windows in a non-overlapping manner, and then an attention map in each window is computed using an attention mechanism with t heads, as shown below. Figure 4 As shown in (c). Assume... and These are the pixels in the i-th row and j-th column of the q-th window. The output of the i-th row of the q-th window (denoted as...) This can be represented as:

[0080]

[0081] Among them, W α and W β It is a cross-channel transformation, W γ and W z This is the weight matrix. To improve the model's learning ability, three residual modules are used after the window attention module, followed by a convolutional layer, then a sigmoid function to generate weight factors ξ∈[0,1]. Finally, more detailed semantic information is obtained through weighted summation and residual operations. and Among them, H s W s and C s These represent the width, height, and number of feature channels of the semantically enhanced feature map, respectively.

[0082] C2. Semantic Adjustment: To effectively reduce the adverse effects of the channel on the model and improve the model's robustness to signal-to-noise ratio (SNR) within a certain range, this invention improves the channel attention mechanism, enabling it to dynamically adjust more information of interest based on the constantly changing channel state information (i.e., SNR μ) of the wireless channel. Specifically, the sum of the standard deviation and the mean, SDM(·), is first used to capture... Global information within the input vector. Compared to global average pooling, SDM(·) better preserves relevant structural, texture, and edge information, which is highly beneficial for enhancing image details. The kth feature channel (defined as z) k The output after SDM(·) can be expressed as:

[0083]

[0084] Next, the obtained global information is concatenated with μ along the feature channel dimension, and then processed by two convolutional layers (denoted as μ). and To predict the weighting factors, a weighted average is then calculated. The adjusted output can be represented as:

[0085]

[0086] Concat(·) represents a join operation. Using the ReLU activation function to learn nonlinear relationships Use the Sigmoid activation function to ensure that the weight factors are between 0 and 1.

[0087] S4: The encoded information in S3 is sent to the channel decoder at the receiving end through a power-normalized wireless channel. The channel decoder dynamically adjusts the semantic information reconstructed by the system according to the current channel state and bandwidth compression ratio.

[0088] Its content includes: once the receiving end receives semantic information y transmitted from the wireless channel, the channel decoder... It will be converted into semantic information. With channel encoder Similarly, channel decoder It consists of an AAC module and a two-dimensional deconvolution layer (denoted as AAC). The AAC module is composed of several components. It can dynamically adjust the semantic information of the model reconstruction and improve the decoding quality according to the current channel state. Used to adjust the dimensions of received semantic information according to the bandwidth compression rate. and the channel encoder in S3 The AAC modules in the S3 have the same structure (detailed in S3), but their parameters are different.

[0089] S5: The semantic decoder at the receiving end recovers the semantic information of the image based on the semantic information reconstructed by the dynamically adjusted system in S4, thereby reconstructing the image;

[0090] Its content includes: based on S4, a semantic decoder Reconstruct the transmitted image. (Compare with the semantic encoder in S2) similar, It contains M semantic information reconstruction (SIR) modules, denoted as... Each The module consists of a two-dimensional deconvolution layer, an inverse generalized division normalization (IGDN) layer, and an activation function. and Represent and The reverse process, in this invention, and The module structure is symmetrical. The semantic decoding process can be described as follows:

[0091] in, express The output of the m-th SIR module, and the output of the last SIR module is the reconstructed image. When m = M

[0092] like Figure 5 , 6 As shown in Figures 7 and 8, this invention provides the proposed model in... and The test images show the relationship between SNR and Peak Signal-to-Noise Ratio (PSNR) on the CIFAR-10 and Kodak datasets. As can be seen from the four figures, the proposed scheme outperforms existing schemes (the joint source-channel coding model based on convolutional neural networks, i.e., comparison scheme 1) at different training SNRs. First, with the increase of the test SNR, the PSNR of all schemes shows an increasing trend, indicating that the improvement in image reconstruction quality is parallel to the improvement in signal quality. Second, in all training with the same SNR and R value, the proposed scheme consistently achieves a higher PSNR compared to the baseline model, especially in the low SNR region. Furthermore, in terms of bandwidth compression ratio... At that time, the peak signal-to-noise ratio of all schemes was generally higher than that of the other two schemes. The proposed scheme achieves this because a higher compression ratio (or higher bandwidth) facilitates the transmission of more information through the channel, thereby improving the image reconstruction quality. Finally, when the training signal-to-noise ratio (SNR) range is [0, 10] dB, the proposed scheme exhibits superior performance at all tested SNR points. This is because the AAC module can dynamically adjust semantic features based on the real-time SNR. Therefore, the proposed scheme can be trained within a specific SNR range, significantly enhancing its robustness in various SNR scenarios.

[0093] This invention proposes a joint source-channel coding method for image semantic communication, which includes a flexible attention-based adaptive encoding / decoding module. Considering resource constraints, this method can capture the correlation between spatially adjacent elements without sacrificing too much computational resources, dynamically weight key local semantic information, and dynamically adjust the model output based on the current channel state information, enabling a single model to adapt to a wide range of signal-to-noise ratio training. Experimental results show that even under channel mismatch conditions, the proposed scheme is more robust than comparative scheme 1, ensuring robust communication in harsh environments and effectively reducing bandwidth pressure caused by large-scale data transmission.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A joint source-channel coding method for image semantic communication, characterized in that: The method includes the following steps: S1: Establish a joint source-channel coding system model based on convolutional neural networks and attention mechanisms; S2: Input the image into the semantic encoder at the transmitter of the joint source-channel coding system model, and the semantic encoder extracts the semantic information of the image from the input image; S3: The semantic information of the image extracted in S2 is forwarded to the channel encoder at the sending end for encoding. The channel encoder dynamically adjusts the semantic information output by the system according to the current channel state and bandwidth compression ratio. In S3, the channel encoder includes: an attention-based adaptive coding AAC module and a two-dimensional convolutional layer, denoted as... ; The AAC module dynamically adjusts the semantic information output by the model to improve coding quality based on the current channel state; This is used to adjust the transmitted information according to the bandwidth compression ratio; The AAC module consists of two parts: semantic enhancement and semantic adjustment. The semantic enhancement component includes: a window attention module (WAB) and several residual modules (RBs); The window attention module weights key regions in the input feature map, allowing the model to process specific regions of the image in more detail, rather than processing the entire image uniformly. The residual module focuses on the areas of the system model that need improvement. The semantic enhancement specifically refers to: The input information of the AAC module is marked as Output information is generated using three residual modules with identical structures. Each residual module consists of three convolutional layers, labeled as... , , The connection of several residual modules can enhance the features of the input image to combat channel noise; at the same time, in order to generate weighting factors... Using one WAB, three residual modules, and one convolutional layer, denoted as... Specifically, the feature maps of the image are divided into non-overlapping groups. The window uses a number of heads. The attention mechanism calculates the attention map for each window separately; assuming and It is the first The first window row and number Pixels in the column, the first The first window Line output, denoted as , represented as: in, and It is a cross-channel transformation. and It is a weight matrix; Three residual modules are used after the window attention module, followed by a convolutional layer, and then a sigmoid function to generate weight factors. Finally, more detailed semantic information is obtained through weighted summation and residual operations. ,and ,in, These represent the width, height, and number of feature channels of the semantically enhanced feature map, respectively. The semantic adjustment improves the channel attention mechanism, enabling it to adjust based on the constantly changing channel state information of the wireless channel, i.e., the signal-to-noise ratio. Dynamically adjust more information that receives attention; The semantic adjustment specifically involves using the sum of the standard deviation and the mean. To capture Global information within; for the input vector =[ ], No. Each feature channel is defined as... ,go through The output is represented as follows: The obtained global information is along the feature channel dimension and Connect using two convolutional layers To predict weighting factors, by weighting The adjusted output is represented as follows: in, Indicates a connection operation. Using the ReLU activation function to learn nonlinear relationships Use the Sigmoid activation function to ensure that the weight factors are between 0 and 1; S4: The encoded information in S3 is sent to the channel decoder at the receiving end through a power-normalized wireless channel. The channel decoder dynamically adjusts the semantic information reconstructed by the system according to the current channel state and bandwidth compression ratio. S5: The semantic decoder at the receiving end recovers the semantic information of the image based on the semantic information reconstructed by the dynamically adjusted system in S4, thereby reconstructing the image.

2. The joint source-channel coding method for image semantic communication according to claim 1, characterized in that: In S1, the joint source-channel coding system model based on convolutional neural network and attention mechanism is established as follows: the system includes a transmitter, a wireless channel with power normalization, and a receiver. The sending end records the input image as: ,in, Represents the set of real numbers. , , These represent the image's height, width, and number of color channels, respectively; during the transmission phase, Sequentially encoded by the semantic encoder and channel encoder Mapped into an l-dimensional complex vector ,in Let be a set of complex numbers; assume that both communicating parties know the signal-to-noise ratio (SNR) of the wireless channel, denoted as . The bandwidth compression ratio is denoted as ,and , A smaller value indicates a greater compression. The wireless channel transmission vector x is used to obtain the output vector y. Under additive white Gaussian noise (AWGN) channel conditions... ,in, The mean is 0 and the variance is The complex Gaussian noise is expressed as: Based on this, ,in, This refers to the transmission power at the transmitting end; during the receiving phase, the receiving end uses a semantic decoder. and channel decoder Composition, utilization , and Based on this knowledge, y is decoded into the reconstructed image. Mean squared error (MSE) is used to measure the original image. With reconstructed image The difference between them, the loss function, is expressed as: in, For the number of samples, For mean square error distortion, This represents the total number of pixels in the image.

3. The joint source-channel coding method for image semantic communication according to claim 2, characterized in that: In S2, the semantic encoder includes several semantic information extraction modules, using... It means that each The module contains a two-dimensional convolutional layer, a generalized segmentation normalization layer (GDN), and an activation function. Two-dimensional convolutional layers have local perceptual capabilities that can extract semantic information, including color, texture, shape, and image content. The generalized segmentation normalization layer provides non-linear operations, which can extract more complex semantic information from the input image and ensure spatial adaptive normalization; The consecutive use of several SIE modules can effectively extract the semantic information of an image. The semantic information output by the semantic encoder, i.e., the semantic information of the first module... The output of each SIE module, denoted as , is represented as: when hour, .

4. The joint source-channel coding method for image semantic communication according to claim 1, characterized in that: The wireless channel in S4 is an additive white Gaussian noise channel.

5. The joint source-channel coding method for image semantic communication according to claim 1, characterized in that: In step S4, the channel decoder, once it receives semantic information transmitted from the wireless channel... Then, the channel decoder Convert it into semantic information ; Channel decoder Includes: AAC module and two-dimensional deconvolution layer ; The AAC module can dynamically adjust the semantic information of the model reconstruction and improve the decoding quality according to the current channel state. Used to adjust the dimension of received semantic information according to the bandwidth compression rate; the AAC module in the channel decoder and channel encoder has the same structure but different parameters.

6. The joint source-channel coding method for image semantic communication according to claim 5, characterized in that: In S5, the semantic decoder Include A semantic information reconstruction SIR module, denoted as Each The module contains a two-dimensional deconvolution layer, an inverse generalized division normalized IGDN layer, and an activation function; and Represent and The reverse process, and The module's structure is symmetrical; Semantic decoding is specifically described as follows: in, express The first in The output of each SIR module is a reconstructed image, with the last SIR module outputting the reconstructed image. ;when hour, .

Images