Semantic communication method and system based on adaptive semantic reconstruction

Abstract

The present invention discloses a semantic communication method and system based on adaptive semantic reconstruction, which belongs to the field of wireless communication technology. It proposes a task-oriented semantic communication framework based on adaptive semantic reconstruction. Through mask operation and design of a semantic reconstruction network with integrated attention mechanism, it realizes flexible compression of semantic information, simulation of data loss during transmission and reconstruction effect adapting to different environments. It also proposes a joint training method of encoding, decoding network and semantic reconstruction network to minimize the loss function of the target task and obtain the approximate optimal solution of neural network parameters. Compared with the existing semantic communication scheme, the proposed scheme has better adaptability to different signal-to-noise ratio, semantic compression rate and semantic loss rate conditions, and can obtain higher task execution performance and semantic transmission benefits in communication.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a semantic communication method and system based on adaptive semantic reconstruction. Background Art

[0002] In recent years, semantic communication has attracted widespread attention in the industry as a key technology to meet the task requirements of the intelligent communication era and overcome the challenges associated with traditional communication systems. The main goal of the classic semantic communication framework is to reconstruct the source sequence within the required fidelity constraints without considering the needs of specific tasks. In applications such as image transmission, different tasks require different image features. Therefore, a task-oriented semantic communication framework has emerged, which only transmits task-related information, greatly improving transmission efficiency. This provides a new way for self-driving cars, robots, and other intelligent entities to efficiently process, transmit, and analyze massive visual data.

[0003] The main challenge of task-oriented semantic communication is how to extract the most relevant semantic information from the data at the sender to facilitate the execution of tasks at the receiver. In addition, since the randomness of the wireless channel affects the forward and backward propagation during training, and the availability of channel state information (CSI) is also affected, it is also challenging to train semantic communication systems in an end-to-end manner. These challenges can be addressed by adaptive schemes based on signal-to-noise ratio, such as the deep joint source channel coding (JSCC) scheme based on autoencoders and the deep JSCC scheme based on attention modules. These schemes use CSI and deep learning (DL)-assisted semantic extraction techniques to show great potential in different information sources and related tasks.

[0004] Despite significant progress in semantic-based communication architectures, the loss of semantic features during transmission is still inevitable, which affects the task performance at the receiving end. Traditional error correction methods such as low-density parity-check codes and retransmission mechanisms can mitigate data loss, but they incur additional overhead and latency. Under extremely adverse channel conditions, these methods may fail, resulting in reduced communication reliability and efficiency. In addition, despite the growing interest in deep learning-based semantic communication image transmission, few studies have considered the impact of semantic compression rate (SCR) and semantic loss rate (SLR) on semantic communication performance in dynamic urban environments. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a semantic communication method based on adaptive semantic reconstruction (SCASR), which can realize the execution of target tasks with minimal distortion at specific channel signal-to-noise ratio, SCR and SLR through the designed semantic communication framework based on adaptive semantic reconstruction.

[0006] Technical solution: To achieve the above-mentioned invention object, the present invention adopts the following technical solution:

[0007] A semantic communication method based on adaptive semantic reconstruction comprises the following steps:

[0008] Step 1: Obtain semantic information through the semantic encoder at the sending end;

[0009] Step 2: compress the semantic information according to the semantic compression rate SCR through the semantic compressor, and simulate the data loss during the transmission process through mask operation according to the semantic loss rate SLR to obtain the compressed semantic information;

[0010] Step 3: Transmit the compressed semantic information to the receiving end;

[0011] Step 4: The receiving end reconstructs the received signal using a semantic reconstruction network to obtain reconstructed semantic information; the semantic reconstruction network includes a combination of stacked multiple residual blocks RB and attention feature blocks AFB; the input of the AFB includes the output of the RB and the channel signal-to-noise ratio, SCR and SLR;

[0012] Step 5: Pass the reconstructed semantic information through the semantic decoder to obtain the result of the target task.

[0013] Preferably, in step 2, compressing the semantic information by a semantic compressor comprises:

[0014] Flatten the semantic information Y into a one-dimensional vector; use the compression function f(Y,r c ) compresses the semantic information Y into a vector Z of size m; wherein m is determined according to SCR; the compression function f(Y, r c ) consists of two steps. First, a mask generation function is designed, and SCR = r c , generate a mask vector M with the same scale as Y 1 , which contains only 0 or 1, the number of 1 is m; then the elements in Y are combined with M 1 Multiply the elements of to get the compressed semantic information Z.

[0015] Preferably, the simulation of data loss during transmission is realized by mask operation in step 2, including:

[0016] The semantic information Z compressed by the semantic compressor is transmitted through the physical channel. The channel coefficient h and noise N are added to Z. The semantic information received by the receiver is Let SLR be r l , the semantic information actually received by the receiver is obtained through the random data loss function Transfer; the random data loss function It consists of two steps. First, a mask generation function is designed, and SLR = r is specified. l , generating a sum The mask vector M of the same size 2 , which contains only 0 or 1, and the number of 1 is m 2 , m 2 According to SLR, M 2 The positions of 0 and 1 in are random to simulate the process of random data loss; then The middle element and M 2 Multiply the elements in to get the semantic information actually received by the receiver.

[0017] Preferably, in step 4, the calculation process of the semantic reconstruction network is:

[0018] set up represents the semantic information of the input of the semantic reconstruction network, then the output of the semantic reconstruction network express:

[0019]

[0020] Among them, Conv 1 (·), Conv 2 (·) and Conv 3 (·) denote the first, second, and third convolutional layers, respectively. represents the output of the last AFB, K is the number of iterations of RB and AFB; Calculated as:

[0021]

[0022] in, represents the output of the (k-1)th AFB, γ, r c and r l They are channel signal-to-noise ratio, SCR and SLR, RB k (·) and AFB k (·) represent the kth RB and the kth AFB, respectively.

[0023] As a preference, the output of the kth RB Calculated as:

[0024]

[0025] Among them, Conv 4 (·) and Conv 5 (·) represent the first and second convolutional layers in RB, σ 1 (·) represents the ReLU activation function in RB, and S is a scaling factor.

[0026] As a preference, an AFB is placed behind each RB to improve the network's ability to adapt to signal-to-noise ratio, SCR, and SLR. represents the features of the kth RB output, where c is the number of features and h×w is the size of each feature;

[0027] First, AFB processes features through a global average pooling function

[0028] Afterwards, and signal-to-noise ratio γ, SCRr c and SLRr l Connect to form context information:

[0029]

[0030] A neural network consisting of two fully connected layers is used to predict the weight coefficient context information of each channel is passed to a two-layer FC neural network to produce a weight coefficient vector Among them, FC 1 (·) and FC 2 (·) represent the first and second fully connected layers in AFB, σ 2 (·) and σ 3 (·) represent the ReLU and sigmoid activation functions in AFB, respectively;

[0031] Finally, by adding the features and weight coefficient W k Multiply them together to get the weighted features

[0032] Furthermore, the semantic encoder, semantic decoder and semantic reconstruction network are trained by a joint training method of the encoding network, the decoding network and the semantic reconstruction network to minimize the loss function of the target task; wherein the loss function of the encoding network and the decoding network is expressed as:

[0033]

[0034] in, It represents the prediction result obtained at the receiving end after encoding, compression, channel transmission, reconstruction and decoding. r represents the true value, the subscript i marks the i-th category, and l represents the number of categories;

[0035] The loss function of the semantic reconstruction network is expressed as:

[0036]

[0037] Among them, Y represents the input semantic information, represents the output of the semantic reconstruction network, the subscript j marks the jth feature, and c represents the number of features;

[0038] The global loss function is defined as:

[0039]

[0040] The model parameters θ, φ, ω are optimized by minimizing the global loss function.

[0041] Preferably, the steps of training the semantic encoder, the semantic decoder and the semantic reconstruction network specifically include:

[0042] (1) For each training round t, the sender extracts B samples X = [X 1 ,X 2 ,···,X B ] for batch training;

[0043] (2) For each sample X i , randomly generate its channel signal-to-noise ratio γ i , and

[0044] (3) Through the semantic encoder E φ Extracting semantic information: Y i =E φ (X i );

[0045] (4) Compress the semantic information through the semantic compressor f: Z i =f(Y i ,r i c );

[0046] (5) Randomly generate channel coefficients and Gaussian white noise in is the noise variance, according to γ i Determine that I is the identity matrix;

[0047] (6) Calculate the received semantic information based on the loss function g:

[0048]

[0049] (7) Using semantic reconstruction network Reconstructing semantic information:

[0050]

[0051] (8) Through the semantic decoder D θ The calculation prediction results are:

[0052]

[0053] (9) Repeat steps (2) to (8) and calculate the average loss based on the global loss function;

[0054] (10) Adopting adaptive moment estimation algorithm to optimize model parameters θ, φ, ω;

[0055] (11) Repeat steps (1) to (10) until all training rounds are completed and the approximate optimal solution for the parameters is obtained.

[0056] The present invention also provides a semantic communication system based on adaptive semantic reconstruction, comprising a semantic encoder and a semantic compressor at a transmitting end, and a semantic reconstruction network and a semantic decoder at a receiving end;

[0057] The semantic encoder is used to encode the input data to obtain semantic information;

[0058] The semantic compressor is used to compress the semantic information obtained by the semantic encoder according to the semantic compression rate SCR, and to simulate the data loss during the transmission process through a mask operation according to the semantic loss rate SLR to obtain the compressed semantic information;

[0059] The semantic reconstruction network is used to semantically reconstruct the compressed semantic information received by the receiving end to obtain the reconstructed semantic information; the semantic reconstruction network includes a combination of stacked multiple residual blocks RB and attention feature blocks AFB; the input of the AFB includes the output of the RB and the channel signal-to-noise ratio, SCR and SLR;

[0060] The semantic decoder is used to decode the reconstructed semantic information to obtain the result of the target task.

[0061] The present invention also provides a computer program product, comprising a computer program / instruction, which implements the steps of the semantic communication method based on adaptive semantic reconstruction when the computer program / instruction is executed by a processor.

[0062] Beneficial effects: The present invention proposes a task-oriented semantic communication framework based on adaptive semantic reconstruction. Through masking operations and the design of a semantic reconstruction network with an integrated attention mechanism, it realizes flexible compression of semantic information, simulation of data loss during transmission, and reconstruction effects that adapt to different environments. At the same time, the present invention also designs a joint training method for encoding, decoding networks, and semantic reconstruction networks to minimize the loss function of the target task and obtain an approximate optimal solution for the neural network parameters. Compared with existing semantic communication schemes, the proposed SCASR scheme has superior adaptability to different signal-to-noise ratios, SCRs, and SLR conditions, and can achieve higher task execution performance and semantic transmission benefits in communication.

[0063] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 The present invention is a flowchart of a semantic communication method based on adaptive semantic reconstruction according to an embodiment of the present invention.

[0065] Figure 2 Schematic diagram of semantic compression and semantic loss based on mask operation in an embodiment of the present invention.

[0066] Figure 3 Schematic diagram of the semantic reconstruction network in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments described below with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention.

[0068] Figure 1 The present invention is a flowchart of a semantic communication method based on adaptive semantic reconstruction disclosed in an embodiment of the present invention.

[0069] like Figure 1 As shown, the semantic communication method based on adaptive semantic reconstruction includes the following steps:

[0070] Step S1: obtaining semantic information through a semantic encoder at the transmitting end. The specific semantic encoder may be an existing semantic encoder, such as mobilenet V2.

[0071] Step S2: The semantic information is flexibly compressed by the semantic compressor, and the simulation of data loss during transmission is realized by mask operation to obtain the compressed semantic information, such as Figure 2 shown.

[0072] Specifically, in one embodiment of the present invention, the semantic information Y is flattened into a one-dimensional vector. Then, the compression function f(Y, r c ) compresses the semantic information Y into a vector Z of size m. Where m is equal to Y multiplied by r c scale.

[0073] Compression function f(Y,r c ) consists of two steps. First, a mask generation function (MGF) is designed, namely MGF 1 . SCR = r c , we can generate a mask vector M with the same scale as Y 1 , which contains only 0 or 1, and the number of 1 is m. Then, compare the elements in Y with M 1 Multiply the elements of to get the compressed semantic information Z, that is:

[0074] Z=f(Y,r c )=Y·MGF 1 (r c )(1)

[0075] The semantic information Z is transmitted through the physical channel. The channel coefficient h and noise N are added to Z. The semantic information received by the receiver is Since data loss during transmission is often inevitable, assume that SLR is r l If there is no retransmission, the semantic information actually received by the receiver should be lost through the random data loss function transfer.

[0076] Random Data Loss Function It also consists of two steps. First, design a mask generation function MGF 2 , SLR = r l , a mask vector m with the same scale as Z can be generated 2 , which contains only 0 or 1, and the number of 1 is m 2 , m 2 equal Multiply by r l Length. 1 Different, M 2 The positions of 0 and 1 in are random to simulate the process of random data loss. Then, The middle element and M 2 Multiply the elements in to get the receiver The semantic information actually received is:

[0077]

[0078] Finally, we use the refactoring function to Convert it to the same scale as Y and use it as the input of the semantic reconstruction network.

[0079] Step S3: After power normalization, the compressed semantic information is transmitted to the receiving end through the physical channel.

[0080] Step S4: The receiving end reconstructs the received signal using the designed semantic reconstruction network to obtain the reconstructed semantic information, such as Figure 3 shown.

[0081] Specifically, in one embodiment of the present invention, the semantic reconstruction network is mainly composed of two modules: residual block (RB) and attention feature block (AFB).

[0082] set up represents the semantic information of the input, then the output of the semantic reconstruction network It can be calculated as:

[0083]

[0084] Among them, Conv 1 (·), Conv 2 (·) and Conv 3 (·) denote the first, second, and third convolutional layers, respectively. is the input semantic information, Indicates the output of the last AFB. It can be calculated as:

[0085]

[0086] Wherein, K is the number of iterations of RB and AFB, and the value of K is 16 in the present invention. represents the output of the (K-1)th AFB, γ, r c and r l They are channel signal-to-noise ratio, SCR and SLR respectively. k (·) and AFB k (·) represent the kth RB and the kth AFB, respectively.

[0087] We remove the batch normalization layer from the network to reduce the complexity of the network, then the output of the kth RB is It can be calculated as:

[0088]

[0089] Among them, Conv 4 (·) and Conv 5 (·) represent the first and second convolutional layers in RB, σ 1(·) represents the ReLU activation function in RB, and S is a scaling factor.

[0090] We place an AFB behind each RB to improve the network's adaptability to signal-to-noise ratio, SCR, and SLR. represents the features of the kth RB output, where c is the number of features and h×w is the size of each feature.

[0091] First, AFB processes features through a global average pooling function Right now:

[0092]

[0093] Afterwards, and signal-to-noise ratio γ, SCRr c and SLRr l Connect to form context information as follows:

[0094]

[0095] In some examples, in order not to increase the complexity of the neural network too much, a simple neural network consisting of two fully connected (FC) layers is used to predict the weight coefficients for each channel. The first FC layer contains a ReLU and the last FC layer contains a sigmoid, which limits the output range to the interval (0,1). Therefore, the context information is passed to a two-layer FC neural network, which produces a weight coefficient vector W k :

[0096]

[0097] Among them, FC 1 (·) and FC 2 (·) represent the first and second fully connected layers in AFB, σ 2 (·) and σ 3 (·) represent the ReLU and sigmoid activation functions in AFB respectively. Finally, by and weight coefficient W k Multiply them together to get the weighted features As shown below:

[0098]

[0099] It should be emphasized that the influence of AFB, SNR, SCR and SLR on the weighting coefficients is considered each time. Therefore, by continuously training the network, different weighting coefficients can be assigned according to the differences in SNR, SCR and SLR, so as to obtain the best results in different communication environments.

[0100] Step S5: Pass the reconstructed semantic information through a semantic decoder to obtain the result of the target task.

[0101] The embodiment of the present invention also designs a joint training method for encoding, decoding networks and semantic reconstruction networks. According to the designed joint training method, neural networks are used to train the encoding, decoding networks and semantic reconstruction networks to minimize the loss function of the target task.

[0102] Among them, the loss function of the encoding and decoding network can be expressed as:

[0103]

[0104] in, It represents the prediction result obtained at the receiving end after encoding, compression, channel transmission, reconstruction and decoding. r represents the ground truth value of the database, the subscript i marks the i-th category, and l represents the number of categories.

[0105] The loss function of the semantic reconstruction network can be expressed as:

[0106]

[0107] Among them, Y represents the input semantic information, Represents the output of the semantic reconstruction network, the subscript j marks the jth feature, and c represents the number of features.

[0108] Then the global loss function can be defined as:

[0109]

[0110] By minimizing the global loss function, the optimization algorithm can be used to optimize the model parameters θ, φ, ω. The optimization problem is expressed as:

[0111]

[0112] Specifically, in one embodiment of the present invention, the specific implementation method is as follows:

[0113] (1) For each training round t, the sender uniformly randomly extracts B samples X = [X 1 ,X 2 ,···,X B ] for batch training;

[0114] (2) For each sample, randomly generate its channel signal-to-noise ratio γ i , and

[0115] (3) Extracting semantic information through semantic encoder:

[0116] Y i =E φ (X i ) (14)

[0117] (4) Compressing semantic information through semantic compressor:

[0118] Z i =f(Y i ,r i c )(15)

[0119] (5) Randomly generate channel coefficients and Gaussian white noise Where I is the identity matrix, the noise variance Calculated according to the following formula:

[0120]

[0121] Where P represents the average power of the transmitted signal. In practice, we will use a power normalization operation to ensure that it is equal to a bounded value.

[0122] (6) Calculate the received semantic information:

[0123]

[0124] (7) Reconstructing semantic information using semantic reconstruction network:

[0125]

[0126] (8) The calculation and prediction results are:

[0127]

[0128] (9) Repeat steps (2) to (8) and calculate the average loss function based on the global loss function:

[0129]

[0130] (10) The Adaptive Moment Estimation (ADAM) algorithm is used to optimize the model parameters θ, φ, ω. In each training round t, the parameters can be updated as:

[0131]

[0132] Where η is the learning rate, ρ t and v t are the first and second moment estimates of the gradient, respectively, and ∈ is a small constant to avoid division by zero.

[0133] (11) Repeat steps (1) to (10) until all training rounds are completed and the approximate optimal solution for the parameters is obtained.

[0134] According to the semantic communication method based on adaptive semantic reconstruction proposed in the embodiment of the present invention, a task-oriented semantic communication framework based on adaptive semantic reconstruction is proposed. Through mask operation and the design of a semantic reconstruction network with integrated attention mechanism, flexible compression of semantic information, simulation of data loss during transmission and reconstruction effect adapting to different environments are realized. A joint training method of encoding, decoding network and semantic reconstruction network is designed to minimize the loss function of the target task and obtain the approximate optimal solution of the neural network parameters. Compared with the existing semantic communication schemes, the proposed SCASR scheme has superior adaptability to different signal-to-noise ratio, SCR and SLR conditions, and can obtain higher task execution performance and semantic transmission benefits in communication.

[0135] The embodiment of the present invention also discloses a semantic communication system based on adaptive semantic reconstruction, including a semantic encoder and a semantic compressor at a transmitting end, a semantic reconstruction network and a semantic decoder at a receiving end; the semantic encoder is used to encode input data to obtain semantic information; the semantic compressor is used to compress the semantic information obtained by the semantic encoder, and simulate the data loss during the transmission process through a mask operation to obtain compressed semantic information; the semantic reconstruction network is used to semantically reconstruct the compressed semantic information received by the receiving end to obtain reconstructed semantic information; the semantic reconstruction network includes a combination of multiple stacked residual blocks RB and attention feature blocks AFB; the input of the AFB includes the output of the RB as well as the channel signal-to-noise ratio, the semantic compression rate SCR and the semantic loss rate SLR; the semantic decoder is used to decode the reconstructed semantic information to obtain the result of the target task.

[0136] The embodiment of the present invention further discloses a computer program product, including a computer program / instruction, which implements the steps of the semantic communication method based on adaptive semantic reconstruction when executed by a processor.

[0137] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0138] The matters not described in detail in the present invention are all known technologies to those skilled in the art.

[0139] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A semantic communication method based on adaptive semantic reconstruction, characterized in that: The following steps are involved: Step 1: Obtain semantic information through the semantic encoder at the sending end; Step 2: compress the semantic information according to the semantic compression rate SCR through the semantic compressor, and simulate the data loss during the transmission process through mask operation according to the semantic loss rate SLR to obtain the compressed semantic information; The mask operation is used to simulate the data loss during the transmission process, including: the semantic information Z compressed by the semantic compressor is transmitted through the physical channel, the channel coefficient h and noise N are added to Z, and the semantic information received by the receiving end is Let SLR be r l , the semantic information actually received by the receiver is obtained through the random data loss function Transfer; the random data loss function It consists of two steps. First, a mask generation function is designed, and SLR = r is specified. l , generating a sum A mask vector M2 of the same size, which contains only 0 or 1, the number of 1 is m2, m2 is determined according to SLR, and the positions of 0 and 1 in M2 are random to simulate the process of random data loss; then Multiply the elements in M2 with the elements in M2 to obtain the semantic information actually received by the receiving end; Step 3: Transmit the compressed semantic information to the receiving end; Step 4: The receiving end reconstructs the received signal using a semantic reconstruction network to obtain reconstructed semantic information; the semantic reconstruction network includes a combination of stacked multiple residual blocks RB and attention feature blocks AFB; the input of the AFB includes the output of the RB and the channel signal-to-noise ratio, SCR and SLR; Step 5: Pass the reconstructed semantic information through the semantic decoder to obtain the result of the target task.

2. The semantic communication method based on adaptive semantic reconstruction according to claim 1, characterized in that: In step 2, the semantic information is compressed by a semantic compressor, including: Flatten the semantic information Y into a one-dimensional vector; use the compression function f(Y,r c ) compresses the semantic information Y into a vector Z of size m; wherein m is determined according to SCR; the compression function f(Y, r c ) consists of two steps. First, a mask generation function is designed, and SCR = r c , generate a mask vector M1 of the same size as Y, which only contains 0 or 1, and the number of 1 is m; then multiply the elements in Y with the elements of M1 to obtain the compressed semantic information Z.

3. The semantic communication method based on adaptive semantic reconstruction according to claim 1, characterized in that: In step 4, the calculation process of the semantic reconstruction network is: set up represents the semantic information of the input of the semantic reconstruction network, then the output of the semantic reconstruction network express: Among them, Conv1(·), Conv2(·), and Conv3(·) represent the first, second, and third convolutional layers, respectively. represents the output of the last AFB, K is the number of iterations of RB and AFB; Calculated as: in, represents the output of the (k-1)th AFB, γ, r c and r l They are channel signal-to-noise ratio, SCR and SLR, RB k (·) and AFB k (·) represent the kth RB and the kth AFB, respectively.

4. The semantic communication method based on adaptive semantic reconstruction according to claim 3 is characterized in that: The output of the kth RB Calculated as: Among them, Conv4(·) and Conv5(·) represent the first and second convolutional layers in RB, respectively, σ1(·) represents the ReLU activation function in RB, and S is a scaling factor.

5. The semantic communication method based on adaptive semantic reconstruction according to claim 3, characterized in that: An AFB is placed behind each RB to improve the network's adaptive ability to signal-to-noise ratio, SCR, and SLR. represents the features of the kth RB output, where c is the number of features and h×w is the size of each feature; First, AFB processes features through a global average pooling function Afterwards, and signal-to-noise ratio γ, SCRr c and SLRr l Connect to form context information: A neural network consisting of two fully connected layers is used to predict the weight coefficient context information of each channel is passed to a two-layer FC neural network to produce a weight coefficient vector Among them, FC1(·) and FC2(·) represent the first and second fully connected layers in AFB, respectively, σ2(·) and σ3(·) represent the ReLU and sigmoid activation functions in AFB, respectively; Finally, by adding the features and weight coefficient W k Multiply them together to get the weighted features 6. The semantic communication method based on adaptive semantic reconstruction according to claim 1, characterized in that: The semantic encoder, the semantic decoder and the semantic reconstruction network are trained by a joint training method of the encoding network, the decoding network and the semantic reconstruction network to minimize the loss function of the target task; wherein the loss function of the encoding network and the decoding network is expressed as: in, It represents the prediction result obtained at the receiving end after encoding, compression, channel transmission, reconstruction and decoding. r represents the true value, the subscript i marks the i-th category, and l represents the number of categories; The loss function of the semantic reconstruction network is expressed as: Among them, Y represents the input semantic information, represents the output of the semantic reconstruction network, the subscript j marks the jth feature, and c represents the number of features; The global loss function is defined as: The model parameters θ, φ, ω are optimized by minimizing the global loss function.

7. The semantic communication method based on adaptive semantic reconstruction according to claim 1, characterized in that: The steps of training the semantic encoder, semantic decoder and semantic reconstruction network specifically include: (1) For each training round t, the sender extracts B samples X = [X1, X2, ···, X B ] for batch training; (2) For each sample X i , randomly generate its channel signal-to-noise ratio γ i , and (3) Through the semantic encoder E φ Extracting semantic information: Y i =E φ (X i ); (4) Compress the semantic information through the semantic compressor f: (5) Randomly generate channel coefficients and Gaussian white noise in is the noise variance, according to γ i Determine that I is the identity matrix; (6) Calculate the received semantic information based on the loss function g: (7) Using semantic reconstruction network Reconstructing semantic information: (8) Through the semantic decoder D θ The calculation prediction results are: (9) Repeat steps (2) to (8) and calculate the average loss based on the global loss function; (10) Adopting adaptive moment estimation algorithm to optimize model parameters θ, φ, ω; (11) Repeat steps (1) to (10) until all training rounds are completed and the approximate optimal solution for the parameters is obtained.

8. A semantic communication system based on adaptive semantic reconstruction, characterized in that: It includes a semantic encoder and a semantic compressor at the sending end, and a semantic reconstruction network and a semantic decoder at the receiving end; The semantic encoder is used to encode the input data to obtain semantic information; The semantic compressor is used to compress the semantic information obtained by the semantic encoder according to the semantic compression rate SCR, and to simulate the data loss during the transmission process through a mask operation according to the semantic loss rate SLR to obtain the compressed semantic information; The mask operation is used to simulate the data loss during the transmission process, including: the semantic information Z compressed by the semantic compressor is transmitted through the physical channel, the channel coefficient h and noise N are added to Z, and the semantic information received by the receiving end is Let SLR be r l , the semantic information actually received by the receiver is obtained through the random data loss function Transfer; the random data loss function It consists of two steps. First, a mask generation function is designed, and SLR = r is specified. l , generating a sum A mask vector M2 of the same size, which contains only 0 or 1, the number of 1 is m2, m2 is determined according to SLR, and the positions of 0 and 1 in M2 are random to simulate the process of random data loss; then Multiply the elements in M2 with the elements in M2 to obtain the semantic information actually received by the receiving end; The semantic reconstruction network is used to semantically reconstruct the compressed semantic information received by the receiving end to obtain the reconstructed semantic information; the semantic reconstruction network includes a combination of stacked multiple residual blocks RB and attention feature blocks AFB; the input of the AFB includes the output of the RB and the channel signal-to-noise ratio, SCR and SLR; The semantic decoder is used to decode the reconstructed semantic information to obtain the result of the target task.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the semantic communication method based on adaptive semantic reconstruction according to any one of claims 1 to 7 are implemented.

Images