Wireless relay semantic communication method and system adaptable to multi-task

Abstract

The present invention provides a multi-task-adaptive wireless relay semantic communication method and system, comprising: step S1: inputting and transmitting an image; training an encoder at the transmitting end and a decoder at the relay base station to restore the image and obtain a relay restored image; step S2: training a classifier at the relay base station on the relay restored image space to obtain classification content output by the relay base station classifier; step S3: using cross-attention to fuse information based on the relay base station encoder, the relay restored image, and the classification content to obtain an information fusion result; step S4: using the relay base station signal and the direct link signal as input to obtain a semantic task result; and step S5: restoring the transmitted image. Based on relevant machine learning technologies and targeted at image transmission, the present invention can effectively improve the multi-task performance of the receiving end by utilizing information in the direct channel, thereby achieving a more efficient semantic communication system.

Description

Technical Field

[0001] The present invention belongs to the technical field of wireless communications, and in particular, relates to a wireless relay semantic communication method and system adaptable to multiple tasks. Background Art

[0002] With the development of mobile communication systems from the first generation 1G to the fifth generation 5G, the physical dimension of information transmission is approaching the limit of Shannon's information theory. However, the new generation of 6G technology proposes semantic communication that realizes high-speed and low-latency interconnection between objects by transmitting key semantic elements. Semantic communication is a communication method that emphasizes the accurate retrieval and transmission of meaning from the source to the receiving end. Unlike traditional communication systems, it no longer focuses on the accurate transmission of symbols, but pays more attention to the meaning of communication. Semantic communication can achieve significant advantages in source data compression, reducing communication bandwidth pressure and supporting new intelligent services. Figure 1 As shown in the figure, the existing end-to-end semantic communication model framework generally uses a deep learning model to replace the source and channel encoders in traditional communication, combines the source channel coding, and uses one model to complete the source-to-signal encoding.

[0003] In real-world applications, when the transmitter and receiver are far apart and the wireless channel environment is poor, a relay base station is often used to forward information. Traditional methods often ignore the semantic information in the direct channel between the transmitter and receiver. However, this semantic information can actually be leveraged to enhance the quality of semantic information at the receiver and achieve specific tasks. By effectively utilizing this semantic information in the direct channel, not only can the reliability and efficiency of communication be improved, but high transmission quality can also be maintained in complex channel environments, further optimizing the performance of the overall communication system.

[0004] Patent document CN115883018A discloses a semantic communication system. This solution selects a corresponding source-channel joint coding model. The source-channel joint coding model encodes semantic coding information, key feature coding information, semantic decoding model parameters, and channel transmission environment parameters to form a second service information and sends it to the receiving end; the receiving end receives the second service information, and performs source-channel decoding and semantic decoding to obtain semantic information and key feature information; the key feature information is used to verify the semantic information, and after passing the verification, the semantic information is restored to obtain the third service information. This reduces the communication transmission volume and ensures the accuracy of communication transmission. However, in solving the problem of multi-task semantic communication, it was found that when the channel environment is poor, the problem of using relay base stations for multi-task semantic communication has not been solved. Summary of the Invention

[0005] In view of the defects in the prior art, the purpose of the present invention is to provide a wireless relay semantic communication method and system that is adaptable to multiple tasks.

[0006] A multi-task adaptive wireless relay semantic communication method provided by the present invention includes:

[0007] Step S1: input and transmit an image; train the encoder at the transmitter and the decoder at the relay base station to restore the image and obtain a relay restored image;

[0008] Step S2: training the relay base station classifier in the relay restored image space to obtain classification content output by the relay base station classifier;

[0009] Step S3: Based on the encoder of the relay base station, the relay restored image and the classified content, cross attention is used to perform information fusion to obtain the information fusion result;

[0010] Step S4: Based on the information fusion result, the relay base station signal and the direct link signal are used as input to obtain the semantic task result;

[0011] Step S5: Restore the transmitted image based on the semantic task result and the relay base station signal.

[0012] Preferably, the transmitting end, the relay base station and the receiving end adopt source-channel joint coding.

[0013] Preferably, in step S5: the loss function is a weighted sum of the mean square error of the decoder and the cross entropy error of the classifier.

[0014] Preferably, in step S1: an MSE function is used to train the encoder of the transmitting end and the decoder of the relay base station.

[0015] Preferably, in step S2: a CE loss function is used to train a classifier of the relay base station in the relay restoration image space.

[0016] According to the present invention, a multi-task adaptive wireless relay semantic communication system is provided, comprising:

[0017] Module M1: Input and transmit an image; train the encoder at the transmitter and the decoder at the relay base station to recover the image and obtain the relay recovered image;

[0018] Module M2: training the relay base station classifier in the relay restoration image space to obtain the classification content output by the relay base station classifier;

[0019] Module M3: Based on the encoder of the relay base station, the relay restores the image and classification content, and uses cross attention to perform information fusion to obtain the information fusion result;

[0020] Module M4: Based on the information fusion results, it takes the relay base station signal and the direct link signal as input to obtain the semantic task results;

[0021] Module M5: Restore the transmitted image based on the semantic task results and the relay base station signal.

[0022] Preferably, the transmitting end, the relay base station and the receiving end adopt source-channel joint coding.

[0023] Preferably, in the module M5: the loss function is a weighted sum of the mean square error of the decoder and the cross entropy error of the classifier.

[0024] Preferably, in the module M1: an MSE function is used to train the encoder of the transmitting end and the decoder of the relay base station.

[0025] Preferably, in the module M2: a CE loss function is used to train a classifier of the relay base station in the relay restoration image space.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] 1. Based on machine learning-related technologies and aimed at image transmission, the present invention provides a wireless relay semantic communication system that simultaneously realizes multi-task transmission. By utilizing the information in the direct connection channel, the system can effectively improve the multi-task performance of the receiving end and realize a more efficient semantic communication system.

[0028] 2. The present invention uses relay semantic communication to integrate information from direct transmission links, solving the problem of how to use relay base stations for multi-task semantic communication when the channel environment is poor.

[0029] 3. The present invention uses the results of completed tasks to assist unfinished tasks through information fusion based on cross-attention, so as to improve the performance of all semantic tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0031] Figure 1 Schematic diagram of the end-to-end semantic communication model provided by the present invention;

[0032] Figure 2 A schematic diagram of a semantic communication model using a direct channel provided by the present invention;

[0033] Figure 3 This is a schematic diagram of the network structure of the communication system provided by the present invention. DETAILED DESCRIPTION

[0034] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0035] The concept behind this invention is to use relay semantic communication to fuse information from direct links, addressing the problem of multi-task semantic communication using relay base stations in poor channel environments. Furthermore, the invention addresses the issue of sharing multi-task results in multi-task semantic communication. This invention uses cross-attention-based information fusion to use the results of completed tasks to assist unfinished tasks, thereby improving the performance of all semantic tasks.

[0036] Specifically, the present invention aims to consider a scenario with a relay base station, and provide a wireless relay semantic communication system for image transmission that simultaneously realizes multi-task transmission, wherein the receiving end uses the direct connection channel between the receiving end and the sending end and the relay communication link from the sending end to the relay base station and then to the receiving end to realize the semantic task of image transmission, and the receiving end uses the relay communication link and the semantic task result to realize the image restoration task of image transmission.

[0037] This invention proposes a method for enhancing the performance of a multi-task relay semantic communication system using direct links in a relay scenario. This method is not limited to any specific codec network, semantic task type, or image content transmission format. Any semantic communication model that adopts this concept falls within the scope of this invention.

[0038] A multi-task adaptive wireless relay semantic communication method provided by the present invention includes:

[0039] Step S1: Input and transmit an image; use the MSE function to train the encoder at the transmitter and the decoder at the relay base station to restore the image and obtain a relay restored image;

[0040] Step S2: Based on the relay restored image, the CE loss function is used to train the relay base station classifier in the relay restored image space to obtain the classification content output by the relay base station classifier;

[0041] Step S3: Based on the encoder of the relay base station, the relay restored image and the classified content, cross attention is used to perform information fusion to obtain the information fusion result;

[0042] Step S4: Based on the information fusion result, the relay base station signal and the direct link signal are used as input to obtain the semantic task result;

[0043] Step S5: Restore the transmitted image based on the semantic task result and the relay base station signal.

[0044] Specifically, during transmission, the goal of multi-task semantic communication is to restore the image and perform classification tasks. A Swin Transformer-based network is used as a joint source channel encoder and decoder. A cross-attention mechanism is added to the relay encoder and receiver decoder to jointly process multiple tasks and multiple signals. The loss function for training the decoder, that is, training the decoder, uses mean squared error (MSE). The loss function for training the classifier, that is, training the classifier, uses cross entropy (CE), and its performance is represented by classification accuracy. The specific training steps include:

[0045] Step 1: Train the encoder at the transmitter and the decoder at the relay base station, using the MSE function as the loss function to complete the image restoration task and obtain the relay restored image;

[0046] Step 2: Train the relay base station classifier in the image space restored by the relay, using the CE loss function.

[0047] Step 3: During training, the encoder at the relay base station and the decoder and classifier at the receiving end perform information fusion.

[0048] Among them, encoder is encoder; decoder is decoder; classifier is classifier;

[0049] Specifically, the encoder of the relay base station takes the image restored by the decoder of the relay base station and the classification result output by the classifier of the relay base station as input, uses cross attention to fuse the information, and sends it to the receiver;

[0050] The receiving classifier takes the relay base station signal and the direct link signal as input and obtains the classification result. In other words, the receiving classifier takes the signal re-encoded by the relay base station and sent to the receiving end as input, and the signal encoded by the transmitting encoder and sent directly to the receiving end as input to obtain the classification result.

[0051] The decoder takes the classification result and the relay base station signal as input and recovers the image. The loss function is the weighted sum of the decoder's mean square error and the classifier's cross entropy error.

[0052] In other words, the technical solution adopted by the present invention is as follows:

[0053] Specifically, the transmitting end, the relay base station and the receiving end all adopt source-channel joint coding, that is, one codec is used to simultaneously complete source and channel coding and decoding.

[0054] The relay base station adopts a semantic forwarding method to recover the semantic information from the signal sent by the transmitter, and then re-encodes the information through the source-channel joint encoding, that is, uses cross-attention to fuse the information before sending it to the receiver.

[0055] After receiving the information from the sender through the direct channel and the relay communication link, the receiver completes the semantic task of the transmitted image through the joint source channel decoder and obtains the semantic task result as the classification result.

[0056] The receiving end inputs the semantic task result as the classification result and the information transmitted from the relay base station, that is, the relay base station signal, into the source-channel joint coding to restore the image transmitted by the sending end.

[0057] The present invention also provides a wireless relay semantic communication system that is adaptable to multiple tasks. The wireless relay semantic communication system that is adaptable to multiple tasks can be implemented by executing the process steps of the wireless relay semantic communication method that is adaptable to multiple tasks. That is, those skilled in the art can understand the wireless relay semantic communication method that is adaptable to multiple tasks as an optimal implementation of the wireless relay semantic communication system that is adaptable to multiple tasks.

[0058] According to the present invention, a multi-task adaptive wireless relay semantic communication system is provided, comprising:

[0059] Module M1: Input and transmit an image; train the encoder at the transmitter and the decoder at the relay base station to recover the image and obtain the relay recovered image;

[0060] Module M2: training the relay base station classifier in the relay restoration image space to obtain the classification content output by the relay base station classifier;

[0061] Module M3: Based on the encoder of the relay base station, the relay restores the image and classification content, and uses cross attention to perform information fusion to obtain the information fusion result;

[0062] Module M4: Based on the information fusion results, it takes the relay base station signal and the direct link signal as input to obtain the semantic task results;

[0063] Module M5: Restore the transmitted image based on the semantic task results and the relay base station signal.

[0064] Specifically, the transmitting end, the relay base station and the receiving end adopt source-channel joint coding.

[0065] Specifically, in the module M5: the loss function is the weighted sum of the mean square error of the decoder and the cross entropy error of the classifier.

[0066] Specifically, in the module M1: the MSE function is used to train the encoder of the transmitting end and the decoder of the relay base station.

[0067] Specifically, in the module M2: a CE loss function is used to train the classifier of the relay base station in the relay restoration image space.

[0068] Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, and units provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same functions of the system and its various devices, modules, and units provided by the present invention in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; the devices, modules, and units for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0069] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A wireless relay semantic communication method adapted to multiple tasks, characterized in that: include: Step S1: input and transmit image; Train the encoder at the transmitter and the decoder at the relay base station to restore the image and obtain the relay restored image; Step S2: training the relay base station classifier in the relay restored image space to obtain classification content output by the relay base station classifier; Step S3: Based on the encoder of the relay base station, the relay restored image and the classified content, cross attention is used to perform information fusion to obtain the information fusion result; Step S4: Based on the information fusion result, the relay base station signal and the direct link signal are used as input to obtain the semantic task result; Step S5: Restore the transmitted image based on the semantic task result and the relay base station signal.

2. The multi-task adaptive wireless relay semantic communication method according to claim 1, characterized in that: The transmitting end, the relay base station and the receiving end adopt source-channel joint coding.

3. The multi-task adaptive wireless relay semantic communication method according to claim 2, characterized in that: In step S5: the loss function is the weighted sum of the mean square error of the decoder and the cross entropy error of the classifier.

4. The multi-task adaptive wireless relay semantic communication method according to claim 3, characterized in that: In the step S1: the MSE function is used to train the encoder of the transmitting end and the decoder of the relay base station.

5. The wireless relay semantic communication method adapted to multiple tasks according to claim 4, characterized in that: In the step S2: using the CE loss function, the classifier of the relay base station is trained in the relay restoration image space.

6. A wireless relay semantic communication system adapted to multiple tasks, characterized in that: include: Module M1: input and transmit image; Train the encoder at the transmitter and the decoder at the relay base station to restore the image and obtain the relay restored image; Module M2: training the relay base station classifier in the relay restoration image space to obtain the classification content output by the relay base station classifier; Module M3: Based on the encoder of the relay base station, the relay restores the image and classification content, and uses cross attention to perform information fusion to obtain the information fusion result; Module M4: Based on the information fusion results, it takes the relay base station signal and the direct link signal as input to obtain the semantic task results; Module M5: Restore the transmitted image based on the semantic task results and the relay base station signal.

7. The multi-task adaptive wireless relay semantic communication system according to claim 6, characterized in that: The transmitting end, the relay base station and the receiving end adopt source-channel joint coding.

8. The multi-task adaptive wireless relay semantic communication system according to claim 7, characterized in that: In the module M5: the loss function is the weighted sum of the mean square error of the decoder and the cross entropy error of the classifier.

9. The multi-task adaptive wireless relay semantic communication system according to claim 8, characterized in that: In the module M1: the MSE function is used to train the encoder of the transmitting end and the decoder of the relay base station.

10. The multi-task adaptive wireless relay semantic communication system according to claim 9, characterized in that: In the module M2: a CE loss function is used to train the classifier of the relay base station in the relay restoration image space.

Images