Semantic communication system and method and related equipment

Abstract

The invention provides a semantic communication system and method and related equipment, and relates to the technical field of communication. The system comprises a transmitter used for calling a corresponding semantic encoder and a channel encoder for data to be transmitted in a target semantic communication scene to perform semantic encoding and channel encoding, and obtaining and sending encoded semantic information; the receiver is used for calling a corresponding channel decoder and a semantic decoder to carry out channel decoding and semantic decoding on the encoded semantic information to obtain recovery data, obtaining the semantic correct rate of the data to be transmitted according to the recovery data and the data to be transmitted, and sending the semantic correct rate to the receiver; and if the semantic correct rate of the to-be-transmitted data is lower than the semantic tolerance of the target semantic communication scene, retransmission indication information is generated and sent to indicate the transmitter to retransmit the to-be-transmitted data. Wherein the semantic tolerance is used for representing the lowest correct rate of the semantic index of the acceptable recovery data in the target semantic communication scene; the numerical value of the semantic index is semantic accuracy.

Description

Background Art

[0002] In related technologies, communication is mainly based on bit transmission, which transmits the entire content of a given bit data stream as quickly and reliably as possible by optimizing traditional key performance indicators such as throughput, latency and packet loss rate. Most wireless communication network systems are built on the basic principle of reliable communication in noisy channels. In particular, at the bottom layer of the communication network, the meaning, importance and effectiveness of the transmitted information are relatively ignored.

[0003] For example, when transmitting an image, the receiver's goal is to accurately receive all pixel values. However, some IoT real-time systems only need to select some meaningful information from the data stream corresponding to the image to achieve specific goals. Ignoring the meaning of information and blindly generating and transmitting large amounts of data will cause great waste, leading to unnecessary network congestion, expiration of useful information, and low spectrum utilization. Summary of the invention

[0004] The disclosed embodiment provides a semantic communication system, including: a transmitter, used to call a corresponding semantic encoder and a channel encoder to perform semantic encoding and channel encoding on the data to be transmitted in the target semantic communication scenario, and obtain and send the encoded semantic information; a receiver, used to receive the encoded semantic information, call a corresponding channel decoder and a semantic decoder to perform channel decoding and semantic decoding on the encoded semantic information, and obtain recovery data, and obtain the semantic accuracy of the data to be transmitted based on the recovery data and the data to be transmitted, if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, then generate and send retransmission indication information to instruct the transmitter to retransmit the data to be transmitted; wherein the semantic tolerance is used to characterize the minimum accuracy of the semantic indicator of the recovered data that can be accepted in the target semantic communication scenario; the value of the semantic indicator is the semantic accuracy.

[0005] An embodiment of the present disclosure provides a semantic communication method, which includes: performing semantic encoding and channel encoding on data to be transmitted in a target semantic communication scenario to obtain encoded semantic information; performing channel decoding and semantic decoding on the encoded semantic information to obtain recovered data, and obtaining the semantic accuracy of the data to be transmitted based on the recovered data and the data to be transmitted, and if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, generating and sending retransmission indication information to indicate retransmission of the data to be transmitted; wherein the semantic tolerance is used to characterize the minimum accuracy of the semantic indicator of the recovered data that can be accepted in the target semantic communication scenario; the value of the semantic indicator is the semantic accuracy.

[0006] The embodiment of the present disclosure provides a semantic communication method, which includes: determining an original semantic additional block; calling corresponding semantic encoders to perform semantic encoding on the original source transmission block and the original semantic additional block in the target semantic communication scenario, respectively, to obtain the semantically encoded source transmission block and semantic check bits; calling a channel encoder to perform channel encoding on the semantically encoded source transmission block and the semantic check bits, to obtain encoded semantic information; sending the encoded semantic information to perform channel decoding and semantic decoding based on the encoded semantic information to obtain a restored source transmission block and a restored semantic additional block, and obtaining a semantic accuracy rate based on the restored semantic additional block and the original semantic additional block, and if the semantic accuracy rate is lower than the semantic tolerance of the target semantic communication scenario, generating and sending retransmission indication information to indicate retransmission of the original source transmission block.

[0007] An embodiment of the present disclosure provides a semantic communication method, which includes: receiving encoded semantic information; performing channel decoding and semantic decoding on the encoded semantic information to obtain recovered data; obtaining the semantic accuracy of the data to be transmitted based on the recovered data and the data to be transmitted; if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, generating and sending retransmission indication information to indicate retransmission of the data to be transmitted; wherein the semantic tolerance is used to characterize the minimum accuracy of the semantic indicator of the recovered data that can be accepted in the target semantic communication scenario; the value of the semantic indicator is the semantic accuracy.

[0008] The embodiment of the present disclosure provides a transmitter, including: a first processing unit, used to determine an original semantic additional block; the first processing unit is also used to call corresponding semantic encoders to perform semantic encoding on the original source transmission block and the original semantic additional block in the target semantic communication scenario, respectively, to obtain the semantically encoded source transmission block and semantic check bits; the first processing unit is also used to call a channel encoder to perform channel encoding on the semantically encoded source transmission block and semantic check bits, to obtain encoded semantic information; a first sending unit is used to send the encoded semantic information, so as to perform channel decoding and semantic decoding based on the encoded semantic information, to obtain a restored source transmission block and a restored semantic additional block, to obtain a semantic accuracy rate according to the restored semantic additional block and the original semantic additional block, and if the semantic accuracy rate is lower than the semantic tolerance of the target semantic communication scenario, to generate and send retransmission indication information to indicate retransmission of the original source transmission block.

[0009] An embodiment of the present disclosure provides a receiver, including: a second receiving unit, used to receive encoded semantic information; a second processing unit, used to perform channel decoding and semantic decoding on the encoded semantic information to obtain recovered data; the second processing unit is also used to obtain the semantic accuracy of the data to be transmitted based on the recovered data and the data to be transmitted; the second processing unit is also used to generate retransmission indication information to indicate retransmission of the data to be transmitted if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario; a second sending unit is used to send the retransmission indication information; wherein the semantic tolerance is used to characterize the minimum accuracy of the semantic indicator of the recovered data that can be accepted in the target semantic communication scenario; the value of the semantic indicator is the semantic accuracy.

[0010] An embodiment of the present disclosure provides a semantic communication system, which includes: a transmitter as described in any embodiment of the present disclosure and a receiver as described in any embodiment of the present disclosure.

[0011] The present disclosure provides a communication device. In one design, the communication device may include a module corresponding to the method / operation / step / action described in any embodiment of the present disclosure. The module may be a hardware circuit, or software, or a combination of a hardware circuit and software. In one design, the communication device includes: a first processing unit, used to determine an original semantic additional block; the first processing unit is also used to call a corresponding semantic encoder to perform semantic encoding on the original source transmission block and the original semantic additional block in the target semantic communication scenario, respectively, to obtain a semantically encoded source transmission block and a semantic check bit; the first processing unit is also used to call a channel encoder to perform channel encoding on the semantically encoded source transmission block and the semantic check bit, to obtain encoded semantic information; a first sending unit is used to send the encoded semantic information, so as to perform channel decoding and semantic decoding based on the encoded semantic information, to obtain a restored source transmission block and a restored semantic additional block, to obtain a semantic accuracy rate according to the restored semantic additional block and the original semantic additional block, and if the semantic accuracy rate is lower than the semantic tolerance of the target semantic communication scenario, then generate and send retransmission indication information to indicate retransmission of the original source transmission block. In another design, the communication device includes: a second receiving unit, used to receive encoded semantic information; a second processing unit, used to perform channel decoding and semantic decoding on the encoded semantic information to obtain recovered data; the second processing unit is also used to obtain the semantic accuracy of the data to be transmitted based on the recovered data and the data to be transmitted; the second processing unit is also used to generate retransmission indication information to indicate retransmission of the data to be transmitted if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario; a second sending unit is used to send the retransmission indication information; wherein the semantic tolerance is used to characterize the minimum accuracy of the semantic indicator of the recovered data that can be accepted in the target semantic communication scenario; the value of the semantic indicator is the semantic accuracy.

[0012] An embodiment of the present disclosure provides a communication device, comprising: a processor. The processor can implement the method in any embodiment of the present disclosure. Optionally, the communication device also includes a memory, and the processor is coupled to the memory, and can be used to execute a computer program in the memory to implement the method in any embodiment of the present disclosure. Optionally, the communication device also includes a communication interface, and the processor is coupled to the communication interface. In the embodiment of the present disclosure, the communication interface can be a transceiver, a pin, a circuit, a bus, a module, or other types of communication interfaces, without limitation.

[0013] The embodiment of the present disclosure further provides a processor, including: an input circuit, an output circuit and a processing circuit. The processing circuit is used to receive a signal through the input circuit and transmit a signal through the output circuit, so that the processor executes the method in any embodiment of the present disclosure.

[0014] In an exemplary embodiment, the processor may be one or more chips, the input circuit may be an input pin, the output circuit may be an output pin, and the processing circuit may be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit may be received and input by, for example, but not limited to, a receiver, and the signal output by the output circuit may be output to a transmitter and transmitted by the transmitter, for example, but not limited to, and the input circuit and the output circuit may be the same circuit, which is used as an input circuit and an output circuit at different times. The embodiments of the present disclosure do not limit the specific implementation of the processor and various circuits.

[0015] The embodiment of the present disclosure also provides a communication system, including at least one communication device in the embodiment of the present disclosure.

[0016] An embodiment of the present disclosure provides a communication device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the semantic communication method in any embodiment of the present disclosure by executing the executable instructions.

[0017] An embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the semantic communication method in any embodiment of the present disclosure is implemented.

[0018] An embodiment of the present disclosure provides a computer program product, including a computer program, which implements the semantic communication method in any embodiment of the present disclosure when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A structural schematic diagram of a semantic communication system in an embodiment of the present disclosure is shown.

[0020] Figure 2 A schematic diagram of a network system architecture of a semantic communication system in an embodiment of the present disclosure is shown.

[0021] Figure 3 A schematic diagram of the architecture of a semantic communication system in an embodiment of the present disclosure is shown.

[0022] Figure 4 A flowchart of a semantic communication method in an embodiment of the present disclosure is shown.

[0023] Figure 5 A flow chart showing another semantic communication method in an embodiment of the present disclosure.

[0024] Figure 6 A flowchart of yet another semantic communication method in an embodiment of the present disclosure is shown.

[0025] Figure 7 A flowchart of another semantic communication method in an embodiment of the present disclosure is shown.

[0026] Figure 8 A structural block diagram of a transmitting end in an embodiment of the present disclosure is shown.

[0027] Fig. 9 A structural block diagram of a receiving end in an embodiment of the present disclosure is shown.

[0028] Fig.10 A structural block diagram of a communication device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0030] In addition, the accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0031] In the present disclosure, at least one (item) can be described as one (item) or more (items), and more (items) can be two (items), three (items), four (items) or more (items), without limitation. " / " can indicate that the objects associated before and after are in an "or" relationship, for example, A / B can indicate A or B; "and / or" can be used to describe that there are three relationships between associated objects, for example, A and / or B can indicate: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In order to facilitate the description of the technical solution of the present application, the words "first", "second", "A", or "B" can be used to distinguish technical features with the same or similar functions. The words "first", "second", "A", or "B" do not limit the quantity and execution order. Moreover, the words "first", "second", "A", or "B" do not necessarily limit them to be different. The words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions, and any design described as "exemplary" or "for example" should not be interpreted as being more preferred or more advantageous than other design solutions. The use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way for easy understanding.

[0032] The technical solution provided in the embodiments of the present disclosure can be applied to wireless communication between communication devices. Wireless communication between communication devices may include: wireless communication between network devices and terminals, wireless communication between network devices and network devices, and wireless communication between terminals. In the embodiments of the present disclosure, the term "wireless communication" may also be referred to as "communication", and the term "communication" may also be described as "data transmission", "information transmission" or "transmission".

[0033] A semantic communication method is provided in an embodiment of the present disclosure, and the method can be executed by any communication device with information processing capabilities. In some embodiments, the semantic communication method provided in the embodiment of the present disclosure can be executed by a terminal of the system architecture; in other embodiments, the semantic communication method provided in the embodiment of the present disclosure can be implemented by a network device in the system architecture. In some further embodiments, the semantic communication method provided in the embodiment of the present disclosure can be implemented by a terminal and a network device in the system architecture through interaction. In some embodiments, the terminal can act as a receiver or sender of information / data. In some embodiments, the network device can act as a sender or receiver of information / data.

[0034] Figure 1 A structural schematic diagram of a semantic communication system in an embodiment of the present disclosure is shown.

[0035] Shannon divided communication into three levels in information theory: syntax, semantics and pragmatics. The pragmatic level mainly explores how the received semantics affects the system behavior in the desired way.

[0036] The grammatical level is a transmission problem, which mainly studies how to accurately transmit the symbols of communication, which can also be called symbol transmission. Traditional communication technology does not consider the specific meaning of the communication content, and designs the communication system from an engineering perspective, which belongs to the category of grammatical communication. The communication systems in related technologies are all at the grammatical level, facing development bottlenecks, and it is difficult to break through the Shannon limit.

[0037] The semantic level is a semantic problem. It mainly studies how to accurately convey the semantics in communication symbols, which can also be called semantic transmission. Semantic communication considers the communication content itself more. With the help of the common semantic knowledge base between the sender and the receiver, only the key content needs to be transmitted to enable the receiver to recover the data information to be sent by the sender, thereby achieving higher communication capacity at a lower cost. Semantic communication designs communication systems from a higher dimension and improves the performance of the system from a new level.

[0038] With the rapid development of Internet of Things (IoT), virtual reality, automatic transportation, environmental monitoring and telehealth, the surge in data transmission in the Internet of Things and multimedia has put forward higher demands on limited wireless spectrum resources and energy resources. Semantic communication can effectively save energy and spectrum resources by transmitting the meaning of the message rather than the message itself, and has strong robustness in low signal-to-noise ratio areas, showing outstanding advantages over traditional communications. The development of technologies such as artificial intelligence and natural language processing has made it possible to explore the second level of semantic communication, and semantic communication has gradually become a major research trend in the field of communications.

[0039] Semantic communication is a task-based, "understanding first, then transmission" communication method. It selectively extracts features, compresses and transmits the original signal, and then uses semantic level information for communication. If grammatical communication is regarded as formal communication, then semantic communication is "meaning communication" and "content communication". The semantic communication system in the embodiment of the present disclosure is a communication system for realizing semantic communication.

[0040] The ultimate goal of communication is for the two parties to interact so that the receiver can understand the information content of the sender, that is, "convey the meaning". Compared with grammatical communication, when transmitting the same semantic information, semantic communication can reduce the amount of transmitted data, reduce the requirements for channel bandwidth, and solve the problem of the gradual saturation of wireless spectrum resources. In addition, semantic communication can also help cope with harsh channel environments and improve the reliability of communication in low signal-to-noise ratio areas.

[0041] Semantic communication may have a wider range of applications in 6G and even 7G networks, and can be combined with vertical industries to create greater application prospects. For example, semantic communication for multimedia data transmission in specific scenarios of 6G networks. This disclosure combines the characteristics of semantic communication and proposes a corresponding network architecture, which is an important deployment for future communication systems.

[0042] like Figure 1 As shown, the semantic communication system 10 provided by the embodiment of the present disclosure includes a transmitter 11 and a receiver 12 .

[0043] The transmitter 11 is used to call the corresponding semantic encoder and channel encoder to perform semantic encoding and channel encoding on the data to be transmitted in the target semantic communication scenario, and obtain and send the encoded semantic information.

[0044] The receiver 12 is used to receive the encoded semantic information, call the corresponding channel decoder and semantic decoder to perform channel decoding and semantic decoding on the encoded semantic information, obtain recovered data, and obtain the semantic accuracy of the data to be transmitted based on the recovered data and the data to be transmitted. If the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, retransmission indication information is generated and sent to instruct the transmitter to retransmit the data to be transmitted.

[0045] The semantic tolerance is used to characterize the minimum correctness of the semantic indicator of the restored data that is acceptable in the target semantic communication scenario; the value of the semantic indicator is the semantic correctness.

[0046] The semantic communication system provided by the embodiments of the present disclosure is a universal semantic communication system, which can be used in different semantic communication scenarios, and the semantic communication scenario currently applicable to it is called the target semantic communication scenario. The target semantic communication scenario refers to the application scenario of semantic communication that the semantic communication system currently wants to implement, such as one or more of the semantic transmission of text information, the semantic transmission of voice information, the semantic transmission of image information, etc.

[0047] In the embodiments of the present disclosure, corresponding semantic indicators and semantic tolerances may be provided for different target semantic communication scenarios, that is, the embodiments of the present disclosure provide a target-oriented general semantic communication system, where the target may refer to the semantic indicators and semantic tolerances set for the target semantic communication scenarios.

[0048] In an exemplary embodiment, when the target semantic communication scenario is the semantic transmission of text information, a time recurrent neural network is used as a semantic encoder and a semantic decoder; the semantic indicators use one or more of the word error rate WER, the bilingual evaluation replacement score BLEU, the sentence similarity (Sentence Similarity), etc.

[0049] In some embodiments, the time recurrent neural network may include a long short-term memory network (Long Short-Term Memory, LSTM), but the present disclosure is not limited thereto. LSTM is a time recurrent neural network that can be used for semantic encoding and decoding of text.

[0050] WER (Word Error Rate) is used to measure the degree of difference between the model output and the reference answer in tasks such as speech recognition or machine translation. WER can be calculated for the entire sentence or text, rather than a single word or character. Therefore, it is very useful in evaluating the overall performance of text transmission.

[0051] BLEU (Bilingual Evaluation Understudy) is a tool used to evaluate the quality of machine translation. The design idea of ​​BLEU is that the closer the machine translation result is to the result of professional human translation, the better. The BLEU algorithm is to judge the similarity between two sentences.

[0052] In an exemplary embodiment, when the target semantic communication scenario is the semantic transmission of voice information, a convolutional neural network is used as a semantic encoder and a semantic decoder, and the semantic indicators use one or more of the signal-distortion ratio SDR and the objective speech quality assessment PESQ.

[0053] CNN (Convolutional Neural Networks) is a type of feed-forward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning and can be used for image and audio processing and restoration.

[0054] SDR (signal distortion rate) is an evaluation index that indicates the similarity between the source signal and the separated signal, which can be expressed in decibels (dB). It can be used to measure the distortion of speech.

[0055] PESQ (Perceptual Evaluation of Speech Quality) is an algorithm for objectively evaluating speech quality. It is designed based on the human perception of speech. The PESQ algorithm predicts the subjective evaluation of speech quality by simulating the processing of speech signals by the human auditory system. The PESQ algorithm takes into account multiple factors that affect speech quality, including background noise, distortion, echo, delay, etc. It calculates a quantitative score to indicate the quality of speech by comparing the difference between the original speech signal and the processed or transmitted speech signal. It evaluates the subjective MOS (Mean Opinion Score) of speech by summing up in frequency and time. MOS is a subjective evaluation standard used to measure user satisfaction with the quality of voice calls in a communication system. It is scored based on the user's subjective perception of speech quality.

[0056] In an exemplary embodiment, when the target semantic communication scenario is the semantic transmission of image information, a convolutional neural network or a graph convolutional neural network is used as a semantic encoder and a semantic decoder, and the semantic indicators use one or more of peak signal-to-noise ratio PSNR, multi-scale structural similarity MS-SSIM, and learning-perceptual image block similarity LPIPS.

[0057] GCN (Graph Convolution Neural Networks) is based on the spectral domain method and defines graph convolution by introducing filters from the perspective of graph signal processing, where the graph convolution operation is interpreted as removing noise from the graph signal and can be used for video compression and reconstruction.

[0058] PSNR (Peak signal-to-noise ratio) is often used to evaluate the performance of measurement methods for signal reconstruction quality in fields such as image compression.

[0059] MS-SSIM (multi-scale structural similarity) is an image quality measurement method that evaluates the similarity between two images from three perspectives: brightness, contrast, and structure. It can be widely used in applications such as image quality assessment, performance evaluation of compression algorithms, image enhancement and restoration, etc.

[0060] LPIPS (Learned Perceptual Image Patch Similarity) is used to measure the difference between two images. LPIPS is not achieved through a simple mathematical formula, but through a deep neural network. The model architecture and parameters are obtained through large-scale training to capture the perceptual information of the image.

[0061] The semantic communication system provided by the embodiments of the present disclosure can be applicable to various target semantic communication scenarios, that is, semantic communication is related to specific communication scenarios. For different target semantic communication scenarios, different semantic encoders, semantic decoders, channel encoders, channel decoders, semantic indicators (also referred to as semantic evaluation indicators / evaluation indicators), semantic tolerance, etc. can be selected. The semantic encoders, semantic decoders, channel encoders, and channel decoders for different target semantic communication scenarios can be trained separately.

[0062] The following three types of semantic communication are used as examples in the embodiments of the present disclosure, but the present disclosure is not limited thereto:

[0063] 1) Text: LSTM can be used as a semantic encoder and semantic decoder. The selected evaluation indicators, such as WER, BLEU, Sentence Similarity, etc., measure the semantic correctness or semantic accuracy from the similarity of phrases and words.

[0064] 2) Speech: CNN can be used as encoder and decoder (e.g. semantic encoder and semantic decoder). The selected evaluation indicators such as SDR and PESQ measure the semantic correctness or semantic accuracy based on the similarity of speech spectrum.

[0065] 3) Pictures / images / videos: CNN / GCN can be used as encoders and decoders (e.g. semantic encoders and semantic decoders). The selected evaluation indicators, such as PSNR and MS-SSIM, measure the semantic correctness or semantic accuracy from the similarity of pixel features. Deep learning-based image similarity indicators such as LPIPS can also be used to measure the semantic correctness or semantic accuracy.

[0066] It should be noted that the above-mentioned target semantic communication scenarios and the corresponding semantic encoders, semantic decoders, channel encoders, channel decoders, and selected semantic indicators are used for illustration purposes only. The present disclosure is not limited thereto, and appropriate semantic encoders, semantic decoders, channel encoders, channel decoders, and semantic indicators may be selected according to actual semantic communication scenarios.

[0067] Semantic communication in related technologies, whether it is the encoding and decoding method or the measurement indicators (such as the selected semantic indicators), are highly correlated with the scene and form, and there is no unified and universal architecture and method. That is, the encoding and decoding specifications and evaluation indicators of semantic communication in related technologies are highly correlated with the communication scene and transmission content, and it is difficult to obtain a unified system architecture and evaluation indicators. The embodiments of the present disclosure provide a relatively universal system architecture design, which enables semantic communication to have a wider range of applications.

[0068] In addition, the semantic communication measurement indicators in the related technologies are mostly based on the similarity of the data itself, rather than the semantic information that people can truly understand. Therefore, the so-called semantic communication indicators (i.e., semantic indicators) still follow the relevant indicators of multimedia processing, which can barely be regarded as "semantic" measurement indicators rather than "semantic communication" measurement indicators. Therefore, it is difficult to directly link it with communication, and therefore difficult to use it to measure the quality of communication. The disclosed embodiment links the evaluation indicators of semantic communication with the grammatical communication indicators through semantic tolerance, which helps to achieve a smooth evolution from grammatical communication to semantic communication.

[0069] The disclosed embodiment proposes key indicator 1: semantic tolerance T, which is used to characterize the minimum accuracy / minimum correctness of the semantic indicator of the recovered data that is acceptable in the current scenario (i.e., the target semantic communication scenario). In specific scenarios, the specific content of the semantic indicator may change. For example, when implementing semantic communication of text, this semantic indicator may be WER, BLEU, Sentence Similarity, etc.; when implementing semantic communication of speech, this semantic indicator may be SDR and PESQ, etc.; when implementing semantic communication of pictures / images / videos, this indicator may be PSNR and MS-SSIM, etc. In the disclosed embodiment, the numerical value of the semantic indicator of interest is called semantic correctness / semantic accuracy / semantic correctness.

[0070] The above-mentioned recovered data may also be referred to as recovered data, and the recovered data may be any data recovered by the receiver after channel decoding, semantic decoding, etc. based on the received encoded semantic information. In some embodiments, when the transmitter sends the original source transmission block after semantic coding and channel coding, but does not include the original semantic additional block, the recovered data includes the recovered source transmission block. In other embodiments, when the transmitter sends the original source transmission block and the original semantic additional block after semantic coding and channel coding, the recovered data includes the recovered source transmission block and the recovered semantic additional block. The semantic tolerance may be for the recovered source transmission block and / or the recovered semantic additional block.

[0071] Semantic indicators such as WER, BLEU, Sentence Similarity, SDR, PESQ, PSNR, MS-SSIM, etc. reflect only the similarity of the data itself. The disclosed embodiments define new indicators such as semantic tolerance to reflect the minimum accuracy / minimum correctness that is actually acceptable in communication engineering design, so as to guide the engineering design of communication systems.

[0072] The disclosed embodiment proposes a general semantic communication system architecture from the perspective of the overall communication system. On the basis of this communication system architecture, different communication targets can be defined for different task objectives, thereby realizing a flexible and efficient semantic communication system and an efficient semantic communication method. The different task objectives and communication targets here specifically refer to defining different semantic tolerances in different semantic communication scenarios. In the following embodiments, different semantic additional blocks may also be included.

[0073] In an exemplary embodiment, the data to be transmitted includes an original source transmission block and an original semantic additional block; and the restored data includes a restored source transmission block and a restored semantic additional block.

[0074] In the embodiments of the present disclosure, the source transmission block may also be referred to as the source or source data, which is data / information that is sent from the transmitter to the receiver according to actual business needs. Here, in order to distinguish the original source transmission block from the source data after encoding and the source transmission block recovered by the receiver, it is called the original source transmission block. The semantic additional block refers to the data additionally added on the basis of the source transmission block, and its function is to assist in determining whether the source transmission block needs to be retransmitted (referred to as "retransmission" for short). In order to distinguish the original semantic additional block from the semantic additional block after encoding and the semantic additional block recovered by the receiver, it is called the original semantic additional block, and the semantic additional block recovered by the receiver is called the recovered semantic additional block.

[0075] The transmitter is also used to call the corresponding semantic encoder to perform semantic encoding on the original source transmission block and the original semantic additional block in the target semantic communication scenario, respectively, to obtain the semantically encoded source transmission block and semantic check bits; and to call the channel encoder to perform channel encoding on the semantically encoded source transmission block and semantic check bits, to obtain encoded semantic information. In some embodiments, the transmitter can also modulate the encoded semantic information, and send the modulated semantic information to the receiver through the channel.

[0076] The receiver is also used to receive the encoded semantic information, call the corresponding channel decoder and semantic decoder to perform channel decoding and semantic decoding on the encoded semantic information, obtain the restored source transmission block and the restored semantic additional block, obtain the semantic accuracy based on the restored semantic additional block and the original semantic additional block, and if the semantic accuracy is lower than the semantic tolerance of the target semantic communication scenario, generate and send retransmission indication information to instruct the transmitter to retransmit the original source transmission block.

[0077] In an embodiment of the present disclosure, the restored semantic additional block and the original semantic additional block can be compared, for example, the degree of difference between the two is compared (the specific comparison algorithm used is not limited by the present disclosure). The higher the degree of difference between the two, the lower the semantic correctness of the data to be transmitted; the lower the degree of difference between the two, the higher the semantic correctness of the data to be transmitted. When the semantic correctness is lower than the semantic tolerance of the target semantic communication scenario, the receiver can send a retransmission indication message to the transmitter to instruct the transmitter to retransmit the original source transmission block. When retransmitting the original source transmission block, the transmitter can reselect an original semantic additional block different from the one selected last time, or continue to use the original semantic additional block selected last time. In some embodiments, the transmitter and the receiver can repeat the above process for the same original source transmission block until its semantic correctness is higher than the semantic tolerance of the target semantic communication scenario, and then the transmitter transmits the next original source transmission block. In other embodiments, a maximum number of retransmissions may be set. When the maximum number of retransmissions is reached, if the semantic accuracy of the same original source transmission block is still lower than the semantic tolerance of the target semantic communication scenario, the next original source transmission block is started to be transmitted.

[0078] In an exemplary embodiment, a semantic additional block and a semantic check bit are introduced in the embodiment of the present disclosure. After the semantic additional block (here, the original semantic additional block) is semantically encoded, a semantic check bit can be obtained, and the semantic check bit can be restored by semantic decoding to obtain the semantic additional block (i.e., the restored semantic additional block). The restored semantic additional block is compared with the original additional block (i.e., the original semantic additional block), and is used as a basis for whether the semantic correctness of this transmission meets the standard (i.e., whether the semantic correctness is lower than the semantic tolerance of the target semantic communication scenario), so as to verify whether this transmission is successful.

[0079] In the process of communication, when faced with different complex signal-noise situations, transmission errors are inevitable. When encountering transmission errors, the errors encountered in the transmission process can be avoided by continuous retransmission. The semantic communication system provided by the disclosed embodiment superimposes the original semantic additional block on the basis of the original source transmission block, so that when determining whether to retransmit, it is not necessary to combine with the previously transmitted information, thereby saving channel resources and improving the accuracy of transmission. That is, it solves the problem of information retransmission in semantic communication.

[0080] Figure 2 A schematic diagram of a network system architecture of a semantic communication system in an embodiment of the present disclosure is shown.

[0081] like Figure 2 As shown, it is assumed here that the transmitter 11 performs joint encoding of semantic coding and channel coding, and correspondingly, the receiver 12 performs joint decoding of channel decoding and semantic decoding. The transmitter 11 superimposes a semantic additional block (i.e., the original semantic additional block) b on the source (i.e., the original source transmission block) s. The transmitter 11 performs semantic encoding on the source s and the semantic additional block b respectively to obtain s'+b', where s' represents the source transmission block after semantic encoding, and b' represents the semantic check bit; and then performs channel encoding on s'+b' to obtain the encoded semantic information x. The transmitter 11 transmits x. After passing through the channel, x becomes y (which can be the encoded semantic information x or the encoded semantic information x superimposed with noise).

[0082] The receiver 12 receives y and performs channel decoding on y to obtain and Respectively represent the source transmission block and semantic check bit after channel decoding; Perform semantic decoding to obtain the recovered source transmission blocks and Indicates the recovery semantics additional block, Send to the destination / receiver.

[0083] Will restore semantic append blocks The semantic additional block b is verified to obtain the semantic correctness. If the semantic correctness / semantic correctness of the restored semantic additional block fails to meet the semantic tolerance T, the source data s is retransmitted.

[0084] The disclosed embodiments provide a universal target-oriented semantic communication system, where the target refers to the semantic correctness, semantic tolerance, and semantic additional blocks set in a specific semantic communication scenario. For example, for different semantic communication scenarios such as text, speech, or image, the respective semantic codecs (including semantic encoders and semantic decoders) are trained respectively, and the respective semantic correctness, semantic tolerance, and semantic additional blocks are defined. The architecture of the semantic communication system is universal.

[0085] In an exemplary embodiment, the data type of the original semantic additional block is the same as that of the original source transmission block. For example, if the original source transmission block is text, the superimposed original semantic additional block is also text. For another example, if the original source transmission block is speech, the superimposed original semantic additional block is also speech. For another example, if the original source transmission block is an image, the superimposed original semantic additional block is also an image.

[0086] In an exemplary embodiment, the data volume of the original semantic additional block is smaller than the data volume of the original source transmission block.

[0087] In an exemplary embodiment, the original semantic additional block is stored in the transmitter and the receiver respectively, that is, the original semantic additional block is known to both the transmitter and the receiver, and the transmitter and the receiver include a transmitter and a receiver.

[0088] The semantic communication system provided by the embodiment of the present disclosure can superimpose a smaller semantic additional block on the data to be transmitted (e.g., the original source transmission block). The data type of the semantic additional block is the same as the data type of the data to be transmitted (e.g., the original source transmission block).

[0089] In some embodiments, the data block with the best performance can be selected as a reference, i.e., as an alternative semantic additional block, during model training. The transmitter can select from the alternative semantic additional blocks as the original semantic additional block. The data block with the best performance here can be, for example, the original training samples whose sample semantic accuracy obtained by the transmitter and the receiver during the training phase is greater than the semantic tolerance of the target semantic communication scenario, and whose sample semantic accuracy is ranked in descending order in the first predetermined number or the first predetermined percentage.

[0090] In an exemplary embodiment, the transmitter is further configured to select one or more of the candidate semantic additional blocks as the original semantic additional blocks.

[0091] In an exemplary embodiment, in one transmission, the number of semantic additional blocks may be one or more.

[0092] In an exemplary embodiment, when the transmitter selects multiple alternative semantic additional blocks as the original semantic additional blocks, the receiver is also used to obtain multiple restored semantic additional blocks, obtain multiple semantic correct rates based on the multiple restored semantic additional blocks and the multiple original semantic additional blocks, obtain an average semantic correct rate based on the multiple semantic correct rates, and if the average semantic correct rate is lower than the semantic tolerance of the target semantic communication scenario, generate and send retransmission indication information to instruct the transmitter to retransmit the original source transmission block.

[0093] When the transmitter selects multiple candidate semantic additional blocks as the original semantic additional blocks, the average semantic correctness (i.e., average semantic correctness) of the multiple original semantic additional blocks can be counted as the basis for whether to retransmit the current transmission. That is, if the average semantic correctness is lower than the semantic tolerance of the target semantic communication scenario, the original source transmission block is retransmitted; otherwise, the original source transmission block is not retransmitted.

[0094] In an exemplary embodiment, according to different services being transmitted, multiple indicators in the semantic communication system provided by the embodiments of the present disclosure can be predefined, including semantic indicators, semantic tolerance, specific values ​​of semantic additional blocks (for example, how many candidate semantic additional blocks are selected by the transmitter as original semantic additional blocks in one transmission, and what is the corresponding semantic tolerance, etc.). The predefinition of these indicators can be achieved through configuration or through standardized definition.

[0095] In an exemplary embodiment, the transmitter is also used to transmit different semantic additional blocks in rotation from the alternative semantic additional blocks. The rotation means that the transmitter selects different alternative semantic additional blocks as the original semantic additional blocks in turn in each transmission, and after all the alternative semantic additional blocks have been selected once, all the alternative semantic additional blocks are traversed again. For example, assuming that there are n alternative semantic additional blocks in total, the first alternative semantic additional block is selected in the first transmission, ..., the nth alternative semantic additional block is selected in the nth transmission; the first alternative semantic additional block is selected in the n+1th transmission, ..., the nth alternative semantic additional block is selected in the 2nth transmission, and the cycle is repeated.

[0096] The goal-oriented universal semantic communication system provided by the embodiments of the present disclosure does not restrict the specific implementation, but defines the relationship between semantic communication performance and communication performance. In the future 6G network, only the semantic index, semantic tolerance, semantic additional block and other indicators need to be defined, which is conducive to standardized application and promotion.

[0097] In an exemplary embodiment, the transmitter is further used to call a corresponding semantic encoder to perform semantic encoding on the original training samples in the target semantic communication scenario, and obtain and send the encoded training samples.

[0098] The receiver is also used to receive the encoded training samples, call the corresponding semantic decoder to perform semantic decoding on the encoded training samples to obtain restored training samples, and obtain the sample semantic accuracy based on the restored training samples and the original training samples.

[0099] The transmitter and the receiver are further configured to select a predetermined number of original training samples whose sample semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario as the candidate semantic additional blocks.

[0100] In the embodiment of the present disclosure, the value of the predetermined number can be set according to actual business needs, and the present disclosure does not limit its specific value.

[0101] In an exemplary embodiment, the transmitter is also used to select an original test semantic additional block from the candidate semantic additional blocks, call corresponding semantic encoders to perform semantic encoding on the original source test block and the original test semantic additional block in the target semantic communication scenario, and obtain and send the semantically encoded source test block and test semantic check bits. In some embodiments, during the training phase, the semantic codec and the channel codec (including the channel encoder and the channel decoder) can be trained independently, and then the independently trained semantic codec and the channel codec are combined into a transmitter and a receiver in the semantic communication system.

[0102] The receiver is also used to receive a semantically encoded source test block and a test semantic check bit, call a corresponding semantic decoder to perform semantic decoding on the semantically encoded source test block and the test semantic check bit, obtain a restored source test block and a restored test semantic additional block, obtain a test additional semantic accuracy rate based on the restored test semantic additional block and the original test semantic additional block, obtain a source semantic accuracy rate based on the original source test block and the restored source test block, and obtain an evaluation index of the communication system based on the test additional semantic accuracy rate, the source semantic accuracy rate and the semantic tolerance of the target semantic communication scenario.

[0103] In an exemplary embodiment, the evaluation index includes at least one of the following:

[0104] The miscommunication rate is used to indicate that the test additional semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario, and the source semantic accuracy is less than the semantic tolerance of the target semantic communication scenario;

[0105] The false alarm rate is used to indicate that the test additional semantic accuracy is less than the semantic tolerance of the target semantic communication scenario, and the source semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario.

[0106] In an exemplary embodiment, the transmitter and the receiver are also used to select a predetermined number of original training samples whose sample semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario as the candidate semantic additional blocks when the false alarm rate and the error transmission rate are lower than a preset threshold; and continue to train the semantic encoder and the semantic decoder when the false alarm rate and the error transmission rate are higher than the preset threshold. In the embodiment of the present disclosure, the value of the preset threshold can be set according to actual business needs, and the present disclosure does not limit this.

[0107] In an exemplary embodiment, the receiver is also used to obtain a test semantic accuracy rate based on the recovered test semantic additional block and the original test semantic additional block, and obtain a nominal semantic block error rate of the semantic communication system based on the test semantic accuracy rate and the semantic tolerance of the target semantic communication scenario; obtain a source semantic accuracy rate based on the original source test block and the recovered source test block, and obtain a true semantic block error rate of the semantic communication system based on the source semantic accuracy rate and the semantic tolerance of the target semantic communication scenario.

[0108] When judging the quality of the semantic communication system provided by the embodiments of the present disclosure, the key indicators / evaluation indicators include at least one of the following:

[0109] 1) Nominal semantic block error rate: The block error rate of the semantic additional block based on the semantic accuracy and semantic tolerance. That is, if the semantic accuracy obtained after comparing the restored semantic additional block with the original semantic additional block (also called the nominal semantic accuracy, such as the BLEU value after comparing the restored semantic additional block with the original semantic additional block) is greater than the semantic tolerance in the target semantic communication scenario, then the original source transmission block is considered to be correctly transmitted, otherwise it is an error. The nominal semantic block error rate is obtained by counting the ratio of the number of original source transmission blocks with transmission errors to the total number of original source transmission blocks transmitted.

[0110] 2) True semantic block error rate: The block error rate based on the semantic accuracy and semantic tolerance. That is, if the semantic accuracy obtained after comparing the restored source transmission block with the original source transmission block (such as the BLEU value after comparing the restored source transmission block with the original source transmission block) is greater than the semantic tolerance in the target semantic communication scenario, then the original source transmission block is considered to be correctly transmitted, otherwise it is an error. The true semantic block error rate is obtained by counting the ratio of the number of original source transmission blocks with transmission errors to the total number of original source transmission blocks transmitted.

[0111] If the true semantic block error rate is greater than the semantic tolerance, then when statistically analyzing the communication performance, the bit error rate of the original source transmission block can be regarded as 0. That is, if the semantic correctness of a source transmission block (here, it refers to the semantic correctness obtained after comparing the restored source transmission block with the original source transmission block, which can also be called the true semantic correctness) is greater than the semantic tolerance in the target semantic communication scenario, then it is considered that the source transmission block can be recovered normally, and therefore, the bit error rate of the source transmission block is considered to be 0.

[0112] 3) Error rate: The semantic accuracy of the semantic additional block meets the standard, but the semantic accuracy of the actual transmission service performance does not meet the standard (for example, the BLEU of the data transmitted by the source is lower than the semantic tolerance). That is, the semantic accuracy obtained after comparing the restored semantic additional block with the original semantic additional block is greater than the semantic tolerance in the target semantic communication scenario, but the semantic accuracy obtained after comparing the restored source transmission block with the original source transmission block is lower than the semantic tolerance in the target semantic communication scenario. This indicator measures the degree to which the real semantic block error rate of the communication system is falsely high.

[0113] 4) False alarm rate: The semantic accuracy of the semantic additional block does not meet the standard, but the semantic accuracy of the actual transmission service performance meets the standard. That is, the semantic accuracy obtained after comparing the restored semantic additional block with the original semantic additional block is lower than the semantic tolerance in the target semantic communication scenario, but the semantic accuracy obtained after comparing the restored source transmission block with the original source transmission block is higher than the semantic tolerance in the target semantic communication scenario. This indicator measures the degree to which the real semantic block error rate of the communication system is falsely low.

[0114] Since the block error rate and false alarm rate are not zero, the nominal semantic block error rate is not equal to the true semantic block error rate.

[0115] In an exemplary embodiment, the transmitter and the receiver are further used to adjust the semantic encoder and the semantic decoder, or adjust the original semantic additional block, respectively, based on a comparison between the semantic accuracy obtained by the restored semantic additional block and the original semantic additional block and the semantic tolerance.

[0116] In an exemplary embodiment, the transmitter is further used to call a corresponding semantic encoder and a channel encoder to perform semantic encoding and channel encoding on the original training samples in the target semantic communication scenario, and obtain and send the encoded training samples.

[0117] The receiver is also used to receive the encoded training samples, call the corresponding channel decoder and semantic decoder to perform semantic decoding on the encoded training samples, obtain the restored training samples, and obtain the sample semantic accuracy according to the restored training samples and the original training samples. In some embodiments, the semantic codec and the channel codec can be jointly encoded and jointly decoded, that is, jointly trained, which can also be called end-to-end training.

[0118] The transmitter and the receiver are further configured to select a predetermined number of original training samples whose sample semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario as the candidate semantic additional blocks.

[0119] In an exemplary embodiment, the transmitter is also used to select an original test semantic additional block from the alternative semantic additional blocks, call the corresponding semantic encoder and channel encoder to perform semantic encoding and channel encoding on the original source test block and the original test semantic additional block in the target semantic communication scenario, and obtain the encoded source test block and test semantic check bits.

[0120] The receiver is also used to receive the encoded source test block and test semantic check bits, call the corresponding channel decoder and semantic decoder to perform channel decoding and semantic decoding on the encoded source test block and test semantic check bits, obtain the restored source test block and the restored test semantic additional block, obtain the test additional semantic accuracy rate according to the restored test semantic additional block and the original test semantic additional block, obtain the source semantic accuracy rate according to the original source test block and the restored source test block, and obtain the evaluation index of the communication system according to the test additional semantic accuracy rate, the source semantic accuracy rate and the semantic tolerance of the target semantic communication scenario.

[0121] In an exemplary embodiment, the evaluation index includes at least one of the following:

[0122] The miscommunication rate is used to indicate that the test additional semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario, and the source semantic accuracy is less than the semantic tolerance of the target semantic communication scenario;

[0123] The false alarm rate is used to indicate that the test additional semantic accuracy is less than the semantic tolerance of the target semantic communication scenario, and the source semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario.

[0124] In an exemplary embodiment, the transmitter and the receiver are also used to select a predetermined number of original training samples whose sample semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario as the alternative semantic additional blocks when the false alarm rate and the error transmission rate are lower than a preset threshold; and continue to train the semantic encoder, the semantic decoder, the channel encoder and the channel decoder when the false alarm rate and the error transmission rate are higher than a preset threshold.

[0125] In an exemplary embodiment, the receiver is also used to obtain a test semantic accuracy rate based on the recovered test semantic additional block and the original test semantic additional block, and obtain a nominal semantic block error rate of the semantic communication system based on the test semantic accuracy rate and the semantic tolerance of the target semantic communication scenario; obtain a source semantic accuracy rate based on the original source test block and the recovered source test block, and obtain a true semantic block error rate of the semantic communication system based on the source semantic accuracy rate and the semantic tolerance of the target semantic communication scenario.

[0126] In an exemplary embodiment, the transmitter and the receiver are also used to adjust the semantic encoder, the semantic decoder, the channel encoder and the channel decoder, or adjust the original semantic additional block, respectively, based on a comparison of the semantic accuracy obtained by the restored semantic additional block and the original semantic additional block and the semantic tolerance.

[0127] The target-oriented general semantic communication system provided by the embodiment of the present disclosure defines the measurement indicators of the communication system and the HARQ (Hybrid Automatic Repeat Request) method, does not restrict the specific implementation, but defines the relationship between semantic communication performance and communication performance, realizes the application of semantic communication in traditional communication systems under any scenario and goal, and realizes that when the channel resources are the same, semantic coding and decoding can achieve better performance than traditional communication (syntactic level). Among them, semantic communication performance refers to semantic indicators such as BLEU that reflect semantic similarity, and communication performance refers to indicators such as semantic tolerance that reflect the performance of the communication system.

[0128] The following is an example of some possible application scenarios to illustrate the solutions provided by the embodiments of the present disclosure.

[0129] In some embodiments, the semantic transmission of text information is achieved through the solution provided by the embodiments of the present disclosure.

[0130] Application scenario: It is known that in a certain communication scenario, the source content of the transmission is text information. In this case, semantic coding + channel coding is used for communication.

[0131] Implementation method: LSTM is used as the semantic encoder, and LDPC (Low-density Parity-check) code is used for channel coding. The BLEU value is used to measure the semantic correctness, and the semantic tolerance is defined as 90%. Assume that the size of each source packet (i.e., the original source transmission block) is 1000 bits.

[0132] The first is the system training phase: During the semantic codec training process, since the channel codec and the semantic codec are performed separately, it can be assumed that the channel has no noise. After the semantic codec is trained, the semantic codec is used to screen out 10 (assuming the predetermined number = 10) 50-bit small-capacity samples (i.e., original training samples) with the highest semantic accuracy (must be greater than the semantic tolerance, i.e., 90%) as candidate semantic additional blocks.

[0133] For example, suppose 100 original training samples are input into a trained semantic codec to obtain restored training samples, and the BLEU between these 100 original training samples and the corresponding restored training samples is calculated. The 10 original training samples with the largest BLEU and BLEU greater than 90% are selected as candidate semantic additional blocks.

[0134] In some embodiments, it is ensured that the false alarm rate and the error rate of the system are below 1%. That is, after the training is completed, some test data blocks are used as data blocks sent by the information source (i.e., the original information source test blocks), and the above-mentioned candidate semantic additional blocks are used (i.e., one or more are selected as the original test semantic additional blocks) to simulate the semantic communication process, and the false alarm rate and the error rate of the semantic communication system are calculated. If it is not less than 1% (i.e., it is assumed that the preset threshold value = 1%, which is only used for illustration), the semantic codec is continued to be trained until the false alarm rate and the error rate of the system are below 1%, and then the final candidate semantic additional blocks are screened out.

[0135] The second is the system implementation stage: the text information is transmitted according to the trained semantic codec (plus the trained channel codec). In actual transmission, different semantic additional blocks are sent in turn. That is, each time a data block sent by the source is transmitted, one of the multiple candidate semantic additional blocks is selected in turn as the semantic additional block.

[0136] The third is the system adjustment stage: according to the recovery of the semantic additional block, the semantic codec is adjusted and synchronized at any time. For example, adjustments can be made from two aspects: one is to adjust the semantic additional block, and the other is to adjust the semantic codec itself, such as using idle time to retrain the semantic codec and adjust the model parameters. If the reception performance of different semantic additional blocks is significantly different, you can consider adding multiple semantic additional blocks at the same time and reducing the system bit rate (adding multiple semantic additional blocks will increase the redundant bits, so the actual transmitted source data will be reduced, thereby reducing the system bit rate) to better judge the current semantic accuracy.

[0137] In some embodiments, semantic transmission of image information is achieved through the solution provided by the embodiments of the present disclosure.

[0138] Application scenario: It is known that in a certain communication scenario, the transmitted source content is image information. In this case, semantic coding + channel joint coding is used for communication.

[0139] Implementation method: CNN is used as the joint encoder for semantic coding and channel coding. The value of MS-SSIM is used to measure the semantic correctness, and the semantic tolerance is defined as 90%. Assume that the size of each source packet is 3000 bits.

[0140] The first is the system training phase: during the joint codec training process, training is performed in combination with the channel state (end-to-end training). After the joint codec is trained, this codec (including both the semantic codec and the channel codec) is used to select 10 100-bit small-capacity samples with the highest semantic accuracy (must be greater than the semantic tolerance, i.e. 90%) as candidate semantic additional blocks. At the same time, the false alarm rate and misrepresentation rate of the system are ensured to be below 1%.

[0141] Exemplarily, the executing entity of the embodiments of the present disclosure may be a training device for a semantic communication system or a semantic communication model (referred to as a training device for short). The training device may be a server, a terminal device, a processor, a chip, etc., which are not listed one by one here.

[0142] For example, if the training device is a server, the training device can be an independent server or a server cluster; it can be a cloud server (such as a server deployed on a cloud platform) or a local server, which is not limited in this embodiment.

[0143] It can be understood that model training is generally carried out on the basis of a preset network model. For example, the training device collects sample data to train the parameters of the preset network model based on the sample data until a target network model that meets the preset requirements is obtained, wherein the preset requirements can be understood as the number of iterative optimization times of the training reaching the preset number requirements, or can be understood as the result of the iterative optimization of the training representing that the difference between the result of the training and the pre-labeled true value meets the preset difference requirements (such as a small difference).

[0144] Accordingly, in this embodiment, the training device can train the basic network model based on the original training samples until a semantic communication model that meets the preset requirements is obtained. The basic network model can include a semantic encoder, which can be used for encoding processing.

[0145] Therefore, this step can be understood as that a basic network model is deployed in the training device, the basic network model includes a semantic encoder, and the training device can encode the original training sample based on the semantic encoder to obtain a data packet represented by the original training sample from a semantic dimension, that is, a sample semantic data packet. This embodiment does not limit the specific implementation of the semantic encoder encoding process.

[0146] Combined with the above analysis, it can be seen that the training method of the semantic communication model of the embodiment of the present disclosure can be applied to different application scenarios, such as text, image, voice, video, and their combination. For different application scenarios, the original training samples can be different objects. For example, for the application scenario of text, the original training sample can be the original sample text; for the application scenario of image, the original training sample can be the original sample image, and so on, which are not listed here one by one. The sample semantic data packet is transmitted in the simulated environmental information (such as a channel), and the sample semantic data packet may change. The changed sample semantic data packet can be referred to as a processed semantic data packet. The basic network model can also include a channel, the output of the semantic encoder is connected to the input of the channel, and accordingly, the sample semantic data packet output by the semantic encoder is used as the input of the channel. The sample semantic data packet changes under the action of the data transmission environment information simulated by the channel to obtain a processed semantic data packet. The processed semantic data is decoded based on the semantic decoder to obtain a restored training sample. The basic network model can also include a semantic decoder, the output of the channel is connected to the input of the semantic decoder, and accordingly, the input of the semantic decoder is the processed semantic data packet output by the channel, and the semantic decoder decodes the processed semantic data packet to restore the original training sample, thereby obtaining a restored training sample. The basic network model is iteratively optimized according to the difference between the original training samples and the restored training samples to obtain a semantic communication model, wherein the basic network model includes a semantic encoder, a channel, and a semantic decoder, and the semantic communication model includes an iteratively optimized semantic encoder and an iteratively optimized semantic decoder.

[0147] Accordingly, the training device can construct a loss function between the original training sample and the restored training sample to characterize the difference between the original training sample and the restored training sample through the loss function, thereby iteratively optimizing the basic network model based on the loss function to complete the training of the basic network model and obtain the semantic communication model. That is, the semantic communication model includes the semantic encoder and the semantic decoder in the trained basic network model.

[0148] The second is the system implementation stage: while transmitting image information according to the trained joint encoder, different semantic additional blocks are sent in turn.

[0149] The third is the system adjustment stage: according to the recovery of the semantic additional blocks, the semantic codec is adjusted and synchronized at any time. If the reception performance of different semantic additional blocks is significantly different, it is possible to consider adding multiple semantic additional blocks at the same time and reducing the system bit rate to better judge the current semantic accuracy.

[0150] In an exemplary embodiment, the transmitter is also used to extract semantic information of the data to be transmitted based on a semantic knowledge base of the transmitting end, call corresponding semantic encoders and channel encoders to perform semantic encoding and channel encoding on the semantic information, and obtain and send the encoded semantic information.

[0151] The receiver is also used to restore the semantic information after channel decoding and semantic decoding by using the receiver semantic knowledge base to obtain the restored data.

[0152] The transmitter and the receiver are also used to synchronize the semantic knowledge base of the transmitter and the semantic knowledge base of the receiver to form a shared knowledge base.

[0153] Figure 3 FIG. 2 is a schematic diagram showing the architecture of a semantic communication system in an embodiment of the present disclosure. Figure 3 As shown, the signal source s is sent to the receiver 12 through the channel after semantic coding and channel coding (joint coding) and modulation by the transmitter 11. Noise may be introduced during transmission in the channel. After demodulation, channel decoding and semantic decoding (correspondingly, joint decoding) by the receiver 12, it is sent to the receiving end 33.

[0154] Continue to refer Figure 3 , the transmitter 11 (i.e., the transmitting end) can construct a transmitting end local knowledge base (i.e., the transmitting end semantic knowledge base) 31, and the receiver 12 (i.e., the receiving end) can construct a receiving end local knowledge base (i.e., the receiving end semantic knowledge base) 32. The transmitting end constructs its own transmitting end local knowledge base, and the receiving end constructs its own receiving end local knowledge base. The transmitting and receiving ends synchronize the knowledge bases, so the contents of the transmitting end local knowledge base and the receiving end local knowledge base are the same, so it is called a shared knowledge base.

[0155] Specifically, before channel coding the source, semantic coding is required based on the semantic information to extract the key semantic information; accordingly, after channel decoding is performed at the receiving end, semantic decoding is also required to recover the original data (i.e., to obtain the recovered data). In some embodiments, semantic coding and channel coding can be performed jointly to improve system efficiency and performance.

[0156] For example, in a semantic encoder, the key semantic information in the original data (such as the original source transmission block) is first extracted using the local knowledge base of the transmitter, and then the extracted key semantic information is encoded to obtain a coding sequence containing the key semantic information. In a semantic decoder, the received key semantic information is first decoded, and then the decoded key semantic information is restored using the local knowledge base of the receiver to restore the original data, that is, to obtain the restored source transmission block.

[0157] In the embodiment of the present disclosure, the semantic codec and the channel codec are jointly performed, which means that during training, the semantic codec and the channel codec are trained simultaneously (end-to-end), and the model parameters of the semantic codec and the channel codec are optimized synchronously. Correspondingly, the semantic codec and the channel codec are trained independently and in parallel, and after the model parameters of the semantic codec and the channel codec are independently adjusted, the trained semantic codec and the channel codec are combined into a Figure 2 or Figure 3 The system shown.

[0158] In the disclosed embodiment, the sending and receiving ends share the same semantic knowledge base and keep them synchronized (i.e., shared knowledge base). The semantic knowledge base is used to store a lot of common non-critical information and is used to extract / restore the complete communication data content, i.e., extract the critical semantic information.

[0159] In the disclosed embodiment, the content of key information and non-key information depends on the corresponding semantic communication scenario. Non-key information refers to information stored in the shared knowledge base at both ends of the transmitter and receiver, that is, information that both ends of the transmitter and receiver already know. Key information refers to information that the receiver does not know yet. The receiver can use the key information to restore the complete original source content that the transmitter wants to send from the non-key information stored in the shared knowledge base. For example, if a certain article is stored in the shared knowledge base, the transmitter only needs to send the title of the article to the receiver, and the receiver can obtain the complete content of the article from the shared knowledge base based on the title.

[0160] In semantic communication, it is a complex but efficient process for the semantic encoder to extract key semantic information using a shared knowledge base. This process involves a combination of multiple steps and techniques to ensure that the most important semantic content can be accurately captured from the input data. The following is a detailed analysis of the process:

[0161] (1) The role of the shared knowledge base: The shared knowledge base plays an important role in the semantic encoder. It contains a large amount of domain knowledge, common sense, and context-specific information, which is crucial for understanding and interpreting the semantics of the input data. By referring to the shared knowledge base, the semantic encoder can more accurately identify the key concepts in the input data and the relationships between them.

[0162] (2) Workflow of semantic encoder:

[0163] Input representation: The semantic encoder first needs to convert the input data (such as text, images, etc.) into a representation that the machine can understand. For example, for text data, words or sentences are converted into word vectors or embedded representations.

[0164] Dependency Analysis: Next, the semantic encoder analyzes the dependencies between different elements in the input data, such as words, phrases, etc. This helps in understanding the structure and meaning of a sentence or document.

[0165] Extract key semantic information using knowledge base: Based on the analysis of dependencies, the semantic encoder uses the information in the shared knowledge base to extract key semantic information. This includes identifying key concepts, entities, and the relationships between them in the input data. The information in the shared knowledge base provides the necessary context and reference for this process.

[0166] For example, in text processing, a semantic encoder may use information such as a synonym database and an entity link database in a knowledge base to identify key concepts and entities in the text and understand the semantic relationships between them.

[0167] Encoding: After extracting the key semantic information, the semantic encoder encodes this information into a compact and easy-to-transmit format. This involves mapping the high-dimensional input data into a low-dimensional vector space while retaining the important semantic information.

[0168] Transmission and decoding: The encoded semantic information will be transmitted to the receiving end and decoded and restored by the corresponding decoder. During the decoding process, the receiving end may again use the information in the shared knowledge base to assist in understanding and interpreting the received semantic information.

[0169] In practical applications, semantic encoders may use a variety of technologies to implement the above process, including but not limited to: Deep learning models: such as recurrent neural networks (RNN), long short-term memory networks (LSTM), Transformer, etc. These models perform well in processing sequence data and capturing semantic information. Attention mechanism: used to focus on the key parts of the input data during the encoding process, thereby extracting more important semantic information. Knowledge graph: As a form of knowledge base, the knowledge graph provides rich entity and relationship information, which helps the semantic encoder understand the semantics of the input data more accurately.

[0170] The semantic encoder extracts key semantic information in semantic communication by using information in the shared knowledge base. This process involves multiple steps such as input representation, dependency analysis, key semantic information extraction, encoding, transmission and decoding. By adopting advanced deep learning models and technical means, the semantic encoder can efficiently extract and transmit semantic information, providing strong support for semantic communication.

[0171] The disclosed embodiments propose a relatively general semantic communication system architecture, which does not impose constraints on specific scenarios and businesses, and defines a series of general key technologies applicable to semantic communication, such as semantic correctness and semantic tolerance, so that the solution provided by the disclosed embodiments can be adapted to more application scenarios and promote the widespread application of semantic communication.

[0172] In other embodiments, the concepts of semantic tolerance and additional semantic blocks are proposed to form a HARQ mechanism for semantic communication. Based on the dependence of the communication network on the scene, the encoding and decoding methods and measurement indicators used are highly correlated with the scene, so that the system gain brought by semantic communication is maximized. At the same time, there is no need to borrow previously transmitted data, which has better universality.

[0173] Figure 4 FIG. 1 is a flowchart of a semantic communication method according to an embodiment of the present disclosure. Figure 4 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0174] In S410, semantic coding and channel coding are performed on the data to be transmitted in the target semantic communication scenario to obtain coded semantic information.

[0175] In S420, the encoded semantic information is channel decoded and semantically decoded to obtain recovered data, and the semantic accuracy of the data to be transmitted is obtained based on the recovered data and the data to be transmitted. If the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, retransmission indication information is generated and sent to indicate retransmission of the data to be transmitted.

[0176] The semantic tolerance is used to characterize the minimum correctness of the semantic indicator of the restored data that is acceptable in the target semantic communication scenario; the value of the semantic indicator is the semantic correctness.

[0177] In an exemplary embodiment, the data to be transmitted includes an original source transmission block and an original semantic additional block; and the restored data includes a restored source transmission block and a restored semantic additional block.

[0178] Among them, semantic coding and channel coding are performed on the data to be transmitted in the target semantic communication scenario to obtain the encoded semantic information, including: semantic coding is performed on the original source transmission block and the original semantic additional block in the target semantic communication scenario respectively to obtain the semantically encoded source transmission block and semantic check bits; channel coding is performed on the semantically encoded source transmission block and semantic check bits to obtain the encoded semantic information.

[0179] Among them, the encoded semantic information is channel decoded and semantically decoded to obtain recovered data, and the semantic accuracy of the data to be transmitted is obtained according to the recovered data and the data to be transmitted. If the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, retransmission indication information is generated and sent, including: receiving the encoded semantic information, channel decoding and semantic decoding of the encoded semantic information, obtaining the recovered source transmission block and the recovered semantic additional block; obtaining the semantic accuracy according to the recovered semantic additional block and the original semantic additional block; if the semantic accuracy is lower than the semantic tolerance of the target semantic communication scenario, retransmission indication information is generated and sent to instruct the transmitter to retransmit the original source transmission block.

[0180] Figure 4 For other contents of the embodiment, reference can be made to other embodiments mentioned above and will not be repeated here.

[0181] Figure 5 FIG. 2 is a flow chart showing another semantic communication method in an embodiment of the present disclosure. Figure 5 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0182] In S51, source data is obtained.

[0183] In S52 , a semantic additional block is selected and added.

[0184] In S53, the source data and the semantic additional block are semantically encoded respectively.

[0185] In S54, the semantically coded data (ie, the semantically coded source transmission block and the semantic check bits) are jointly channel coded, and the coded data is sent.

[0186] S54 can be performed simultaneously with S53, that is, joint coding can be performed.

[0187] In S55, the receiving end performs channel decoding and semantic decoding.

[0188] In S56, the semantic additional block is separated (ie, the restored semantic additional block is obtained) to determine its semantic correctness.

[0189] In S57, it is determined whether the semantic tolerance is met, that is, whether the semantic correctness is higher than the semantic tolerance; if so, step S58 is executed; otherwise, the process returns to the above step S52.

[0190] In S58, the result is output, that is, the restored source transmission block is sent to the receiving end or the destination.

[0191] Figure 5 For other contents of the embodiment, reference can be made to other embodiments mentioned above and will not be repeated here.

[0192] Figure 6 FIG. 2 is a flowchart of another semantic communication method in an embodiment of the present disclosure. Figure 6 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0193] In S610 , an original semantic additional block is determined.

[0194] In S620, the corresponding semantic encoders are called to perform semantic encoding on the original source transmission block and the original semantic additional block in the target semantic communication scenario, respectively, to obtain the semantically encoded source transmission block and semantic check bits.

[0195] In S630, a channel encoder is called to perform channel coding on the semantically coded source transmission block and semantic check bits to obtain coded semantic information.

[0196] In S640, the encoded semantic information is sent so as to perform channel decoding and semantic decoding based on the encoded semantic information to obtain a recovered source transmission block and a recovered semantic additional block, and a semantic accuracy is obtained based on the recovered semantic additional block and the original semantic additional block. If the semantic accuracy is lower than the semantic tolerance of the target semantic communication scenario, retransmission indication information is generated and sent to indicate retransmission of the original source transmission block.

[0197] Figure 6 For other contents of the embodiment, reference can be made to other embodiments mentioned above and will not be repeated here.

[0198] Figure 7 FIG. 2 is a flowchart of another semantic communication method according to an embodiment of the present disclosure. Figure 7 As shown, the method provided by the embodiment of the present disclosure may include the following steps.

[0199] In S710, the encoded semantic information is received.

[0200] In S720, channel decoding and semantic decoding are performed on the encoded semantic information to obtain restored data.

[0201] In S730, the semantic correctness of the data to be transmitted is obtained according to the restored data and the data to be transmitted.

[0202] In S740, if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, retransmission indication information is generated and sent to indicate retransmission of the data to be transmitted;

[0203] The semantic tolerance is used to characterize the minimum correctness of the semantic indicator of the restored data that is acceptable in the target semantic communication scenario; the value of the semantic indicator is the semantic correctness.

[0204] Figure 7 For other contents of the embodiment, reference can be made to other embodiments mentioned above and will not be repeated here.

[0205] Figure 8 FIG. 2 shows a structural block diagram of a transmitter in an embodiment of the present disclosure. Figure 8 As shown, the transmitter 80 provided in the embodiment of the present disclosure includes a first processing unit 81 and a first sending unit 82 .

[0206] The first processing unit 81 is used to determine the original semantic additional block.

[0207] The first processing unit 81 is further used to call corresponding semantic encoders to perform semantic encoding on the original source transmission block and the original semantic additional block in the target semantic communication scenario, respectively, to obtain the semantically encoded source transmission block and semantic check bits.

[0208] The first processing unit 81 is further used to call a channel encoder to perform channel coding on the semantically coded source transmission block and semantic check bits to obtain coded semantic information.

[0209] The first sending unit 82 is used to send the encoded semantic information so as to perform channel decoding and semantic decoding based on the encoded semantic information to obtain a restored source transmission block and a restored semantic additional block, and obtain a semantic accuracy rate according to the restored semantic additional block and the original semantic additional block. If the semantic accuracy rate is lower than the semantic tolerance of the target semantic communication scenario, a retransmission indication information is generated and sent to indicate the retransmission of the original source transmission block.

[0210] Figure 8 For other contents of the embodiment, reference can be made to other embodiments mentioned above and will not be repeated here.

[0211] Fig. 9 FIG. 2 shows a structural block diagram of a receiver in an embodiment of the present disclosure. Fig. 9 As shown, the receiver 90 provided by the embodiment of the present disclosure includes a second receiving unit 91, a second processing unit 92 and a second sending unit 93.

[0212] The second receiving unit 91 is used to receive the encoded semantic information.

[0213] The second processing unit 92 is used to perform channel decoding and semantic decoding on the encoded semantic information to obtain restored data.

[0214] The second processing unit 92 is further configured to obtain a semantic accuracy rate of the data to be transmitted according to the restored data and the data to be transmitted.

[0215] The second processing unit 92 is further configured to generate retransmission indication information to instruct retransmission of the data to be transmitted if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario.

[0216] The second sending unit 93 is used to send the retransmission indication information.

[0217] The semantic tolerance is used to characterize the minimum correctness of the semantic indicator of the restored data that is acceptable in the target semantic communication scenario; the value of the semantic indicator is the semantic correctness.

[0218] Fig. 9 For other contents of the embodiment, reference can be made to other embodiments mentioned above and will not be repeated here.

[0219] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0220] Refer to the following Fig.10 The communication device 800 according to this embodiment of the present disclosure is described. Fig.10 The communication device 800 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0221] like Fig.10 As shown, the communication device 800 is in the form of a general-purpose computing device. The components of the communication device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

[0222] The storage unit 820 stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 executes the steps described in the above “Exemplary Method” section of this specification according to various exemplary embodiments of the present disclosure.

[0223] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0224] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0225] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0226] The communication device 800 may also communicate with one or more external devices 840 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the communication device 800, and / or may communicate with any device that enables the communication device 800 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 850. Furthermore, the communication device 800 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the communication device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the communication device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0227] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0228] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer program product, which includes: a computer program, which implements the above semantic communication method when executed by a processor.

[0229] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. The computer-readable storage medium stores a program product capable of implementing the above-mentioned method of the present disclosure. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal / network device, the program code is used to cause the terminal / network device to execute the steps described in the above “Exemplary Method” section of this specification according to various exemplary implementations of the present disclosure.

[0230] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0231] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0232] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

Claims

1. A semantic communication system, characterized in that: include: A transmitter, used to call a corresponding semantic encoder and a channel encoder to perform semantic encoding and channel encoding on the data to be transmitted in the target semantic communication scenario, and obtain and send the encoded semantic information; A receiver is used to receive the encoded semantic information, call the corresponding channel decoder and semantic decoder to perform channel decoding and semantic decoding on the encoded semantic information to obtain recovered data, obtain the semantic accuracy of the data to be transmitted according to the recovered data and the data to be transmitted, and if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, generate and send retransmission indication information to instruct the transmitter to retransmit the data to be transmitted; The semantic tolerance is used to characterize the minimum correctness of the semantic indicator of the restored data that is acceptable in the target semantic communication scenario; the value of the semantic indicator is the semantic correctness.

2. The system according to claim 1, characterized in that The data to be transmitted includes an original source transmission block and an original semantic additional block; the restored data includes a restored source transmission block and a restored semantic additional block; The transmitter is also used to call the corresponding semantic encoder to perform semantic encoding on the original source transmission block and the original semantic additional block in the target semantic communication scenario, respectively, to obtain the semantically encoded source transmission block and semantic check bits; call the channel encoder to perform channel encoding on the semantically encoded source transmission block and semantic check bits, to obtain encoded semantic information; The receiver is also used to receive the encoded semantic information, call the corresponding channel decoder and semantic decoder to perform channel decoding and semantic decoding on the encoded semantic information, obtain the restored source transmission block and the restored semantic additional block, obtain the semantic accuracy based on the restored semantic additional block and the original semantic additional block, and if the semantic accuracy is lower than the semantic tolerance of the target semantic communication scenario, generate and send retransmission indication information to instruct the transmitter to retransmit the original source transmission block.

3. The system according to claim 2, characterized in that The data type of the original semantic additional block is the same as that of the original source transmission block.

4. The system according to claim 2, characterized in that The data volume of the original semantic additional block is smaller than the data volume of the original source transmission block.

5. The system according to claim 2, characterized in that The original semantic additional blocks are stored in the transmitter and the receiver respectively.

6. The system according to claim 2, characterized in that The transmitter is further configured to select one or more of the candidate semantic additional blocks as the original semantic additional blocks.

7. The system according to claim 6, characterized in that When the transmitter selects multiple alternative semantic additional blocks as the original semantic additional blocks, the receiver is also used to obtain multiple restored semantic additional blocks, obtain multiple semantic correct rates based on the multiple restored semantic additional blocks and the multiple original semantic additional blocks, and obtain an average semantic correct rate based on the multiple semantic correct rates. If the average semantic correct rate is lower than the semantic tolerance of the target semantic communication scenario, retransmission indication information is generated and sent to instruct the transmitter to retransmit the original source transmission block.

8. The system according to claim 2, characterized in that The transmitter is further configured to transmit different semantic additional blocks in turn from candidate semantic additional blocks.

9. The system according to any one of claims 6 to 8, characterized in that The transmitter is also used to call a corresponding semantic encoder to perform semantic encoding on the original training samples in the target semantic communication scenario, and obtain and send the encoded training samples; The receiver is also used to receive the encoded training sample, call the corresponding semantic decoder to perform semantic decoding on the encoded training sample to obtain a restored training sample, and obtain the sample semantic accuracy according to the restored training sample and the original training sample; The transmitter and the receiver are further configured to select a predetermined number of original training samples whose sample semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario as the candidate semantic additional blocks.

10. The system according to claim 9, characterized in that The transmitter is further used to select an original test semantic additional block from the candidate semantic additional blocks, call corresponding semantic encoders to perform semantic encoding on the original source test block and the original test semantic additional block in the target semantic communication scenario, respectively, and obtain and send the semantically encoded source test block and test semantic check bits; The receiver is also used to receive a semantically encoded source test block and a test semantic check bit, call a corresponding semantic decoder to perform semantic decoding on the semantically encoded source test block and the test semantic check bit, obtain a restored source test block and a restored test semantic additional block, obtain a test additional semantic accuracy rate based on the restored test semantic additional block and the original test semantic additional block, obtain a source semantic accuracy rate based on the original source test block and the restored source test block, and obtain an evaluation index of the communication system based on the test additional semantic accuracy rate, the source semantic accuracy rate and the semantic tolerance of the target semantic communication scenario.

11. The system according to claim 10, characterized in that The evaluation indicators include at least one of the following: The miscommunication rate is used to indicate that the test additional semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario, and the source semantic accuracy is less than the semantic tolerance of the target semantic communication scenario; The false alarm rate is used to indicate that the test additional semantic accuracy is less than the semantic tolerance of the target semantic communication scenario, and the source semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario.

12. The system according to claim 11, characterized in that The transmitter and the receiver are also used to select a predetermined number of original training samples whose sample semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario as the alternative semantic additional blocks when the false alarm rate and the error transmission rate are lower than a preset threshold; and continue to train the semantic encoder and the semantic decoder when the false alarm rate and the error transmission rate are higher than the preset threshold.

13. The system according to claim 10, characterized in that The receiver is also used to obtain a test semantic accuracy rate based on the recovered test semantic additional block and the original test semantic additional block, and obtain a nominal semantic block error rate of the semantic communication system based on the test semantic accuracy rate and the semantic tolerance of the target semantic communication scenario; obtain a source semantic accuracy rate based on the original source test block and the recovered source test block, and obtain a true semantic block error rate of the semantic communication system based on the source semantic accuracy rate and the semantic tolerance of the target semantic communication scenario.

14. The system according to any one of claims 6 to 8, characterized in that The transmitter is also used to call the corresponding semantic encoder and channel encoder to perform semantic encoding and channel encoding on the original training samples in the target semantic communication scenario, and obtain and send the encoded training samples; The receiver is also used to receive the encoded training samples, call the corresponding channel decoder and semantic decoder to perform semantic decoding on the encoded training samples to obtain restored training samples, and obtain the sample semantic accuracy according to the restored training samples and the original training samples; The transmitter and the receiver are further configured to select a predetermined number of original training samples whose sample semantic accuracy is greater than the semantic tolerance of the target semantic communication scenario as the candidate semantic additional blocks.

15. The system according to claim 2, characterized in that The transmitter and the receiver are further used to adjust the semantic encoder and the semantic decoder, or adjust the original semantic additional block, respectively, according to the comparison between the semantic accuracy and the semantic tolerance obtained by the restored semantic additional block and the original semantic additional block.

16. The system according to claim 1, characterized in that The transmitter is also used to extract the semantic information of the data to be transmitted based on the semantic knowledge base of the transmitting end, call the corresponding semantic encoder and channel encoder to perform semantic encoding and channel encoding on the semantic information, and obtain and send the encoded semantic information; The receiver is also used to recover the semantic information after channel decoding and semantic decoding using the receiver semantic knowledge base to obtain the recovered data; The transmitter and the receiver are also used to synchronize the semantic knowledge base of the transmitter and the semantic knowledge base of the receiver to form a shared knowledge base.

17. A semantic communication method, characterized in that: include: Perform semantic coding and channel coding on the data to be transmitted in the target semantic communication scenario to obtain coded semantic information; Performing channel decoding and semantic decoding on the encoded semantic information to obtain recovered data, obtaining the semantic accuracy of the data to be transmitted according to the recovered data and the data to be transmitted, and if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, generating and sending retransmission indication information to instruct retransmission of the data to be transmitted; The semantic tolerance is used to characterize the minimum correctness of the semantic indicator of the restored data that is acceptable in the target semantic communication scenario; the value of the semantic indicator is the semantic correctness.

18. The method according to claim 17, characterized in that The data to be transmitted includes an original source transmission block and an original semantic additional block; the restored data includes a restored source transmission block and a restored semantic additional block; Among them, semantic coding and channel coding are performed on the data to be transmitted in the target semantic communication scenario to obtain coded semantic information, including: Semantically encoding the original source transmission block and the original semantic additional block in the target semantic communication scenario respectively to obtain a semantically encoded source transmission block and a semantic check bit; Channel coding is performed on the semantically coded source transmission block and the semantic check bit to obtain coded semantic information; The method includes performing channel decoding and semantic decoding on the encoded semantic information to obtain recovered data, obtaining the semantic accuracy of the data to be transmitted according to the recovered data and the data to be transmitted, and generating and sending retransmission indication information if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, including: Receiving the encoded semantic information, performing channel decoding and semantic decoding on the encoded semantic information, and obtaining the restored information source transmission block and the restored semantic additional block; Obtaining the semantic accuracy rate according to the restored semantic additional block and the original semantic additional block; If the semantic accuracy is lower than the semantic tolerance of the target semantic communication scenario, retransmission indication information is generated and sent to indicate retransmission of the original source transmission block.

19. A semantic communication method, characterized in that: include: Determine the original semantic additional block; Calling corresponding semantic encoders to perform semantic encoding on the original source transmission block and the original semantic additional block in the target semantic communication scenario, respectively, to obtain the semantically encoded source transmission block and semantic check bits; Calling a channel encoder to perform channel coding on the semantically coded source transmission block and the semantic check bit to obtain coded semantic information; The encoded semantic information is sent so as to perform channel decoding and semantic decoding based on the encoded semantic information to obtain a restored source transmission block and a restored semantic additional block, and a semantic accuracy rate is obtained according to the restored semantic additional block and the original semantic additional block. If the semantic accuracy rate is lower than the semantic tolerance of the target semantic communication scenario, a retransmission indication information is generated and sent to indicate the retransmission of the original source transmission block.

20. A semantic communication method, characterized in that: include: receiving encoded semantic information; Performing channel decoding and semantic decoding on the encoded semantic information to obtain restored data; Obtaining semantic accuracy of the data to be transmitted according to the recovered data and the data to be transmitted; If the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario, generating and sending retransmission indication information to instruct retransmission of the data to be transmitted; The semantic tolerance is used to characterize the minimum correctness of the semantic indicator of the restored data that is acceptable in the target semantic communication scenario; the value of the semantic indicator is the semantic correctness.

21. A transmitter, characterized in that: include: A first processing unit, configured to determine an original semantic additional block; The first processing unit is further used to call corresponding semantic encoders to perform semantic encoding on the original source transmission block and the original semantic additional block in the target semantic communication scenario, respectively, to obtain the semantically encoded source transmission block and semantic check bits; The first processing unit is further used to call a channel encoder to perform channel coding on the semantically coded source transmission block and the semantic check bit to obtain coded semantic information; The first sending unit is used to send the encoded semantic information so as to perform channel decoding and semantic decoding based on the encoded semantic information to obtain a restored source transmission block and a restored semantic additional block, and obtain a semantic accuracy rate according to the restored semantic additional block and the original semantic additional block. If the semantic accuracy rate is lower than the semantic tolerance of the target semantic communication scenario, retransmission indication information is generated and sent to indicate retransmission of the original source transmission block.

22. A receiver, characterized in that: include: A second receiving unit, used for receiving the encoded semantic information; A second processing unit is used to perform channel decoding and semantic decoding on the encoded semantic information to obtain restored data; The second processing unit is further used to obtain the semantic correctness of the data to be transmitted according to the restored data and the data to be transmitted; The second processing unit is further configured to generate retransmission indication information to instruct retransmission of the data to be transmitted if the semantic accuracy of the data to be transmitted is lower than the semantic tolerance of the target semantic communication scenario; A second sending unit, used to send the retransmission indication information; The semantic tolerance is used to characterize the minimum correctness of the semantic indicator of the restored data that is acceptable in the target semantic communication scenario; the value of the semantic indicator is the semantic correctness.

23. A communication device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the method of any one of claims 17 to 18, or the method of claim 19, or the method of claim 20 by executing the executable instructions.

24. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the method described in any one of claims 17 to 18, or implements the method described in claim 19, or implements the method described in claim 20.

25. A computer program product, comprising a computer program, which executes the method according to any one of claims 17 to 18, or the method according to claim 19, or the method according to claim 20 when the computer program is executed.

Images