Semantic communication method and device, network equipment and medium

Abstract

The invention provides a semantic communication method and device, network equipment and a medium, and relates to the technical field of communication. The method comprises the following steps: judging whether an information source channel joint encoder between a sending end and a receiving end has corresponding semantic coding processing capability or not under the condition of determining that information to be transmitted needs to be transmitted through a semantic coding channel; and when it is determined that the source-channel joint encoder does not have the semantic encoding processing capability, the to-be-transmitted information is sent to the cloud server, and the cloud server transmits the to-be-transmitted information to the receiving end, so that target communication in different scenes is realized, the communication requirements of different users are met, and the user experience is improved. Customized communication services are provided for diversification of communication scenes and diversification of user communication targets, communication resources are saved, and network working efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a semantic communication method, device, network equipment and medium. Background Art

[0002] Communication systems have reshaped social operations and human lifestyles and are an important engine driving the development of the digital economy. Since the next generation of mobile communications (6G) has the performance requirements of low latency, high reliability, high spectrum efficiency, and high-density large links, mobile communication technologies guided by classical information theory have been close to the limit of Shannon's physical capacity and are difficult to meet the communication needs of 6G mobile communication users. Therefore, it is urgent to study new levels of information communication methods to achieve target communications in different scenarios and meet the communication needs of different users. Summary of the invention

[0003] The purpose of the present invention is to provide a semantic communication method, device, network equipment and medium, so as to achieve target communication in different scenarios and meet the communication needs of different users.

[0004] To achieve the above object, an embodiment of the present invention provides a semantic communication method, which is applied to a sending end and includes:

[0005] In the case where it is determined that the information to be transmitted needs to be transmitted through a semantic coding channel, determining whether the source channel joint encoder between the transmitting end and the receiving end has a corresponding semantic coding processing capability;

[0006] When it is determined that the source-channel joint encoder does not have the semantic coding processing capability, the information to be transmitted is sent to a cloud server, and the cloud server transmits the information to be transmitted to a receiving end.

[0007] Optionally, the method further comprises:

[0008] Determining whether the information to be transmitted is suitable for transmission to the receiving end through a channel encoder channel according to a channel selection strategy;

[0009] In the case where it is determined that the information to be transmitted is not eligible for transmission through the channel encoder channel, it is determined that the information to be transmitted is transmitted to the receiving end through the semantic coding channel.

[0010] Optionally, the method, wherein the step of judging whether the information to be transmitted is suitable for transmission to the receiving end through a channel encoder channel according to a channel selection strategy, comprises:

[0011] Establishing an ontology model; wherein the ontology model includes a channel selection strategy;

[0012] Acquire the signal power of the information to be transmitted, the noise power of the channel encoder channel and the computing power value of the transmitting end device according to the channel selection strategy;

[0013] Acquire a signal-to-noise ratio of the channel encoder channel according to the signal power and the noise power;

[0014] It is determined whether the information to be transmitted is suitable for transmission to the receiving end through the channel encoder channel according to the signal-to-noise ratio and the computing power value of the sending end device.

[0015] Optionally, in the method, sending the information to be transmitted to the cloud server comprises at least one of the following:

[0016] After semantic coding of the information to be transmitted, the information is sent to the cloud server;

[0017] The information to be transmitted is directly sent to the cloud server.

[0018] Optionally, the method further comprises:

[0019] When it is determined that the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capability, the information to be transmitted is transmitted to the receiving end through the source-channel joint encoder.

[0020] Optionally, the method, wherein transmitting the information to be transmitted to the receiving end through the source-channel joint encoder, comprises:

[0021] Identifying and extracting target semantic features of the information to be transmitted;

[0022] Encoding the target semantic features to obtain coded transmission information;

[0023] The coded transmission information is transmitted to the receiving end through a noisy channel for semantic decoding processing.

[0024] Optionally, in the method, the determining whether the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capability includes:

[0025] Acquire a first computing power value of the sending end, and acquire a second computing power value of the receiving end;

[0026] Determine whether the source-channel joint encoder has a corresponding semantic information processing model to obtain a first determination result;

[0027] It is determined whether the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capabilities according to the first computing power value, the second computing power value and the first judgment result.

[0028] In order to achieve the above object, the present invention also provides a semantic communication method, which is applied to a cloud server and comprises:

[0029] Receiving information to be transmitted sent by a transmitting end: wherein the information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder does not have the semantic encoding processing capability;

[0030] Send the information to be transmitted to the receiving end.

[0031] Optionally, in the method, the receiving the information to be transmitted sent by the transmitting end includes:

[0032] Receive the information to be transmitted after semantic coding processing sent by the sending end.

[0033] Optionally, in the method, the sending the information to be transmitted to the receiving end comprises:

[0034] The information to be transmitted after semantic coding processing is sent to the receiving end.

[0035] Optionally, the method, wherein the sending the information to be transmitted to the receiving end, further comprises:

[0036] Performing semantic decoding processing on the information to be transmitted;

[0037] The information to be transmitted after semantic decoding processing is sent to the receiving end.

[0038] Optionally, in the method, after receiving the information to be transmitted sent by the transmitting end, the method further includes:

[0039] Perform semantic coding processing on the information to be transmitted.

[0040] Optionally, in the method, the sending the information to be transmitted to the receiving end comprises:

[0041] The information to be transmitted after semantic coding processing is sent to the receiving end.

[0042] In order to achieve the above object, the present invention also provides a semantic communication method, wherein, applied to a receiving end, the method comprises:

[0043] Receive first information to be transmitted sent by the cloud server; wherein, the first information to be transmitted is sent to the cloud server when the sending end determines that the source-channel joint encoder does not have the semantic coding processing capability.

[0044] Optionally, the method further comprises:

[0045] Receive the second information to be transmitted sent by the transmitting end; wherein the second information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder has the semantic coding processing capability.

[0046] Optionally, in the method, the receiving the first information to be transmitted sent by the cloud server includes at least one of the following:

[0047] In a case where the receiving end does not have semantic communication capability, receiving the first information to be transmitted after semantic decoding processing sent by the cloud server;

[0048] In the case where the receiving end has semantic communication capability, the first information to be transmitted that has been semantically encoded and sent by the cloud server is received, and semantic decoding is performed on the first information to be transmitted.

[0049] In order to achieve the above object, the present invention further provides a semantic communication device, which is applied to a sending end and includes:

[0050] A first judgment module is used to judge whether the source channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capabilities when it is determined that the information to be transmitted needs to be transmitted through the semantic coding channel;

[0051] The first sending module is used to send the information to be transmitted to the cloud server when it is determined that the source-channel joint encoder does not have the semantic coding processing capability, and the cloud server transmits the information to be transmitted to the receiving end.

[0052] In order to achieve the above object, the present invention further provides a semantic communication device, which is applied to a cloud server and comprises:

[0053] A first receiving module is used to receive information to be transmitted sent by a transmitting end: wherein the information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder does not have the semantic coding processing capability;

[0054] The second sending module is used to send the information to be transmitted to the receiving end.

[0055] In order to achieve the above object, the present invention further provides a semantic communication device, which is applied to a receiving end and comprises:

[0056] The second receiving module is used to receive the first information to be transmitted sent by the cloud server; wherein, the first information to be transmitted is sent to the cloud server when the sending end determines that the source-channel joint encoder does not have the semantic coding processing capability.

[0057] In order to achieve the above-mentioned purpose, the present invention also provides a network device, including: a transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; wherein, when the processor executes the program or instruction, it implements the semantic communication method applied to the sending end as described above, or implements the semantic communication method applied to the cloud server as described above, or implements the semantic communication method applied to the receiving end as described above.

[0058] In order to achieve the above-mentioned objectives, the present invention also provides a readable storage medium having a program or instruction stored thereon, wherein when the program or instruction is executed by a processor, the steps of the semantic communication method applied to the sending end as described above are implemented, or the steps of the semantic communication method applied to the cloud server as described above are implemented, or the steps of the semantic communication method applied to the receiving end as described above are implemented.

[0059] The beneficial effects of the above technical solution of the present invention are as follows:

[0060] The semantic communication method of an embodiment of the present invention, when the transmission channel of the information to be transmitted is determined to be a semantic coding channel, determines whether the information to be transmitted needs to be sent to the cloud server and then transmitted to the receiving end by judging whether the source channel joint encoder has the semantic coding processing capability, thereby realizing the use of different target communication methods applicable to different information to be transmitted and meeting the communication needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of a semantic communication method applied to a sending end according to an embodiment of the present invention;

[0062] Figure 2 A schematic diagram of a semantic communication method applied to a cloud server according to an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of a semantic communication method applied to a receiving end according to an embodiment of the present invention;

[0064] Figure 4 A flowchart of the semantic communication method according to an embodiment of the present invention;

[0065] Figure 5 A flow chart of a source-channel joint encoder transmission path of a semantic communication method according to an embodiment of the present invention;

[0066] Figure 6 A schematic diagram of a residual block of the semantic communication method according to an embodiment of the present invention;

[0067] Figure 7 A schematic diagram of a normal block of the semantic communication method according to an embodiment of the present invention;

[0068] Figure 8 A schematic diagram of a semantic communication device applied to a sending end according to an embodiment of the present invention;

[0069] Fig. 9 A schematic diagram of a semantic communication device applied to a cloud server according to an embodiment of the present invention;

[0070] Fig.10 It is a schematic diagram of a semantic communication device applied to a receiving end according to an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0072] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0073] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0074] Additionally, the terms "system" and "network" are often used interchangeably herein.

[0075] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0076] For ease of understanding, some contents involved in the embodiments of the present invention are described below:

[0077] like Figure 1 As shown, a semantic communication method of an embodiment of the present invention, which is applied to a sending end, includes:

[0078] Step S10, when it is determined that the information to be transmitted needs to be transmitted through a semantic coding channel, judging whether the source channel joint encoder between the transmitting end and the receiving end has a corresponding semantic coding processing capability;

[0079] It should be noted that if Figure 4 As shown, when the transmission channel of the information to be transmitted is determined to be a semantic coding channel, that is, the channel encoder of the traditional communication method cannot support the transmission of the information to be transmitted, then considering entering the semantic coding channel for communication, and again judging whether the source channel joint encoder has the semantic coding processing capability through the computing power discriminator.

[0080] Step S20, when it is determined that the source-channel joint encoder does not have the semantic coding processing capability, sending the information to be transmitted to a cloud server, and the cloud server transmits the information to be transmitted to a receiving end;

[0081] It should be noted that when it is determined that the source-channel joint encoder does not have the semantic coding processing capability, the information to be transmitted is transmitted to the receiving end through the cloud server.

[0082] Optionally, the method further comprises:

[0083] Step S30, judging whether the information to be transmitted is suitable for transmission to the receiving end through a channel encoder channel according to a channel selection strategy;

[0084] It should be noted that, the channel selection strategy of the transmitting end is used to determine whether the information to be transmitted can be transmitted to the receiving end using the channel encoder channel to complete the communication.

[0085] Step S40, when it is determined that the information to be transmitted does not conform to transmission through the channel encoder channel, determining that the information to be transmitted is transmitted to the receiving end through the semantic coding channel;

[0086] In a case where the channel quality of the channel encoder channel cannot support the transmission of the information to be transmitted, it is determined to use the semantic coding channel for communication.

[0087] Optionally, in the method, step S30 includes:

[0088] A10, establishing an ontology model; wherein the ontology model includes a channel selection strategy;

[0089] It should be noted that the sending end establishes an ontology model (M s ,C s ,P), where M sThe semantic extraction model information contained in the knowledge base is stored in C s It represents the computing power of the sending device, and P is the channel selection strategy.

[0090] A20, obtaining the signal power of the information to be transmitted, the noise power of the channel encoder channel, and the computing power value of the transmitting end device according to the channel selection strategy;

[0091] It should be noted that the channel selection strategy P is expressed as P=(P S ,P N ,C S ), where P S is the signal power of the information to be transmitted, P N is the noise power of the channel encoder channel, C S is the computing power value of the sending device.

[0092] A30, acquiring a signal-to-noise ratio of the channel encoder channel according to the signal power and the noise power;

[0093] It should be noted that the formula for the signal-to-noise ratio of the channel encoder channel is as follows:

[0094]

[0095] Among them, P S is the signal power of the information to be transmitted, P N is the noise power of the channel encoder channel.

[0096] A40, judging whether the information to be transmitted is suitable for transmission to the receiving end through the channel encoder channel according to the signal-to-noise ratio and the computing power value of the transmitting end device;

[0097] It should be noted that the channel selection strategy is determined by the communication environment between the transmitting device and the receiving device, that is, the signal-to-noise ratio of the channel encoder channel, the information to be transmitted at the transmitting end s = (s1, s2, ..., s n ) and the computing power of the sending device;

[0098] If the information to be transmitted can use the communication mode of the channel encoder channel and the corresponding computing power is sufficient, the communication mode of the channel encoder channel is used;

[0099] If the channel quality of the information to be transmitted cannot be supported by the communication mode of the channel encoder channel, and the computing power of the sending end and the receiving end can support the communication mode of the semantic coding channel, the semantic coding channel is entered for communication.

[0100] Optionally, in the method, step S20 includes at least one of the following:

[0101] A50, performing semantic coding on the information to be transmitted and sending the information to the cloud server;

[0102] It should be noted that, in the transmitting end device main body model (M s ,C s ,P), where C s Indicates the computing power value of the sending device, from M s The appropriate semantic information extraction method is searched in the process to obtain semantic information from the information to be transmitted, and the semantic information is transmitted to the cloud device, and the computing power required for semantic coding communication is recorded. When the computing power value of the sending end device is sufficient to support the computing power required for semantic coding communication and the communication from the sending end to the cloud server can support the relevant rate, the sending end performs semantic coding processing on the information to be transmitted and then sends it to the cloud server.

[0103] A60, directly sending the information to be transmitted to the cloud server;

[0104] It should be noted that, when the computing power of the sending end is not sufficient to support semantic coding communication, the information to be transmitted is directly sent to the cloud server.

[0105] Optionally, the method further comprises:

[0106] Step S50, when it is determined that the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capability, transmitting the information to be transmitted to the receiving end through the source-channel joint encoder;

[0107] It should be noted that when the computing power discriminator determines that the source-channel joint encoder has the processing capacity of the corresponding semantic encoding and has sufficient computing power to complete the communication, the following is used: Figure 5 The semantic communication method of end-to-end joint encoding (decoding) shown in the figure uses a source-channel joint encoder to transmit the information to be transmitted to the receiving end. Specifically, the traditional communication system is a separated source-channel modular encoding and decoding. Common source coding methods include three lossless coding methods such as Huffman coding, arithmetic coding, and LZ coding, as well as lossy source coding technologies such as pulse code modulation (PCM) and differential pulse code modulation (DPCM). Different from the encoding and decoding methods of traditional communication systems, the semantic communication system adds semantic encoding and semantic decoding modules to the traditional communication channel to construct a semantic communication channel, and transmits the information to be transmitted to the receiving end.

[0108] Optionally, in the method, step S50 includes:

[0109] A70, identifying and extracting target semantic features of the information to be transmitted through the source-channel joint encoder and encoding the target semantic features to obtain encoded transmission information;

[0110] It should be noted that the method of using the source-channel joint encoder communication identifies and extracts the target semantic features of the information to be transmitted at the transmitting end, and encodes the target semantic features using a joint source-channel coding (JSCC) scheme.

[0111] A80, transmitting the coded transmission information to the receiving end through a noisy channel for semantic decoding processing;

[0112] It should be noted that the coded transmission information is transmitted through a noisy channel, and a semantic decoder is used at the receiving end to decode the received signal and extract semantic features, and after logical reasoning, the received signal is restored to a signal in a form understandable to the receiving end user.

[0113] In this embodiment, the receiving end restores the received signal into a form understandable to the receiving end user, including a text signal, an image signal, a voice signal and a multi-modal signal.

[0114] For text signals, an end-to-end source-channel joint encoding / decoding scheme is designed based on the converter and self-attention mechanism.

[0115]

[0116] A measure of the information recovered by the receiver The sentence similarity between the information s sent by the sender and

[0117]

[0118] B φ () is the BERT model, which is pre-trained on billions of sentences and maps each sentence to its semantic vector space.

[0119] For image signals, a source-channel joint coding (decoding) scheme combining image transmission and scene classification is used. By using deep reinforcement learning to identify the most basic semantic features based on the transmission task, a residual block (such as Figure 6 As shown) solves the gradient diffusion problem in back propagation, Figure 7At the same time, by using compressed sensing, a high-resolution image signal can be generated from a low-resolution image signal.

[0120] Assume the input signal is X lr ,lr means the input image is a low-resolution image and the channel gain is g. θ It is a ResNet18 deep learning model for image feature extraction. ψ is a multi-layer perceptron (MLP), f φ It is also an MLP for joint decision making, with θ, φ, and ψ as corresponding parameters.

[0121] When the channel gain information is g, the input image signal X lr The semantic extraction probability is:

[0122] p = sigmoid(f ψ (f θ (X lr )+f φ (g)))

[0123] After the above process, when the channel gain information is g, the input image signal X lr Processing to obtain high-resolution image X lh .

[0124] For speech signals, the transmission quality not only involves the fidelity, loudness, frequency, and pitch of the speech signal, but also involves the emotions of the sender, that is, the same text may express different emotions. For example, "What's wrong with you?" If said in a soft tone, it expresses concern for a person; if said in an anxious tone, it expresses a complaint. In addition, it is also difficult to identify dialects from speech signals. By studying the semantic transmission scheme of speech sources, the robustness of the network to noise can be improved by establishing an attention mechanism to squeeze-excite the network.

[0125] For multimodal signals, the E2E semantic communication system of IMs with multimodal information can be studied by studying the correlation between multimodal data from multiple intelligent users. For example, based on advanced natural language processing technology, speech can be effectively translated into text, and then the text can be converted into semantic information for communication. Or the sending end IMs extract the semantic information of heterogeneous messages from different signal transmitters, and then the receiving end IMs directly convert the received semantic information into modal information that the destination user can understand according to the needs of the destination user.

[0126] Optionally, in the method, step S10 includes:

[0127] A90, obtaining a first computing power value of the sending end, and obtaining a second computing power value of the receiving end;

[0128] A100, determining whether the source-channel joint encoder has a corresponding semantic information processing model, and obtaining a first determination result;

[0129] A110, judging whether the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capability according to the first computing power value, the second computing power value and the first judgment result.

[0130] In this embodiment, the computing power discriminator determines that the source-channel joint encoder has a corresponding semantic information processing model and that the sending end and the receiving end have sufficient computing power to complete the communication. Then, the information to be transmitted reaches the receiving end through the source-channel joint encoder to complete the communication.

[0131] like Figure 2 As shown, in order to achieve the above-mentioned purpose, the present invention also provides a semantic communication method, which is applied to a cloud server and includes:

[0132] Step B10, receiving information to be transmitted sent by the transmitting end: wherein the information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder does not have the semantic encoding processing capability;

[0133] It should be noted that when the computing power discriminator at the sending end determines that the source-channel joint encoder does not have the semantic coding processing capability, it enters the cloud server channel to complete the semantic communication.

[0134] Step B20: sending the information to be transmitted to the receiving end.

[0135] Optionally, in the method, step B10 comprises:

[0136] C10, receiving the information to be transmitted after semantic coding processing sent by the sending end;

[0137] It should be noted that the communication between the sending end and the cloud server can support the relevant rate, and the computing power of the sending end is sufficient for semantic communication, the sending end performs semantic coding on the information to be transmitted, and the cloud server receives the information to be transmitted after the semantic coding sent by the sending end.

[0138] Optionally, in the method, step B20 includes:

[0139] C20, sending the information to be transmitted after semantic coding processing to the receiving end;

[0140] It should be noted that, when the sending end and the receiving end have semantic communication capabilities, the sending end performs semantic coding on the information to be transmitted, and then the cloud server forwards the semantically coded information to the receiving end.

[0141] Optionally, in the method, step B20 further comprises:

[0142] C30, performing semantic decoding processing on the information to be transmitted;

[0143] C40, sending the information to be transmitted after semantic decoding processing to the receiving end.

[0144] In this embodiment, when the sending end has semantic communication capability and the receiving end does not have semantic communication capability, the cloud server performs semantic decoding on the information to be transmitted after semantic encoding processing on the sending end, and then sends it to the receiving end.

[0145] Optionally, the method, wherein after step B10, the method further comprises:

[0146] B30, performing semantic coding processing on the information to be transmitted;

[0147] It should be noted that, when the sending end does not have semantic communication capability, the cloud server performs semantic encoding processing on the information to be transmitted.

[0148] Optionally, in the method, step B20 includes:

[0149] C50, sending the information to be transmitted after semantic coding processing to the receiving end;

[0150] It should be noted that, after the cloud server performs semantic coding on the information to be transmitted, the cloud server sends the semantically coded information to the receiving end.

[0151] like Figure 3 As shown, in order to achieve the above-mentioned purpose, the present invention also provides a semantic communication method, wherein, applied to a receiving end, the method comprises:

[0152] D10, receiving first information to be transmitted sent by the cloud server; wherein, the first information to be transmitted is sent to the cloud server when the sending end determines that the source-channel joint encoder does not have the semantic coding processing capability.

[0153] In this embodiment, when it is determined that the source-channel joint encoder does not have the semantic coding processing capability, a cloud server is introduced as a relay for information transmission, and the first information to be transmitted is sent to the receiving end through the cloud server.

[0154] Optionally, the method further comprises:

[0155] D20, receiving the second information to be transmitted sent by the transmitting end; wherein the second information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder has the semantic coding processing capability;

[0156] It should be noted that, when it is determined that the source-channel joint encoder has the semantic coding processing capability, the transmitting end directly sends the second information to be transmitted to the receiving end through the source-channel joint encoder channel.

[0157] Optionally, in the method, step D10 includes at least one of the following:

[0158] E10, receiving the first information to be transmitted after semantic decoding and sent by the cloud server when the receiving end does not have semantic communication capability;

[0159] It should be noted that the receiving end device body model is (M r , C r ), where M r The semantic recovery model information contained in the knowledge base is stored in C r The computing power value of the device at the receiving end is represented. When it is determined by the receiving end device ontology model that the communication between the cloud server and the receiving end cannot support the relevant rate, or the computing power of the cloud server and the receiving end is insufficient for semantic communication, the cloud server performs semantic decoding processing on the first information to be transmitted, and the receiving end receives the first information to be transmitted after semantic decoding processing sent by the cloud server.

[0160] E20, when the receiving end has semantic communication capability, receiving the first information to be transmitted after semantic encoding processing sent by the cloud server, and performing semantic decoding processing on the first information to be transmitted;

[0161] It should be noted that, when the communication from the cloud server to the receiving end can support the relevant rate and the computing power of the cloud server and the receiving end is sufficient for semantic communication, the receiving end receives the first information to be transmitted after semantic encoding and performs semantic decoding processing.

[0162] In the embodiment of the present invention, a dual-channel model of a channel encoder channel and a semantic encoding channel is set in a communication system according to the signal-to-noise ratio, the first information to be transmitted and the computing power to process the first information to be transmitted in different situations; a source channel joint encoding (decoding) module is introduced to perform semantic extraction and recovery on the first information to be transmitted; a cloud server is introduced to solve the problem of insufficient system computing power; and a cloud server is introduced as a relay device to forward and enhance the signal to support the smooth progress of semantic communication. This effectively improves the reliability of communication, expands the coverage of the signal, saves a lot of computing power resources, facilitates network communication, and improves the working efficiency of the network.

[0163] like Figure 8 As shown, in order to achieve the above-mentioned purpose, the present invention also provides a semantic communication device, which is applied to a sending end and includes:

[0164] A first judgment module 81 is used to judge whether the source channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capabilities when it is determined that the information to be transmitted needs to be transmitted through the semantic coding channel;

[0165] The first sending module 82 is used to send the information to be transmitted to the cloud server when it is determined that the source-channel joint encoder does not have the semantic coding processing capability, and the cloud server transmits the information to be transmitted to the receiving end.

[0166] Optionally, the device further comprises:

[0167] A second judgment module is used to judge whether the information to be transmitted is suitable for transmission to the receiving end through a channel encoder channel according to a channel selection strategy;

[0168] The first determination module is used to determine that the information to be transmitted is transmitted to the receiving end through the semantic coding channel when it is determined that the information to be transmitted is not compliant with being transmitted through the channel encoder channel.

[0169] Optionally, in the method, the second judgment module includes:

[0170] A first establishing unit, configured to establish an ontology model; wherein the ontology model includes a channel selection strategy;

[0171] A first acquisition unit, configured to acquire, according to the channel selection strategy, the signal power of the information to be transmitted, the noise power of the channel encoder channel, and the computing power value of the transmitting end device;

[0172] A second acquisition unit, configured to acquire a signal-to-noise ratio of the channel encoder channel according to the signal power and the noise power;

[0173] The first judgment unit is used to judge whether the information to be transmitted is suitable for transmission to the receiving end through the channel encoder channel according to the signal-to-noise ratio and the computing power value of the sending end device.

[0174] Optionally, in the device, the first sending module 62 includes at least one of the following:

[0175] A first sending unit, configured to perform semantic coding on the information to be transmitted and then send the information to the cloud server;

[0176] The second sending unit is used to send the information to be transmitted directly to the cloud server.

[0177] Optionally, the device further comprises:

[0178] The first transmission module is used to transmit the information to be transmitted to the receiving end through the source-channel joint encoder when it is determined that the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capability.

[0179] Optionally, in the device, the first transmission module includes:

[0180] A third acquisition unit is used to identify and extract the target semantic features of the information to be transmitted through the source-channel joint encoder, and encode the target semantic features to obtain encoded transmission information;

[0181] The first transmission unit is used to transmit the coded transmission information to the receiving end through a noisy channel for semantic decoding processing.

[0182] Optionally, in the device, the first determining module 61 includes:

[0183] A fourth acquisition unit, configured to acquire a first computing power value of the sending end;

[0184] A fifth acquisition unit, configured to acquire a second computing power value of the receiving end;

[0185] a sixth acquisition unit, configured to determine whether the source-channel joint encoder has a corresponding semantic information processing model, and obtain a first determination result;

[0186] A second judgment unit is used to judge whether the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capability according to the first computing power value, the second computing power value and the first judgment result.

[0187] like Fig. 9As shown, in order to achieve the above-mentioned purpose, the present invention also provides a semantic communication device, which is applied to a cloud server, and the device includes:

[0188] A first receiving module 91 is used to receive information to be transmitted sent by a transmitting end: wherein the information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder does not have the semantic coding processing capability;

[0189] The second sending module 92 is used to send the information to be transmitted to the receiving end.

[0190] Optionally, in the device, the first receiving module 91 includes:

[0191] The first receiving unit is used to receive the information to be transmitted after semantic coding processing sent by the sending end.

[0192] Optionally, in the device, the second sending module 92 includes:

[0193] The third sending unit is used to send the information to be transmitted after semantic coding processing to the receiving end.

[0194] Optionally, in the device, the second sending module 92 further includes:

[0195] A first processing unit, configured to perform semantic decoding processing on the information to be transmitted;

[0196] The fourth sending unit is used to send the information to be transmitted after semantic decoding processing to the receiving end.

[0197] Optionally, the device further comprises:

[0198] The first processing module is used to perform semantic coding processing on the information to be transmitted.

[0199] Optionally, in the device, the second sending module 92 includes:

[0200] The fifth sending unit is used to send the information to be transmitted after semantic coding processing to the receiving end.

[0201] like Fig.10 As shown, in order to achieve the above-mentioned purpose, the present invention also provides a semantic communication device, wherein, applied to a receiving end, the device comprises:

[0202] The second receiving module 101 is used to receive first information to be transmitted sent by the cloud server; wherein, the first information to be transmitted is sent to the cloud server when the sending end determines that the source-channel joint encoder does not have the semantic coding processing capability.

[0203] Optionally, the device further comprises:

[0204] The third receiving module is used to receive second information to be transmitted sent by the transmitting end; wherein, the second information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder has the semantic coding processing capability.

[0205] Optionally, in the device, the second receiving module includes at least one of the following:

[0206] A second receiving unit, configured to receive the first information to be transmitted after semantic decoding and sent by the cloud server when the receiving end does not have semantic communication capability;

[0207] The third receiving unit is used to receive the first information to be transmitted after semantic encoding processing sent by the cloud server, and perform semantic decoding processing on the first information to be transmitted when the receiving end has semantic communication capability.

[0208] It should be noted here that the above-mentioned device provided in the embodiment of the present invention can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0209] In order to achieve the above-mentioned purpose, the present invention also provides a network device, including: a transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; wherein, when the processor executes the program or instruction, it implements the semantic communication method applied to the sending end as described above, or implements the semantic communication method applied to the cloud server as described above, or implements the semantic communication method applied to the receiving end as described above.

[0210] In order to achieve the above-mentioned purpose, the present invention also provides a readable storage medium having a program or instruction stored thereon, wherein the program or instruction, when executed by the processor, implements the various processes in the embodiment of the semantic communication method applied to the sending end as described above, or implements the various processes in the embodiment of the semantic communication method applied to the cloud server as described above, or implements the various processes in the embodiment of the semantic communication method applied to the receiving end as described above, and can achieve the same technical effect, which will not be described here to avoid repetition. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0211] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present invention is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0212] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0213] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A semantic communication method, characterized in that: Applied to the sending end, including: In the case where it is determined that the information to be transmitted needs to be transmitted through a semantic coding channel, determining whether the source channel joint encoder between the transmitting end and the receiving end has a corresponding semantic coding processing capability; When it is determined that the source-channel joint encoder does not have the semantic coding processing capability, the information to be transmitted is sent to a cloud server, and the cloud server transmits the information to be transmitted to a receiving end.

2. The method according to claim 1, characterized in that The method further comprises: Determining whether the information to be transmitted is suitable for transmission to the receiving end through a channel encoder channel according to a channel selection strategy; In the case where it is determined that the information to be transmitted is not eligible for transmission through the channel encoder channel, it is determined that the information to be transmitted is transmitted to the receiving end through the semantic coding channel.

3. The method according to claim 2, characterized in that The determining, according to the channel selection strategy, whether the information to be transmitted is suitable for transmission to the receiving end through the channel encoder channel includes: Establishing an ontology model; wherein the ontology model includes a channel selection strategy; Acquire the signal power of the information to be transmitted, the noise power of the channel encoder channel and the computing power value of the transmitting end device according to the channel selection strategy; Acquire a signal-to-noise ratio of the channel encoder channel according to the signal power and the noise power; It is determined whether the information to be transmitted is suitable for transmission to the receiving end through the channel encoder channel according to the signal-to-noise ratio and the computing power value of the sending end device.

4. The method according to claim 1, characterized in that: Sending the information to be transmitted to the cloud server includes at least one of the following: After semantic coding of the information to be transmitted, the information is sent to the cloud server; The information to be transmitted is directly sent to the cloud server.

5. The method according to claim 1, characterized in that The method further comprises: When it is determined that the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capability, the information to be transmitted is transmitted to the receiving end through the source-channel joint encoder.

6. The method according to claim 5, characterized in that Transmitting the information to be transmitted to the receiving end through the source-channel joint encoder includes: Identifying and extracting target semantic features of the information to be transmitted through the source-channel joint encoder, and encoding the target semantic features to obtain encoded transmission information; The coded transmission information is transmitted to the receiving end through a noisy channel for semantic decoding processing.

7. The method according to claim 1, characterized in that The determining whether the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capability includes: Acquire a first computing power value of the sending end, and acquire a second computing power value of the receiving end; Determine whether the source-channel joint encoder has a corresponding semantic information processing model to obtain a first determination result; Whether the source-channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capability is determined according to the first computing power value, the second computing power value and the first judgment result.

8. A semantic communication method, characterized in that: Applied to a cloud server, the method comprises: Receiving information to be transmitted sent by a transmitting end: wherein the information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder does not have the semantic encoding processing capability; Send the information to be transmitted to the receiving end.

9. The method according to claim 8, characterized in that The receiving end sends the information to be transmitted, including: Receive the information to be transmitted after semantic coding processing sent by the sending end.

10. The method according to claim 9, characterized in that The sending the information to be transmitted to the receiving end includes: The information to be transmitted after semantic coding processing is sent to the receiving end.

11. The method according to claim 9, characterized in that The sending of the information to be transmitted to the receiving end further includes: Performing semantic decoding processing on the information to be transmitted; The information to be transmitted after semantic decoding processing is sent to the receiving end.

12. The method according to claim 8, characterized in that After receiving the information to be transmitted sent by the transmitting end, the method further includes: Perform semantic coding processing on the information to be transmitted.

13. The method according to claim 12, characterized in that The sending the information to be transmitted to the receiving end includes: The information to be transmitted after semantic coding processing is sent to the receiving end.

14. A semantic communication method, characterized in that: Applied to the receiving end, the method includes: Receive first information to be transmitted sent by the cloud server; wherein, the first information to be transmitted is sent to the cloud server when the sending end determines that the source-channel joint encoder does not have the semantic coding processing capability.

15. The method according to claim 14, characterized in that The method further comprises: Receive second information to be transmitted sent by the transmitting end; wherein the second information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder has the semantic coding processing capability.

16. The method according to claim 14, characterized in that The receiving of the first information to be transmitted sent by the cloud server includes at least one of the following: In a case where the receiving end does not have semantic communication capability, receiving the first information to be transmitted after semantic decoding processing sent by the cloud server; In the case where the receiving end has semantic communication capability, the first information to be transmitted that has been semantically encoded and sent by the cloud server is received, and semantic decoding is performed on the first information to be transmitted.

17. A semantic communication device, characterized in that: Applied to the sending end, including: A first judgment module is used to judge whether the source channel joint encoder between the transmitting end and the receiving end has corresponding semantic coding processing capabilities when it is determined that the information to be transmitted needs to be transmitted through the semantic coding channel; The first sending module is used to send the information to be transmitted to the cloud server when it is determined that the source-channel joint encoder does not have the semantic coding processing capability, and the cloud server transmits the information to be transmitted to the receiving end.

18. A semantic communication device, characterized in that: Applied to a cloud server, the device comprises: A first receiving module is used to receive information to be transmitted sent by a transmitting end: wherein the information to be transmitted is sent by the transmitting end when it is determined that the source-channel joint encoder does not have the semantic coding processing capability; The second sending module is used to send the information to be transmitted to the receiving end.

19. A semantic communication device, characterized in that: Applied to a receiving end, the device comprises: The second receiving module is used to receive the first information to be transmitted sent by the cloud server; wherein, the first information to be transmitted is sent to the cloud server when the sending end determines that the source-channel joint encoder does not have the semantic coding processing capability.

20. A network device comprising: A transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; characterized in that when the processor executes the program or instruction, it implements the semantic communication method applied to the sending end as described in any one of claims 1 to 7, or implements the semantic communication method applied to the cloud server as described in any one of claims 8 to 13, or implements the semantic communication method applied to the receiving end as described in any one of claims 14 to 16.

21. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by the processor, it implements the steps of the semantic communication method applied to the sending end as described in any one of claims 1-7, or implements the steps of the semantic communication method applied to the cloud server as described in any one of claims 8-13, or implements the steps of the semantic communication method applied to the receiving end as described in any one of claims 14-16.

Images