Semantic communication methods, communication device and communication system

Abstract

The present disclosure relates to the technical field of communications. Provided are semantic communication methods, a communication device and a communication system. A semantic communication method comprises: a first device, which serves as a transmitting end, first acquiring semantic information at different levels from a semantic recognition result of data to be sent; then performing superposition coded modulation on the semantic information at different levels to obtain a modulated signal; and then sending the modulated signal to a plurality of second devices, which serve as receiving ends, such that the second devices can obtain semantic information at corresponding levels on the basis of channel conditions thereof, thereby meeting the communication requirements of different receiving ends under different channel conditions.

Description

Semantic communication methods, communication devices and communication systems Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a semantic communication method, communication device and communication system. Background Technology

[0002] With the rapid development of mobile communication networks, new communication scenarios, such as the metaverse and drones, are emerging, marking a shift in communication towards human-machine collaboration. Simultaneously, the massive amounts of data generated in this new era, such as radar data under new modulation techniques, are significantly impacting traditional communication systems. To address information overload, semantic communication (SC) represents a new paradigm for meeting the demands of massive data transmission in modern mobile communications. Summary of the Invention

[0003] This disclosure proposes a semantic communication method, communication device, and communication system, which can solve the technical problem of how to achieve different levels of semantic information required by receivers under different channel conditions.

[0004] A first aspect of this disclosure provides a semantic communication method executed by a first device. The method includes: obtaining semantic information at different levels from the semantic recognition results of data to be transmitted; superimposing and encoding the semantic information at different levels to obtain a modulated signal; and transmitting the modulated signal to a plurality of second devices, wherein the modulated signal is used to determine semantic information at a level corresponding to the channel conditions of the second devices.

[0005] A second aspect of this disclosure provides a semantic communication method executed by a second device. The method includes: receiving a modulation signal sent by a first device, the modulation signal being obtained by superimposing and encoding modulation based on semantic information of different levels of data to be transmitted; and determining semantic information of a level corresponding to the channel conditions of the second device based on the modulation signal.

[0006] A third aspect of this disclosure provides a first device, comprising: a processing module and a transceiver module, wherein the processing module is configured to obtain semantic information at different levels from the semantic recognition results of data to be transmitted; to superimpose and encode the semantic information at different levels to obtain a modulated signal; and the transceiver module is configured to transmit the modulated signal to a plurality of second devices, wherein the modulated signal is used to determine semantic information at a level corresponding to the channel conditions of the second devices.

[0007] A fourth aspect of this disclosure provides a second device, including a transceiver module and a processing module. The transceiver module is configured to receive a modulation signal transmitted by a first device, the modulation signal being obtained by superimposing and encoding modulation based on semantic information of different levels of data to be transmitted; the processing module is configured to determine semantic information corresponding to the channel conditions of the second device based on the modulation signal.

[0008] A fifth aspect of this disclosure provides a communication device for performing the method described in the first aspect embodiment or the method described in the second aspect embodiment.

[0009] A sixth aspect of this disclosure provides a communication system including a first device and a second device, wherein the first device is configured to implement the method described in the first aspect embodiment, and the second device is configured to implement the method described in the second aspect embodiment.

[0010] A seventh aspect embodiment of this disclosure provides a storage medium that, when the instructions are executed on a communication device, causes the communication device to perform the method as described in the first aspect embodiment or the method as described in the second aspect embodiment.

[0011] An eighth aspect of this disclosure provides a program product including at least one program and instructions, which, when executed by a communication device, implement the method described in the first aspect embodiment or the method described in the second aspect embodiment.

[0012] The technical solution provided in this disclosure, for semantic communication, allows a first device, acting as a transmitter, to first obtain semantic information at different levels from the semantic recognition results of the data to be transmitted; then, the semantic information at different levels is superimposed, encoded, and modulated to obtain a modulated signal; and then, the modulated signal is sent to multiple second devices, acting as receivers, so that the second devices can obtain semantic information at the corresponding level according to their own channel conditions, thus meeting the communication needs of different receivers under different channel conditions.

[0013] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings required for the description of the embodiments are introduced below. The following drawings are only some embodiments of this disclosure and do not impose specific limitations on the protection scope of this disclosure.

[0015] Figure 1 is a schematic diagram of the architecture of a communication system provided in an embodiment of this disclosure.

[0016] Figure 2A is an interactive schematic diagram of a semantic communication method provided in an embodiment of this disclosure.

[0017] Figure 2B is a schematic diagram of an example provided by an embodiment of this disclosure;

[0018] Figure 2C is a schematic diagram of another example provided by the embodiments of this disclosure;

[0019] Figure 2D is a schematic diagram of yet another example provided in the embodiments of this disclosure;

[0020] Figure 2E is a schematic diagram of yet another example provided in the embodiments of this disclosure;

[0021] Figure 3 is a schematic diagram of an example of a semantic communication method provided in an embodiment of this disclosure.

[0022] Figure 4A is a structural block diagram of a first device provided in an embodiment of this disclosure.

[0023] Figure 4B is a structural block diagram of a second device provided in an embodiment of this disclosure.

[0024] Figure 5A is a schematic diagram of the structure of a communication device provided in an embodiment of this disclosure.

[0025] Figure 5B is a schematic diagram of the structure of a chip provided in an embodiment of this disclosure. Detailed Implementation

[0026] The embodiments of this disclosure are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure. It should be noted that, unless otherwise specified, the embodiments of this disclosure and the features in the embodiments can be combined with each other.

[0027] This disclosure presents a semantic communication method, communication device, and communication system.

[0028] In a first aspect, embodiments of this disclosure propose a semantic communication method executed by a first device. The method includes: obtaining semantic information at different levels from the semantic recognition results of data to be transmitted; superimposing and encoding the semantic information at different levels to obtain a modulated signal; and transmitting the modulated signal to a plurality of second devices, wherein the modulated signal is used to determine semantic information at a level corresponding to the channel conditions of the second devices.

[0029] The technical solutions provided in this disclosure can meet the communication needs of different receivers under different channel conditions.

[0030] In conjunction with some embodiments of the first aspect, superimposed encoding and modulation of the semantic information at different levels to obtain a modulated signal includes: encoding the semantic information at different levels into a first feature vector and at least one second feature vector, wherein the first feature vector is used to determine the basic semantic features of the data to be transmitted, and the at least one second feature vector is used to determine the enhanced semantic features that are progressively refined from the basic semantic features; and superimposed modulation based on the first feature vector and the at least one second feature vector to obtain the modulated signal.

[0031] This method encodes semantic information at different levels and then superimposes and modulates it, thus accurately obtaining the modulated signal.

[0032] In conjunction with some embodiments of the first aspect, the modulation signal is obtained by superimposing modulation based on the first feature vector and the at least one second feature vector, including: generating a first constellation sequence based on the first feature vector, and generating at least one second constellation sequence based on the at least one second feature vector; and generating the modulation signal by superimposing the first constellation sequence and the at least one second constellation sequence.

[0033] This method allows for accurate superposition modulation, thereby accurately generating a modulated signal that can meet the communication needs of different receivers under different channel conditions.

[0034] In conjunction with some embodiments of the first aspect, generating at least one second constellation sequence based on the at least one second feature vector includes: performing decorrelation processing on the at least one second feature vector according to the first feature vector; and generating the at least one second constellation sequence based on the decorrelated at least one second feature vector.

[0035] This method minimizes redundancy in the communication process and reduces the amount of data transmitted.

[0036] In conjunction with some embodiments of the first aspect, decorrelation processing of the second feature vector based on the first feature vector includes: extracting residual information unrelated to the first feature vector from the second feature vector using a linear minimum mean square error (LMMSE) decorrelator; and determining the decorrelated second feature vector based on the residual information.

[0037] This method enables decorrelation processing of the second feature vector, thereby reducing redundancy in the communication process and reducing the amount of data transmitted.

[0038] In conjunction with some embodiments of the first aspect, generating a first constellation sequence based on the first feature vector includes: generating the first constellation sequence based on random sampling of the first feature vector using a first transition probability.

[0039] This method can accurately generate the corresponding constellation sequence, which is convenient for merging to generate the modulation signal.

[0040] In conjunction with some embodiments of the first aspect, generating a second constellation sequence based on the second feature vector includes: generating the second constellation sequence based on random sampling of the second feature vector using a second transition probability.

[0041] This method can accurately generate the corresponding constellation sequence, making it easy to merge and generate the modulated signal.

[0042] In conjunction with some embodiments of the first aspect, the data to be transmitted includes image data, and the modulation signal is further used to determine the image corresponding to the channel conditions of the second device.

[0043] In this way, if the data to be sent includes image data, the receiving second device can recover the corresponding image based on its channel conditions during semantic communication.

[0044] In conjunction with some embodiments of the first aspect, the channel conditions are determined by at least one of the following:

[0045] Signal-to-noise ratio; signal strength; bit error rate, etc.

[0046] These parameters allow for the accurate determination of the channel conditions for each secondary device.

[0047] Secondly, this disclosure provides a semantic communication method executed by a second device. The method includes: receiving a modulation signal sent by a first device, the modulation signal being obtained by superimposing and encoding modulation based on semantic information of different levels of data to be transmitted; and determining semantic information of a level corresponding to the channel conditions of the second device based on the modulation signal.

[0048] The technical solutions provided in this disclosure can meet the communication needs of different receivers under different channel conditions.

[0049] In conjunction with some embodiments of the second aspect, determining semantic information corresponding to the channel conditions of the second device based on the modulation signal includes: determining a receiving sequence corresponding to the channel conditions based on the modulation signal; and determining semantic information corresponding to the channel conditions based on the receiving sequence.

[0050] In this way, semantic information corresponding to the channel conditions of the second device can be accurately obtained.

[0051] In conjunction with some embodiments of the second aspect, determining the semantic information corresponding to the channel condition level based on the received sequence includes: demodulating the received sequence to obtain a feature vector; and decoding the feature vector to obtain the semantic information corresponding to the channel condition level.

[0052] By demodulating and then decoding in this way, semantic information corresponding to the channel conditions of the second device can be accurately obtained.

[0053] In some embodiments of the second aspect, the data to be transmitted is image data, and the method further includes: decoding based on the feature vector to obtain the image corresponding to the channel conditions.

[0054] In this way, if the data to be sent includes image data, the receiving second device can recover the corresponding image based on its channel conditions during semantic communication.

[0055] In conjunction with some embodiments of the second aspect, the channel conditions are determined by at least one of the following:

[0056] Signal-to-noise ratio; signal strength; bit error rate, etc.

[0057] These parameters allow for the accurate determination of the channel conditions for each secondary device.

[0058] Thirdly, embodiments of this disclosure propose a first device, the first device comprising: a processing module and a transceiver module, the processing module being configured to obtain semantic information at different levels from the semantic recognition results of data to be transmitted; to superimpose and encode the semantic information at different levels to obtain a modulated signal; the transceiver module being configured to transmit the modulated signal to a plurality of second devices, the modulated signal being used to determine semantic information at a level corresponding to the channel conditions of the second devices.

[0059] Fourthly, embodiments of this disclosure propose a second device, the second device comprising: a transceiver module and a processing module, the transceiver module being configured to receive a modulation signal transmitted by a first device, the modulation signal being obtained by superimposing and coding modulation based on semantic information of different levels of data to be transmitted; the processing module being configured to determine semantic information corresponding to the channel conditions of the second device based on the modulation signal.

[0060] Fifthly, embodiments of this disclosure provide a communication device for performing the method as described in the first aspect embodiment or the method as described in the second aspect embodiment.

[0061] In a sixth aspect, embodiments of this disclosure provide a communication system, including: a first device and a second device; the first device is configured to perform the method described in the first aspect embodiment, and the second device is configured to perform the method described in the second aspect embodiment.

[0062] In a seventh aspect, embodiments of this disclosure provide a storage medium that, when the instructions are executed on a communication device, causes the communication device to perform the method as described in the first aspect embodiment or the second aspect embodiment.

[0063] Eighthly, embodiments of this disclosure provide a program product comprising at least one of a program and instructions, wherein the program and at least one of the instructions, when executed by a communication device, implement the method as described in the first aspect embodiment or the second aspect embodiment.

[0064] In a ninth aspect, embodiments of this disclosure provide a computer program that, when run on a computer, causes the computer to perform the methods described in the first aspect embodiment or the second aspect embodiment.

[0065] In a tenth aspect, embodiments of this disclosure provide a chip or chip system. The chip or chip system includes processing circuitry configured to perform the methods described in the first aspect embodiment, the second aspect embodiment, or the seventh aspect embodiment.

[0066] It is understood that the aforementioned first device, second device, communication device, communication system, storage medium, etc., are all used to execute the methods proposed in the embodiments of this disclosure. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0067] This disclosure provides a semantic communication method, communication device, and communication system. In some embodiments, the terms semantic communication method, information processing method, information sending method, and information receiving method can be used interchangeably; the terms semantic communication device, information processing device, information sending device, and information receiving device can be used interchangeably; and the terms information processing system, communication system, information sending system, and information receiving system can be used interchangeably.

[0068] This disclosure is not exhaustive, but merely illustrative of some embodiments, and is not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0069] In each of the disclosed embodiments, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0070] The terminology used in the embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure.

[0071] In this embodiment of the disclosure, unless otherwise stated, elements expressed in the singular form, such as "a," "an," "the," "the," "the," "the," "the," "the," "this," etc., can mean "one and only one," or "one or more," "at least one," etc. For example, when using articles such as "a," "an," "the," etc. in translation, the noun following the article can be understood as either a singular expression or a plural expression.

[0072] In the embodiments disclosed herein, "multiple" refers to two or more.

[0073] In some embodiments, the terms “at least one of”, “at least one of”, “at least one of”, “one or more”, “a plurality of”, “multiple”, etc., may be used interchangeably.

[0074] The descriptions in this disclosure, such as "at least one of A, B, C..." or "A and / or B and / or C...", include the case where any one of A, B, C... exists alone, as well as the case where any combination of any of A, B, C... exists alone. Each case can exist alone. For example, "at least one of A, B, C" includes the cases of A alone, B alone, C alone, A and B combination, A and C combination, B and C combination, and A and B and C combination. For example, A and / or B includes the cases of A alone, B alone, and A and B combination.

[0075] In some embodiments, the notation "in one case A, in another case B" or "in response to one case A, in response to another case B" may include the following technical solutions depending on the situation: A is executed regardless of B, i.e., A is executed in some embodiments; B is executed regardless of A, i.e., B is executed in some embodiments; A and B are selectively executed, i.e., A and B are selected for execution in some embodiments; A and B are both executed, i.e., A and B are executed in some embodiments. The same applies when there are more branches such as A, B, and C.

[0076] The prefixes "first," "second," etc., used in the embodiments of this disclosure are merely for distinguishing different descriptive objects and do not impose restrictions on the position, order, priority, quantity, or content of the descriptive objects. The description of the descriptive objects is found in the claims or the context of the embodiments, and the use of prefixes should not constitute unnecessary restrictions. For example, if the descriptive object is a "field," the ordinal numbers preceding "field" in "first field" and "second field" do not restrict the position or order of the "fields." "First" and "second" do not restrict whether the "fields" they modify are in the same message, nor do they restrict the order of "first field" and "second field." Similarly, if the descriptive object is a "level," the ordinal numbers preceding "level" in "first level" and "second level" do not restrict the priority between "levels." Furthermore, the number of descriptive objects is not limited by ordinal numbers and can be one or more. For example, in "first device," the number of "devices" can be one or more. Furthermore, the objects modified by different prefixes can be the same or different. For example, if the object being described is "device", then "first device" and "second device" can be the same device or different devices, and their types can be the same or different. Similarly, if the object being described is "information", then "first information" and "second information" can be the same information or different information, and their content can be the same or different.

[0077] In some embodiments, “including A,” “containing A,” “for indicating A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0078] In some embodiments, the terms “in response to…”, “in response to determining…”, “in the case of…”, “when…”, “if…”, “if…”, etc., can be used interchangeably.

[0079] In some embodiments, the terms “greater than,” “greater than or equal to,” “not less than,” “more than,” “more than or equal to,” “not less than,” “higher than,” “higher than or equal to,” “not lower than,” and “above” can be used interchangeably, as can the terms “less than,” “less than or equal to,” “not greater than,” “less than,” “less than or equal to,” “not more than,” “lower than,” “lower than or equal to,” “not higher than,” and “below”.

[0080] In some embodiments, devices, etc., can be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. Terms such as “device”, “equipment”, “circuit”, “network element”, “node”, “function”, “unit”, “section”, “system”, “network”, “chip”, “chip system”, “entity”, and “subject” can be used interchangeably.

[0081] In some embodiments, "network" can be interpreted as devices included in a network (e.g., access network devices, core network devices, etc.).

[0082] In some embodiments, the terms "access network device (AN device)," "radio access network device (RAN device)," "base station (BS)," "radio base station," "fixed station," "node," "access point," "transmission point (TP)," "reception point (RP)," "transmission / reception point (TRP)," "panel," "antenna panel," "antenna array," "cell," "macro cell," "small cell," "femto cell," "pico cell," "sector," "cell group," "carrier," "component carrier," and "bandwidth part (BWP)" can be used interchangeably.

[0083] In some embodiments, the terms "terminal", "terminal device", "user equipment (UE)", "user terminal", "mobile station (MS)", "mobile terminal (MT)", "subscriber station", "mobile unit", "subscriber unit", "wireless unit", "remote unit", "mobile device", "wireless device", "wireless communication device", "remote device", "mobile subscriber station", "access terminal", "mobile terminal", "wireless terminal", "remote terminal", "handset", "user agent", "mobile client", "client", and "narrowband Internet of Things (NB-IoT) device" can be used interchangeably.

[0084] In some embodiments, access network devices, core network devices, or network devices can be replaced by terminals. For example, embodiments of this disclosure can also be applied to structures that replace communication between access network devices, core network devices, or network devices and terminals with communication between multiple terminals (e.g., also referred to as device-to-device (D2D), vehicle-to-everything (V2X), etc.). In this case, the structure can also be configured such that the terminal has all or part of the functions of the access network device. Furthermore, terms such as "uplink" and "downlink" can be replaced with terms corresponding to communication between terminals (e.g., "sidelink"). For example, uplink channel, downlink channel, etc., can be replaced with sidelink channel, uplink link, downlink link, etc., can be replaced with sidelink link.

[0085] In some embodiments, the terminal may be replaced by an access network device, a core network device, or a network device. In this case, the access network device, core network device, or network device may also be configured to have all or some of the functions of the terminal.

[0086] In some embodiments, the acquisition of data, information, etc., may comply with the laws and regulations of the country where the location is situated.

[0087] In some embodiments, data, information, etc., may be obtained with the user's consent.

[0088] In some embodiments, the threshold mentioned in this embodiment may be a numerical value, a constant, or some fixed value.

[0089] Furthermore, each element, each row, or each column in the table of this disclosure can be implemented as an independent embodiment, and any combination of any element, any row, or any column can also be implemented as an independent embodiment.

[0090] The correspondences shown in the tables of this disclosure can be configured or predefined. The values ​​of the information in each table are merely examples and can be configured to other values; this disclosure is not limiting. When configuring the correspondences between information and parameters, it is not necessarily required to configure all the correspondences shown in each table. For example, the correspondences shown in some rows of the tables in this disclosure may not be configured. Furthermore, appropriate modifications and adjustments can be made based on the above tables, such as splitting, merging, etc. The names of the parameters shown in the headers of the above tables can also use other names that the communication device can understand, and the values ​​or representations of the parameters can also be other values ​​or representations that the communication device can understand. In the implementation of the above tables, other data structures can also be used, such as arrays, queues, containers, stacks, linear lists, pointers, linked lists, trees, graphs, structures, classes, heaps, hash tables, or hash tables, etc.

[0091] The predefined terms in this disclosure can be understood as defined, predefined, stored, pre-stored, pre-negotiated, pre-configured, solidified, or pre-burned.

[0092] The semantic communication method, communication equipment, and communication system provided in this disclosure will be described in detail below with reference to the accompanying drawings.

[0093] Figure 1 shows a structural diagram of a communication system according to an embodiment of the present disclosure. As shown in Figure 1, the system architecture may include a first device 101 and a second device 102.

[0094] In some embodiments, the first device 101 may be a semantic transmitter (or sender, etc.) in semantic communication. In some examples, the first device 101 may be a network device or a terminal, etc.

[0095] In some embodiments, the second device 102 may be a device that is a semantic receiver in semantic communication. In some examples, the second device 102 may be a terminal or network device, etc.

[0096] In some embodiments, network devices may include access network devices, etc.

[0097] In some embodiments, the access network device is, for example, a node or device that connects a terminal to a wireless network. The access network device may include at least one of the following in a 5G communication system: evolved Node B (eNB), next-generation eNB (ng-eNB), next-generation Node B (gNB), node B (NB), home node B (HNB), home evolved node B (HeNB), radio backhaul device, radio network controller (RNC), base station controller (BSC), base transceiver station (BTS), base band unit (BBU), mobile switching center, base station in a 6G communication system, open RAN, cloud RAN, base station in other communication systems, and access node in a Wi-Fi system, but is not limited thereto.

[0098] In some embodiments, the technical solutions of this disclosure can be applied to the Open RAN architecture. In this case, the interfaces between or within access network devices involved in the embodiments of this disclosure can be transformed into internal interfaces of Open RAN. The processes and information interactions between these internal interfaces can be implemented by software or programs.

[0099] In some embodiments, the access network device may be composed of a central unit (CU) and a distributed unit (DU). The CU may also be called a control unit. The CU-DU structure can separate the protocol layer of the access network device. Some of the protocol layer functions are centrally controlled by the CU, while the remaining part or all of the protocol layer functions are distributed in the DU and centrally controlled by the CU. However, this is not the only possibility.

[0100] In some embodiments, the terminal includes, but is not limited to, at least one of the following: mobile phone, wearable device, Internet of Things device, car with communication function, smart car, tablet computer, computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal device in industrial control, wireless terminal device in self-driving, wireless terminal device in remote medical surgery, wireless terminal device in smart grid, wireless terminal device in transportation safety, wireless terminal device in smart city, and wireless terminal device in smart home.

[0101] It is understood that the communication system described in this disclosure is for the purpose of more clearly illustrating the technical solutions of this disclosure, and does not constitute a limitation on the technical solutions proposed in this disclosure. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions proposed in this disclosure are also applicable to similar technical problems.

[0102] The following embodiments of this disclosure can be applied to the communication system shown in FIG1, or some of the subjects, but are not limited thereto. The subjects shown in FIG1 are illustrative. The communication system may include all or some of the subjects in FIG1, or may include other subjects other than those in FIG1. ​​The number and form of each subject are arbitrary. The connection relationship between the subjects is illustrative. The subjects may not be connected to each other or may be connected in any way. The connection may be direct or indirect, wired or wireless.

[0103] The embodiments disclosed herein can be applied to satellite communications, Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G NR, Future Radio Access (FRA), New-Radio Access Technology (RAT), New Radio (NR), New Radio Access (NX), Future Generation Radio Access (FX), Global System for Mobile Communications (GSM), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), and IEEE 802.20, Ultra-Wideband (UWB), Bluetooth (a registered trademark), Public Land Mobile Network (PLMN) networks, Device-to-Device (D2D) systems, Machine-to-Machine (M2M) systems, Internet of Things (IoT) systems, Vehicle-to-Everything (V2X) systems, systems utilizing other semantic communication methods, and next-generation systems built upon them, etc. Furthermore, multiple systems can be combined (e.g., a combination of LTE or LTE-A with 5G).

[0104] In some embodiments, semantic communication can significantly improve transmission efficiency and quality of service compared to traditional Shannon communication. Semantic communication leverages the advantages of deep learning, primarily employing neural networks (NNs) for semantic encoding, demonstrating superior performance in peer-to-peer communication scenarios. In particular, neural network-based semantic encoding has been used to replace traditional source coding and / or channel coding for transmitting various source data types, including voice, text, images, video, and multimodal data.

[0105] In some embodiments, semantic communication systems generally assume that each receiver needs the same semantic information. However, different receivers may have different channel conditions, and therefore require different levels of semantic information.

[0106] To address this, embodiments of this disclosure propose a communication scheme that solves the technical problem of how to achieve different levels of semantic information required by receivers under different channel conditions.

[0107] Figure 2A is an interactive schematic diagram of a semantic communication method according to an embodiment of the present disclosure. As shown in Figure 2A, the method includes the following steps:

[0108] In step S201, the first device obtains semantic information at different levels from the semantic recognition results of the data to be sent.

[0109] In some embodiments, the first device may be a semantic transmitter (or sender, etc.) in semantic communication, and the second device may be a semantic receiver in semantic communication. For example, unlike the first device sending data to be sent to the second device, in semantic communication, the first device sends the semantic information corresponding to the data to be sent to the second device.

[0110] In some embodiments, before sending the semantic information corresponding to the data to be sent, semantic recognition is first performed on the data to be sent, for example, the data to be sent is preprocessed, and then the model is used to perform semantic recognition on the preprocessed data to be sent.

[0111] In some embodiments, semantic information at different levels can be obtained from the semantic recognition results of the data to be sent, wherein the semantic information at two adjacent levels can be hierarchical semantic information. For example, the data to be sent can be animal data. When performing semantic recognition on the animal data, the first level of semantic information can be the general category of the animal, such as cat or dog; the second level of semantic information can be the specific species within these categories, such as tabby cat, Persian cat, husky or schnauzer.

[0112] In step S202, the first device superimposes and encodes semantic information at different levels to obtain a modulated signal.

[0113] In some embodiments, semantic information at different levels can first be encoded into a first feature vector and at least one second feature vector. The first feature vector can be used to determine the basic semantic features of the data to be transmitted, and the at least one second feature vector can be used to determine the enhanced semantic features that are progressively refined from the basic semantic features. For example, for the number of semantic information levels, the same number of neural semantic encoders can be used to encode the semantic information at these levels. The first layer of semantic information can be encoded into basic semantic features, the second layer of semantic information can be encoded into enhanced semantic features that are further refined from the basic semantic features, the third layer of semantic information (if any) can be encoded into enhanced semantic features that are even more refined than the enhanced semantic features, and so on. In this way, encoding can be performed accurately. Then, the first feature vector and the at least one second feature vector obtained by encoding are superimposed and modulated to obtain a modulated signal.

[0114] In some examples, a first constellation sequence is generated based on the first feature vector, and at least one second constellation sequence is generated based on the at least one second feature vector. For example, the first constellation sequence can be generated based on the first feature vector through random sampling with a first transition probability, and the second constellation sequence can be generated based on the second feature vector through random sampling with a second transition probability. Then, the first constellation sequence and at least one second constellation sequence are superimposed to generate a modulated signal. In this way, a modulated signal can be accurately modulated, which may contain different levels of semantic information, allowing the second device at the receiving end to obtain semantic information corresponding to its channel conditions based on the modulated signal.

[0115] In some examples, to reduce redundancy, the at least one second feature vector can first be decorrelated based on the first feature vector; then the at least one second constellation sequence can be generated based on the decorrelated at least one second feature vector, and then superimposed with the first constellation sequence to generate a modulated signal.

[0116] In some examples, the LMMSE decorrelationer can be used to extract residual information that is not related to the first feature vector from the second feature vector; and then the second feature vector after decorrelation processing can be determined based on the residual information.

[0117] In some embodiments, based on the above description, it is equivalent to proposing a deep learning-based DeepSCM (Deep Coding and Modulation) scheme, which can be used for hierarchical semantic communication over Gaussian degenerate broadcast channels. For example, neural networks are used to extract and encode semantic information at different levels from a common source, and then superimposed coding is used to combine them before broadcasting. For instance, hierarchical semantic information is first extracted and encoded into basic feature vectors and enhanced feature vectors. Then, a linear minimum mean square error (LMMSE) decorrecorder is used to obtain refinement from enhanced features that are not related to the basic features. Finally, after probabilistic modulation, the basic features and their refinement are superimposed to achieve broadcasting.

[0118] In some embodiments, the modulated signal is used to determine semantic information at a level corresponding to the channel conditions of each second device. For example, each receiver (i.e., the second device) can decode its required level of semantic information from the superimposed structure symbols according to its own channel conditions.

[0119] In some examples, the channel conditions can be determined by at least one of the following:

[0120] Signal-to-noise ratio (SNR); signal strength; bit error rate; noise; fading; path loss, etc.

[0121] In some embodiments, the data to be transmitted may include image data, and the modulation signal is also used to determine the image corresponding to the channel conditions of the second device.

[0122] In some embodiments, image semantic broadcasting with two receivers (i.e., two second devices) is used as an example. Coarse classification and fine classification are treated as hierarchical semantic tasks for image semantic broadcasting to two receivers. At the transmitting end, multi-level semantic decomposition can be performed followed by superimposed coding and modulation. At the receiving end, various receivers can demodulate the signal as needed. First, two independent neural semantic encoders are used to map the common observable source into a basic coded feature vector (EFV) (for receivers with poor channel conditions) and an enhanced EFV (for receivers with good channel conditions), which are usually correlated. Superimposed coding, as a classic capacity implementation scheme for degraded broadcast channels, is suitable for independent data streams. At the transmitting end, a learning-based LMMSE decorrecorder module is proposed. This decorrecorder aims to extract residual information uncorrelated with the basic EFV from the enhanced EFV, called the layer-by-layer refinement vector of the basic EFV. Then, through a superimposed modulation strategy, the basic EFV is associated with the inner layer, and the layer-by-layer refinement vector is associated with the outer layer. Therefore, receivers with poor channel conditions can decode the inner layer semantic information to obtain the coarse classification information of the image, essentially recovering the basic semantic information. Receivers with good channel conditions can further decode the outer semantic information to obtain fine classification information of the image, thereby restoring the enhanced semantic information level.

[0123] In some examples, as shown in Figure 2B, the overall framework of the DeepSCM scheme on a Gaussian degraded broadcast channel with two receivers (i.e., two second devices) is illustrated. It is assumed that the channel conditions of receiver 2 are better than those of receiver 1, such as the channel SNR of receiver 2 being higher than that of receiver 1. The data to be transmitted is image data, i.e., the observable source X (the bird image in Figure 2B). Two levels of semantic information are associated with this observable source X: coarse-grained semantic information S1 and fine-grained semantic information S2. These form a Markov chain S1→S2→X. Receiver 1 has relatively lower requirements for the observable source X and requires coarse-grained semantic information S1, which is then recovered and denoted as... and Meanwhile, due to the superior channel conditions and larger channel capacity of receiver 2, the requirements for observable sources are higher, and fine-grained semantic information S2 is needed. Their recovery is denoted as follows: and

[0124] In some examples, the coarse-grained semantic information S1∈{1,2,…,L1} above represents a coarse image classification label with class L1. Similarly, the fine-grained semantic information S2∈{1,2,…,L2} represents a fine-grained image classification label with class L2, which is more refined than class L1. This hierarchical structure of observable sources and semantics can well simulate real-world scenarios. For example, coarse-grained semantic information S1 can represent general categories of animals, such as birds and bears, while fine-grained semantic information S2 can represent specific species within these categories, such as sparrows, cardinals, black bears, and brown bears.

[0125] In some examples, in addition to image classification, the hierarchical semantic information model based on Markov chains described above is also applicable to other tasks, such as text classification, speech classification, and so on.

[0126] In some examples, the transmitter (i.e., the first device) extracts semantic features in a hierarchical manner, corresponding to the hierarchical sources used by the two receivers (i.e., the two second devices). At the transmitter, two Joint Coding-Modulation (JCM) modules are responsible for generating two constellation sequences in a superimposed structure. The first JCM module generates the inner constellation sequence Y1 for both receivers, carrying first-level semantic features for recovering the observable source X and coarse-grained semantic information S1. The second JCM module generates the outer constellation sequence Y2 for the receivers. The outer constellation sequence carries additional semantic features, which, together with Y1, recover the observable source X and fine-grained semantic information S2.

[0127] For example, the first JCM module can consist of a basic semantic encoder and a modulator. The basic semantic encoder extracts and encodes basic semantic features U1 from an observable source X. Then, a probabilistic modulator with parameter ρ1 first learns a transition probability p(Y1|U1,ρ1), and then randomly samples a sequence based on this transition probability to generate Y1 from U1. The second JCM module can consist of an enhanced semantic encoder, an LMMSE decorrelation unit, and a modulator. The enhanced semantic encoder extracts and encodes enhanced semantic features U2 from the observable source X, which contains the semantic features of X and S2. Considering the redundancy of hierarchical semantics in U2, the LMMSE decorrelation unit projects U2 onto the space of U1 to obtain a refined feature vector R of U1, making R uncorrelated with U1. Then, in the same way as generating Y1, a probabilistic modulator with parameter ρ2 learns a transition probability p(Y2|R,ρ2), and then randomly samples to obtain Y2 based on R.

[0128] In some examples, for the modulation process, taking Quadrature Amplitude Modulation (QAM) as an example, digital transmission is achieved by superimposing Y1 and Y2. Each element in Y1 obtains its value from the M1-QAM constellation, and each element in Y2 obtains its value from the M2-QAM constellation, where M1 and M2 satisfy M1×M2=M. The two parameters M1 and M2 specify the number of constellation points in the QAM modulation scheme used by Y1 and Y2, respectively. For example, if M1=4 and M2=4, then Y1 and Y2 respectively use 4-QAM. Figures 2C and 2D show examples of overlapping inner and outer constellation sequences. Figure 2C shows a super constellation formed by superimposing two 4-QAM constellations (the inner constellation sequence is 4-QAM, and the outer constellation sequence is 4-QAM), denoted as a 4-QAM×4-QAM super constellation. Figure 2D shows a super constellation formed by superimposing constellations 4-QAM (inner constellation sequence is 4-QAM) and 16-QAM (outer constellation sequence is 16-QAM), denoted as 4-QAM×16-QAM super constellation.

[0129] In some examples, the neural network architecture of the first JCM module can be designed as shown in Figure 2E. The basic semantic encoder consists of multiple ResNet blocks that map an image X of dimension W×H×C to a basic EFV U1 of dimension 2n, where W, H, and C represent the width, height, and channels of the image, respectively. U1 is then modulated to obtain Y1. To avoid the inherent nondifferentiability problem of digital modulation, the modulation process is learned as a probabilistic model. A multilayer perceptron (MLP) with a PReLu activation function and a normalization layer outputs the transition probability, denoted as p(Y1|U1,ρ1), which is the distribution of the transition probability of Y1 given U1 and parameter ρ1. Since Y1=(Y 11 ,Y 12 , ..., Y 1n ) with M n If the discrete distribution has one possible value, then the transition probability has M. 1n The discrete probability distribution of each category is used. To simplify the learning process, each element of Y1 is modeled as conditionally independent, reducing the total number of probability categories to be learned to n·M1. That is, for each element of Y1, the MLP outputs M1 probabilities. Based on this probability distribution, the constellation symbol generator uses the Gumbel-Softmax method to sample constellation symbols for that element. As a differentiable sampling technique, the Gumbel-Softmax method is used to reparameterize discrete variables Y1 as independent random variables and deterministic functions of transition probabilities, thus enabling backpropagation through a single sample of Y1.

[0130] In some examples, the neural network architecture of the second JCM module can use a similar design to that of the first JCM block, the difference being that the second JCM module may have an LMMSE decorrelationer to remove redundancy.

[0131] In some embodiments, in addition to M-QAM digital modulation, other modulation schemes may be used, such as M-phase shift keying (PSK) modulation.

[0132] In some embodiments, the DeepSCM framework proposed in this disclosure can be applied not only to dual receivers (i.e., two second devices), but also to cases with three or more receivers, such as K receivers, where K>2. Consider an observable source X with semantic information S1, S2, ..., S... K This forms a Markov chain S1→S2→...→S K →X. Receiver i, for i∈[K], needs an observable source X and semantic information S. i Similar to the dual-receiver case, there are K JCM modules at the transmitter (i.e., the first device). The first JCM module can consist of a basic semantic encoder and a modulator; the second JCM module can consist of an enhanced semantic encoder, an LMMSE decorreductor, and a modulator, and so on. The Kth JCM module can consist of an enhanced semantic encoder, an LMMSE decorreductor, and a modulator. The LMMSE decorreductor in the K JCM modules can... K with U K-1 At least one of the elements in U1 is decorrelated to reduce redundancy. The resulting constellation sequences Y1 to Y2 are processed using K JCM modules. K Digital transmission is achieved through overlay.

[0133] In step S203, the first device sends a modulation signal to multiple second devices.

[0134] In some embodiments, the first device may broadcast modulated signals to a plurality of second devices.

[0135] In some embodiments, the second device may receive a modulated signal sent by the first device.

[0136] In step S204, the second device determines the semantic information of the layer corresponding to its channel conditions based on the received modulation signal.

[0137] In some embodiments, the second device may determine a received sequence corresponding to its channel conditions based on the modulation signal; and then determine semantic information of the level corresponding to its channel conditions based on the received sequence.

[0138] In some examples, the second device can demodulate the received sequence to obtain the corresponding feature vector. For instance, if the first device at the transmitting end uses M-QAM modulation, the second device can demodulate it using M-QAM demodulation. Then, the second device decodes the feature vector to obtain semantic information corresponding to its channel conditions. For example, the second device at the receiving end may have a neural decoder deployed, which can then be used to obtain semantic information corresponding to its channel conditions.

[0139] In some embodiments, the data to be transmitted may be image data, and the second device may also decode based on feature vectors to obtain an image corresponding to its channel conditions.

[0140] In some examples, taking dual receivers (i.e., two second devices) as an example, as shown in Figure 2B, the channel conditions of receiver 2 are better than those of receiver 1. Receiver 1 determines the receiving sequence Z1 corresponding to its channel conditions based on the modulation signal, and receiver 2 determines the receiving sequence Z2 corresponding to its channel conditions based on the modulation signal. Receiver 1 demodulates the receiving sequence Z1 to obtain a basic semantic feature vector, and then decodes the basic semantic feature vector through a neural decoder to recover the observable source image at the first resolution (i.e., and coarse-grained semantic information Receiver 2 demodulates the received sequence Z2 to obtain an enhanced semantic feature vector (basic semantic feature vector + refined feature vector), and then decodes this enhanced semantic feature vector using a neural decoder to recover the observable source image at the second resolution (i.e., and fine-grained semantic information The second resolution is higher than the first resolution. For example, for each receiving end's second device, a neural decoder is deployed, respectively from Z... i In (i∈1,2), recover the observable source image and semantic information S corresponding to its channel conditions. i . Where ψi is the decoder parameter used for image classification in receiver i, and ηi is the decoder parameter used for image restoration in receiver i.

[0141] This disclosure proposes a communication scheme for semantic communication. A first device, acting as a transmitter, can first obtain semantic information at different levels from the semantic recognition results of the data to be transmitted. Then, it can superimpose and encode the semantic information at different levels to obtain a modulated signal. Finally, it can send the modulated signal to multiple second devices, acting as receivers, so that the second devices can obtain the corresponding level of semantic information according to their own channel conditions, thus meeting the communication needs of different receivers under different channel conditions.

[0142] Figure 3 is an interactive schematic diagram of a semantic communication method according to an embodiment of the present disclosure. As shown in Figure 3, the method includes:

[0143] In step S301, the first device sends a modulation signal to multiple second devices.

[0144] In some embodiments, the modulated signal is obtained by superimposing and encoding the semantic information of different levels of the data to be transmitted.

[0145] In some embodiments, the second device determines semantic information corresponding to the level of its channel conditions based on the received modulated signal.

[0146] Optionally, the alternative implementations of step S301 can be found in Figure 2A, and will not be repeated here.

[0147] In some embodiments, the steps and their optional implementations in other embodiments described before or after this embodiment, as well as other related parts in the specification, can be referred to, and will not be repeated here.

[0148] This disclosure proposes a communication scheme for semantic communication. A first device, acting as a transmitter, can first obtain semantic information at different levels from the semantic recognition results of the data to be transmitted. Then, it can superimpose and encode the semantic information at different levels to obtain a modulated signal. Finally, it can send the modulated signal to multiple second devices, acting as receivers, so that the second devices can obtain the corresponding level of semantic information according to their own channel conditions, thus meeting the communication needs of different receivers under different channel conditions.

[0149] The following are some exemplary specific solutions proposed in the embodiments of this disclosure:

[0150] The purpose of this embodiment is to propose an effective digital semantic communication framework that can utilize the hierarchical structure of semantic information and adapt to different receivers under different channel conditions.

[0151] In some embodiments, this invention presents a novel deep learning-based DeepSCM (DeepSCM) scheme for hierarchical semantic communication over Gaussian degenerate broadcast channels. The main idea is to utilize neural networks to extract and encode different levels of semantic information from a common source, and then combine them using superposition coding before broadcasting. Therefore, each receiver can decode its desired level of semantic information from the superposition structure symbols according to its own channel conditions.

[0152] In some embodiments, hierarchical semantic information is first extracted and encoded into basic feature vectors and enhanced feature vectors. Then, a linear minimum mean square error (LMMSE) decorrector is used to obtain refinement from the enhanced features that are not related to the basic features. Finally, the basic features and their refinements are superimposed after probability modulation to achieve broadcasting.

[0153] The following explanation uses dual-receiver image semantic broadcasting as an example. Coarse and fine classification are treated as hierarchical semantic tasks for dual-receiver image semantic broadcasting. At the transmitting end, multi-level semantic decomposition can be performed followed by superimposed coding and modulation. At the receiving end, multiple receivers can be used for demodulation as needed.

[0154] First, two independent neural semantic encoders are used to map the common observable source into a basic encoded feature vector (EFV) (for poor receivers) and an enhanced EFV (for good receivers), respectively. These two EFVs are typically correlated. Superposition coding, a classic capacity implementation scheme for degraded broadcast channels, is suitable for independent data streams. Therefore, a novel learning-based linear minimum mean square error (LMMSE) decorrecorder module is introduced into the transmitter design. This decorrecorder aims to extract residual information uncorrelated with the basic EFV from the enhanced EFV, referred to as the successively refined vector of the basic EFV. Then, a superposition modulation strategy is designed to associate the basic EFV with the inner layer and the successively refined vector with the outer layer. Thus, poor receivers can decode the inner layer to recover the semantic information of the first layer. Simultaneously, good receivers can further decode the outer layer to recover the enhanced semantic information level. Furthermore, to ensure the convergence and performance of the training process, a three-stage DeepSCM training strategy is designed based on the superposition structure.

[0155] Figure 2B illustrates the overall framework of the DeepSCM scheme on a two-user Gaussian degraded broadcast channel. Without loss of generality, it is assumed that the channel signal-to-noise ratio of receiver 2 is higher than that of receiver 1.

[0156] There is an observable source X associated with implicit hierarchical semantic information, namely coarse-grained semantic information S1 and fine-grained semantic information S2. Naturally, they form a Markov chain S1→S2→X.

[0157] Receiver 1 needs an observable source X and coarse-grained semantic information S1, which are then recovered and denoted as follows: and

[0158] Meanwhile, due to the larger channel capacity of receiver 2, the requirements for observable sources X and fine-grained semantic information S2 are higher, and their recovery is denoted as follows: and

[0159] Specifically, in this embodiment, we focus on image semantic communication for image restoration and classification. That is, observable source X∈R d Let be image data, where d is the dimension of the image and R represents a real number. Coarse-grained semantic information S1∈{1,2,…,L1} represents coarse image classification labels with class L1. Similarly, fine-grained semantic information S2∈{1,2,…,L2} represents fine-grained image classification labels with class L2, where L2>L1. This hierarchical structure of observable sources and semantics can well simulate real-world scenarios. For example, semantic information S1 can represent general categories of animals, such as birds and bears, while semantic information S2 can represent specific species within these categories, such as sparrows, cardinals, black bears, and brown bears.

[0160] Besides image classification, the hierarchical semantic information model based on Markov chains described above is also applicable to other tasks.

[0161] The transmitter extracts semantic features in a hierarchical manner, corresponding to the hierarchical sources for the two receivers. At the transmitter, two JCM blocks are responsible for generating two constellation sequences in the overlay structure. The first JCM block generates the inner constellation sequence Y1 = (Y11, Y12, ..., Y1n) ∈ Cn for both receivers, carrying first-level semantic features for recovering the observable source X and coarse-grained semantic information S1. Here, n represents the number of channels used. The second JCM block generates the outer constellation sequence Y2 = (Y21, Y22, ..., Y2n) ∈ Cn for receiver 2. The outer constellation sequence carries additional semantic features, which, together with Y1, recover the observable source X and fine-grained semantic information S2.

[0162] The first JCM module consists of a basic semantic encoder and a modulator. The parameterized basic semantic encoder fθ1 extracts and encodes basic semantic features from X, outputting a 2n-dimensional real-valued basic feature EFV U1∈R2n. U1=fθ1(X).

[0163] Then, a probability modulator with parameter ρ1 first learns a transition probability p(Y1|U1,ρ1), and then randomly samples a sequence based on this transition probability to generate Y1 from U1. The second JCM module consists of an enhanced semantic encoder, an LMMSE decorrelation unit, and a modulator. The enhanced semantic encoder fθ2 with parameter θ2 generates an enhanced EFV U2∈R2n from X, which contains the semantic features of X and S2. U2=fθ2(X).

[0164] Considering the redundancy of hierarchical semantics in U2, the LMMSE decorrelation projectes U2 onto the space of U1 to obtain a successively refined vector R of U1, making R uncorrelated with U1. Then, in the same way as generating Y1, a probability modulator with parameter ρ2 learns the transition probability p(Y2|R,ρ2), and then randomly samples Y2 based on R.

[0165] This embodiment uses M-QAM modulation as an example, achieving digital transmission through the superposition of Y1 and Y2. Each element Y1 obtains a value from the M1-QAM constellation C1, and each element in Y2 obtains a value from the M2-QAM constellation C2, where M1 and M2 satisfy M1×M2=M. Receiver 1 receives the sequence Z1=Y+∈1, where ∈1~CN(0,σ12In×n). Similarly, receiver 2 receives a sequence Z2=Y+∈2, where ∈2~CN(0,σ22In×n), and σ1>σ2. For i in {1,2}, the channel condition of receiver i is characterized by the channel signal-to-noise ratio.

[0166] Figures 2C and 2D show examples of overlapping inner and outer constellations. In Figure 2C, a superconstellation formed by the superposition of two 4QAM constellations is denoted as a 4QAM×4QAM superconstellation. In Figure 2D, the inner constellation is 4QAM and the outer constellation is 16QAM, denoted as a 4QAM×16QAM superconstellation. Furthermore, M-QAM digital modulation can be easily extended to other modulation schemes, such as M-PSK.

[0167] At each receiver, two neural decoders are deployed in parallel, one from Z... i Recover X and S from (i∈1,2) i . η i (Z i ), where ψ i These are the decoder parameters for receiver i used for classification, η i These are the decoder parameters for receiver i used for image restoration.

[0168] The DeepSCM framework above can be easily extended to K (K>2) users.

[0169] More specifically, consider an observable source X with implicit hierarchical semantic information S1, S2, ..., S... K This forms a Markov chain S1→S2→...→S K →X.

[0170] For receiver i ∈ [K], source data X and semantic information S are required. i Similar to the dual-user case, there are K JCM blocks at the transmitter.

[0171] Joint coding and modulation:

[0172] For broadcast settings, two JCM blocks are used to enable digital modulation of both EFVs.

[0173] For example, the neural network architecture of the first JCM block can use the design shown in Figure 2E. The basic semantic encoder consists of multiple ResNet blocks that map an image X of dimension W×H×C to a basic EFV U1 of dimension 2n, where W, H, and C represent the width, height, and channels of the image, respectively, resulting in d = W×H×C.

[0174] Then, U1 is modulated to obtain Y1. To avoid the inherent non-differentiability problem of digital modulation, the modulation process is learned as a probabilistic model. A multilayer perceptron (MLP) with a PReLu(·) activation function and a normalization layer outputs the transition probability, denoted as p(Y1|Y1,ρ1). Since Y1 = (Y11,Y12,…,Y1n) has one possible value as a discrete distribution with Mn, the transition probability is a discrete probability distribution with M1n categories. To simplify the learning process, each element of Y1 is modeled as conditionally independent, reducing the total number of probability categories to be learned to n·M1. That is, for each element of Y1, the MLP outputs M1 probabilities. Based on this probability distribution, the constellation symbol generator uses the Gumbel-Softmax method to sample constellation symbols for that element. As a differentiable sampling technique, the Gumbel-Softmax method is used to reparameterize the discrete variable Y1 as an independent random variable and a deterministic function of the transition probability, thereby enabling backpropagation through a sample of Y1.

[0175] This disclosure proposes that semantic features for different receivers are encoded into a basic EFV and its successively refined vectors, which are then associated with different layers of a superconstellation. To minimize redundancy in broadcasting, an LMMSE decorrelation is used to ensure that the two vectors are virtually uncorrelated with each other. This superposition coding structure can accommodate the communication needs of different receivers under different channel conditions.

[0176] This disclosure not only provides an efficient method for semantic broadcasting, but also demonstrates a promising approach that combines theoretical coding schemes with neural network-based coding methods.

[0177] This disclosure also proposes an apparatus (also referred to as a communication device, etc.) for implementing any of the above methods. For example, an apparatus is proposed that includes units or modules for implementing the steps performed by the terminal in any of the above methods. Furthermore, another apparatus is proposed that includes units or modules for implementing the steps performed by a network device (e.g., an access network device, a core network functional node, a core network device, etc.) in any of the above methods.

[0178] It should be understood that the division of units or modules in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units or modules in the device can be implemented by a processor calling software: for example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of the units or modules in the above device. The processor can be, for example, a general-purpose processor, such as a Central Processing Unit (CPU) or a microprocessor, and the memory can be internal or external to the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits. The functionality of some or all of the units or modules can be achieved through the design of these hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC). The functionality of some or all of the units or modules is achieved through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a programmable logic device (PLD). Taking a field-programmable gate array (FPGA) as an example, it can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby achieving the functionality of some or all of the units or modules. All units or modules of the above device can be implemented entirely through processor-called software, entirely through hardware circuits, or partially through processor-called software with the remaining parts implemented through hardware circuits.

[0179] In this embodiment, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a Central Processing Unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the aforementioned hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a Neural Network Processing Unit (NPU), a Tensor Processing Unit (TPU), or a Deep Learning Processing Unit (DPU).

[0180] Figure 4A is a schematic diagram of the structure of a first device according to an embodiment of this disclosure. The first device is used to perform any of the above methods. In some embodiments, as shown in Figure 4A, the first device may include at least one of a transceiver module 5101, a processing module 5102, etc. In some embodiments, the processing module 5102 is configured to obtain semantic information at different levels from the semantic recognition result of the data to be transmitted; to superimpose and encode the semantic information at different levels to obtain a modulated signal; the transceiver module 5101 is configured to transmit the modulated signal to a plurality of second devices, the modulated signal being used to determine semantic information at a level corresponding to the channel conditions of the second devices. Optionally, the transceiver module is used to perform at least one of the communication steps such as transmission and / or reception performed by the first device in any of the above methods, which will not be described in detail here. Optionally, the processing module is used to perform at least one of the other steps performed by the first device in any of the above methods, which will not be described in detail here.

[0181] Figure 4B is a schematic diagram of the structure of the second device proposed in an embodiment of this disclosure. The second device is used to perform any of the above methods. In some embodiments, as shown in Figure 4B, the second device may include at least one of a transceiver module 5201, a processing module 5202, etc. In some embodiments, the transceiver module is configured to receive a modulation signal sent by the first device, the modulation signal being obtained by superimposing and coding modulation based on semantic information of different levels of data to be transmitted; the processing module 5202 is further configured to determine semantic information corresponding to the channel conditions of the second device based on the modulation signal. Optionally, the transceiver module is used to perform the communication steps such as sending and / or receiving performed by the second device in any of the above methods, which will not be described in detail here. Optionally, the processing module is used to perform other steps performed by the second device in any of the above methods, which will not be described in detail here.

[0182] In some embodiments, the transceiver module may include a transmitting module and / or a receiving module, which may be separate or integrated. Optionally, the transceiver module may be interchangeable with a transceiver.

[0183] In some embodiments, the processing module may be a single module or may include multiple sub-modules. Optionally, the multiple sub-modules may each perform all or part of the steps required by the processing module.

[0184] In some embodiments, the processing module can be interchanged with the processor, and the transceiver module can be interchanged with the transceiver.

[0185] Figure 5A is a schematic diagram of the structure of the communication device 6100 proposed in an embodiment of this disclosure. The communication device 6100 can be the first device or the second device described above, specifically it can be a network device (e.g., an access network device), a terminal (e.g., a user equipment), a chip, chip system, or processor that supports the network device in implementing any of the above methods, or a chip, chip system, or processor that supports the terminal in implementing any of the above methods. The communication device 6100 can be used to implement the methods described in the above method embodiments; for details, please refer to the descriptions in the above method embodiments.

[0186] As shown in Figure 5A, the communication device 6100 is used to execute any of the above methods. In some embodiments, the communication device 6100 includes one or more processors 6101. The processor 6101 may be a general-purpose processor or a special-purpose processor, such as a baseband processor or a central processing unit. The baseband processor may be used to process communication protocols and communication data, and the central processing unit may be used to control communication devices (e.g., base stations, baseband chips, terminal devices, terminal device chips, DUs or CUs, etc.), execute programs, and process program data. Optionally, the communication device 6100 is used to execute any of the above methods. Optionally, one or more processors 6101 are used to invoke instructions to cause the communication device 6100 to execute any of the above methods.

[0187] In some embodiments, the communication device 6100 further includes one or more transceivers 6102. When the communication device 6100 includes one or more transceivers 6102, the transceivers 6102 perform the communication steps such as sending and / or receiving in the above method, and the processor 6101 performs other processing steps. In optional embodiments, the transceiver may include a receiver and / or a transmitter, which may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, transceiver circuit, interface circuit, interface, etc., can be used interchangeably; the terms transmitter, sending unit, transmitter, sending circuit, etc., can be used interchangeably; the terms receiver, receiving unit, receiver, receiving circuit, etc., can be used interchangeably.

[0188] In some embodiments, the communication device 6100 further includes one or more memories 6103 for storing data and / or instructions. Optionally, one or more processors 6101 are used to invoke instructions stored in the memory 6103 to cause the communication device 6100 to perform any of the above methods. Optionally, all or part of the memory 6103 may also be located outside the communication device 6100. In an optional embodiment, the communication device 6100 may include one or more interface circuits 6104. Optionally, the interface circuit 6104 is connected to the memory 6103 and can be used to receive data and / or instructions from the memory 6103 or other devices, and can be used to send data and / or instructions to the memory 6103 or other devices. For example, the interface circuit 6104 can read data and / or instructions stored in the memory 6103 and send the data and / or instructions to the processor 6101.

[0189] The communication device 6100 described in the above embodiments may be a network device or a terminal, but the scope of the communication device 6100 described in this disclosure is not limited thereto, and the structure of the communication device 6100 may not be limited by FIG5A. The communication device may be a standalone device or may be part of a larger device. For example, the communication device may be: (1) a standalone integrated circuit IC, or chip, or chip system or subsystem; (2) a collection of one or more ICs, optionally, the IC collection may also include storage components for storing data, programs and / or instructions; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, terminal device, smart terminal device, cellular phone, wireless device, handheld device, mobile unit, vehicle device, network device, cloud device, artificial intelligence device, etc.; (6) others, etc.

[0190] Figure 5B is a schematic diagram of the structure of the chip 6200 proposed in an embodiment of this disclosure. For cases where the communication device 6100 can be a chip or a chip system, the schematic diagram of the chip 6200 shown in Figure 5B can be referenced, but the invention is not limited thereto.

[0191] Chip 6200 includes one or more processors 6201. Chip 6200 is used to perform any of the methods described above.

[0192] In some embodiments, chip 6200 further includes one or more interface circuits 6202. Optionally, terms such as interface circuit, interface, and transceiver pin can be used interchangeably. In some embodiments, chip 6200 further includes one or more memories 6203 for storing data and / or instructions. Optionally, all or part of the memories 6203 may be located outside of chip 6200. Optionally, interface circuit 6202 is connected to memory 6203, and interface circuit 6202 can be used to receive data and / or instructions from memory 6203 or other devices, and interface circuit 6202 can be used to send data and / or instructions to memory 6203 or other devices. For example, interface circuit 6202 can read data and / or instructions stored in memory 6203 and send the data and / or instructions to processor 6201.

[0193] In some embodiments, the interface circuit 6202 performs communication steps such as sending and / or receiving in the above-described method. For example, the interface circuit 6202 performing communication steps such as sending and / or receiving in the above-described method refers to the interface circuit 6202 performing data and / or instruction interaction between the processor 6201, the chip 6200, the memory 6203, or the transceiver device. In some embodiments, the processor 6201 performs other processing steps.

[0194] The modules and / or devices described in the various embodiments, such as virtual devices, physical devices, and chips, can be combined or separated arbitrarily as needed. Optionally, some or all steps can also be performed collaboratively by multiple modules and / or devices, which is not limited here.

[0195] This disclosure also proposes a storage medium storing instructions that, when executed on a communication device, cause the communication device to perform any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but not limited thereto; it may also be a storage medium readable by other devices. Optionally, the storage medium may be a non-transitory storage medium, but not limited thereto; it may also be a temporary storage medium.

[0196] This disclosure also proposes a program product, including a program and / or instructions, which, when executed by a communication device, cause the communication device to perform any of the above methods. Optionally, the program product is a computer program product. Optionally, the program product is stored on the storage medium.

[0197] This disclosure also proposes a computer program that, when run on a computer, causes the computer to perform any of the above methods.

Claims

1. A semantic communication method, characterized in that, Performed by the first device, the method includes: Obtain semantic information at different levels from the semantic recognition results of the data to be sent; The semantic information at different levels is superimposed, encoded, and modulated to obtain a modulated signal; The modulated signal is sent to a plurality of second devices, the modulated signal being used to determine semantic information at a level corresponding to the channel conditions of the second devices.

2. The method according to claim 1, characterized in that, The modulation signal is obtained by superimposing and encoding the semantic information at different levels, including: The semantic information at different levels is encoded into a first feature vector and at least one second feature vector. The first feature vector is used to determine the basic semantic features of the data to be sent, and the at least one second feature vector is used to determine the enhanced semantic features that are refined layer by layer from the basic semantic features. The modulated signal is obtained by superimposing and modulating the first feature vector and the at least one second feature vector.

3. The method according to claim 2, characterized in that, The modulated signal is obtained by superimposing and modulating the first feature vector and the at least one second feature vector, including: A first constellation sequence is generated based on the first feature vector, and at least one second constellation sequence is generated based on the at least one second feature vector; The modulation signal is generated by superimposing the first constellation sequence and the at least one second constellation sequence.

4. The method according to claim 3, characterized in that, Generating at least one second constellation sequence based on the at least one second feature vector includes: The at least one second feature vector is subjected to decorrelation processing based on the first feature vector; The at least one second constellation sequence is generated based on at least one second feature vector after decorrelation processing.

5. The method according to claim 4, characterized in that, The second feature vector is decorrelated based on the first feature vector, including: The linear minimum mean square error (LMMSE) decorrecorder is used to extract residual information that is not related to the first feature vector from the second feature vector. The second feature vector after decorrelation is determined based on the residual information.

6. The method according to claim 3, characterized in that, Generate a first constellation sequence based on the first feature vector, including: The first constellation sequence is generated based on the first feature vector through random sampling with the first transition probability.

7. The method according to any one of claims 3 to 6, characterized in that, The second constellation sequence is generated based on the second feature vector, including: The second constellation sequence is generated based on the second feature vector through random sampling with the second transition probability.

8. The method according to any one of claims 1 to 7, characterized in that, The data to be transmitted includes image data, and the modulation signal is also used to determine the image corresponding to the channel conditions of the second device.

9. The method according to claims 1 to 8, characterized in that, The channel conditions are determined by at least one of the following: Signal-to-noise ratio; Signal strength; Bit error rate.

10. A semantic communication method, characterized in that, Performed by the second device, the method includes: The device receives a modulation signal sent by a first device, the modulation signal being obtained by superimposing and encoding modulation based on the semantic information of different levels of the data to be transmitted; Based on the modulation signal, semantic information corresponding to the channel conditions of the second device is determined.

11. The method according to claim 10, characterized in that, Based on the modulation signal, semantic information corresponding to the channel conditions of the second device is determined, including: The receiving sequence corresponding to the channel conditions is determined based on the modulation signal; The semantic information corresponding to the channel condition level is determined based on the received sequence.

12. The method according to claim 11, characterized in that, Determining the semantic information corresponding to the channel condition level based on the received sequence includes: The feature vector is obtained by demodulating the received sequence. The semantic information corresponding to the channel condition level is obtained by decoding based on the feature vector.

13. The method according to claim 12, characterized in that, The data to be sent is image data, and the method further includes: The image corresponding to the channel conditions is obtained by decoding based on the feature vector.

14. The method according to claims 10 to 13, characterized in that, The channel conditions are determined by at least one of the following: Signal-to-noise ratio; Signal strength; Bit error rate.

15. A communication system, characterized in that, The device includes a first device and a second device, the first device being configured to implement the method of any one of claims 1 to 9, and the second device being configured to implement the method of any one of claims 10 to 14.

16. A communication device, characterized in that, The communication device is used to perform the method according to any one of claims 1 to 9 or 10 to 14.

17. A storage medium storing instructions, characterized in that, When the instructions are executed on a communication device, the communication device performs the method of any one of claims 1 to 9 or 10 to 14.

18. A program product comprising at least one of a program and instructions, characterized in that, When at least one of the programs and instructions is executed by a communication device, it implements the method of any one of claims 1 to 9 or 10 to 14.

Images