Semantic communication methods and systems, electronic devices, computer-readable storage media
By using pre-trained encoders and decoders and importance templates in semantic communication, semantic features are filtered and recovered, solving the problem of noise impact in traditional communication and achieving efficient semantic information transmission under low signal-to-noise ratio.
Patent Information
- Application Number
- CN202510147657.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-10
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2045-02-10
AI Technical Summary
In traditional communication, noise and interference severely affect the accurate transmission of information, leading to information distortion and loss. Existing semantic communication methods increase overhead and have high requirements for channel conditions when reducing the amount of data transmitted.
A pre-trained semantic encoder and decoder are used, and semantic features are filtered by an importance template. Only the parts with higher importance are transmitted. The receiving end recovers the discarded parts by using the same template, reducing the need for additional index transmission.
Accurate transmission of semantic information was achieved at a low signal-to-noise ratio, reducing the overhead of transmission index and improving the noise resistance of communication.
Smart Images

Figure CN120034292B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the fields of semantic communication and artificial intelligence. Specifically, this disclosure relates to a semantic communication method and system, an electronic device, and a computer-readable storage medium. Background Technology
[0002] Traditional communication emphasizes the accurate transmission of bits. Noise and interference can seriously affect the accurate transmission of information, leading to information distortion and loss.
[0003] Semantic communication is a communication method that emphasizes the meaning of information and the accurate transmission of semantics. Compared with traditional communication, it pays more attention to semantic understanding and expression, rather than relying solely on the accurate transmission of bits.
[0004] The core of semantic communication lies in transforming natural language or other forms of information into machine-understandable semantic representations, thereby more accurately capturing the meaning and purpose of the information. In semantic communication, the transmitter sends content with semantic information, while the receiver interprets and understands the meaning of this semantic information. By focusing on the semantic level of information during communication, more accurate and targeted transmission and understanding can be achieved. Summary of the Invention
[0005] This disclosure provides a semantic communication method and system, an electronic device, and a computer-readable storage medium.
[0006] According to a first aspect of this disclosure, a semantic communication method is provided for a semantic communication system, the semantic communication system including a transmitter, a receiver, and a communication channel connecting the transmitter and the receiver, the method comprising:
[0007] The source data transmitted by the transmitter is input into a pre-trained semantic encoder for semantic encoding to obtain the source semantic features of the source data;
[0008] The semantic features of the information source are filtered by a pre-acquired importance template, and the filtered semantic features of the information source are converted into a bit stream.
[0009] The bit stream is transmitted to the receiving end through the communication channel;
[0010] The importance template is used to restore the bitstream received by the receiving end into restored semantic features, and the restored semantic features are input into a pre-trained semantic decoder for semantic decoding to obtain the restored information.
[0011] The importance template is obtained by training on the target dataset; the semantic encoder and the semantic decoder are obtained by training based on the importance template.
[0012] According to a second aspect of this disclosure, a semantic communication system is provided, the system comprising:
[0013] The transmitting end is used to transmit source data, input the source data into a pre-trained semantic encoder for semantic encoding, and obtain the source semantic features of the source data; the source semantic features are filtered by a pre-obtained importance template, and the filtered source semantic features are converted into a bit stream;
[0014] A communication channel for transmitting the bit stream from the transmitting end to the receiving end;
[0015] The receiving end is used to receive the bit stream, restore the bit stream to restored semantic features through the importance template, and input the restored semantic features into a pre-trained semantic decoder for semantic decoding to obtain the restored information;
[0016] The importance template is obtained by training on the target dataset; the semantic encoder and the semantic decoder are obtained by training based on the importance template.
[0017] According to a third aspect of this disclosure, an electronic device is provided, the electronic device comprising:
[0018] At least one processor; and
[0019] A memory communicatively connected to at least one of the aforementioned processors; wherein,
[0020] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable at least one processor to perform the semantic communication method.
[0021] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the semantic communication method described above.
[0022] According to a fifth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the semantic communication method described above.
[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0024] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:
[0025] Figure 1This is a flowchart illustrating a semantic communication method provided in an embodiment of this disclosure;
[0026] Figure 2 This is a flowchart illustrating some steps of another semantic communication method provided in this embodiment of the present disclosure;
[0027] Figure 3 This is a flowchart illustrating some steps of another semantic communication method provided in this embodiment of the present disclosure;
[0028] Figure 4 This is a flowchart illustrating some steps of another semantic communication method provided in this embodiment of the present disclosure;
[0029] Figure 5 This is a flowchart illustrating some steps of another semantic communication method provided in this embodiment of the present disclosure;
[0030] Figure 6 This is a schematic diagram of a specific embodiment of a semantic communication method provided in this disclosure;
[0031] Figure 7 This is a schematic diagram of the structure of a semantic communication device provided in an embodiment of this disclosure;
[0032] Figure 8 This is a block diagram of an electronic device used to implement the semantic communication method of the embodiments of this disclosure. Detailed Implementation
[0033] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0034] The following are the terms that may appear in the embodiments of this disclosure and their corresponding explanations:
[0035] Semantic encoding: Extracting the semantic information contained within business information. Based on different semantic types, it can be further subdivided into text semantic encoding, image semantic encoding, audio semantic encoding, video semantic encoding, point cloud semantic encoding, etc. The possible categories of semantic encoding are related to the possible categories of business information; it can be semantic encoding of all possible types of communication transmission. The generation of semantic encoding is also related to the semantic encoding model, which can employ models from any possible discipline such as artificial intelligence, deep learning, and pattern recognition.
[0036] Transmitter and Receiver: A channel connection exists between the transmitter and receiver. The transmitter (transmitter) has an encoder, and the receiver (receiver) has a decoder. This invention is based on semantic encoding and semantic decoding; therefore, the transmitter of this invention has a semantic encoder, and the receiver of this invention has a semantic decoder. The transmitter and receiver of this invention operate at the semantic layer, while the underlying layer remains the Shannon physical layer.
[0037] Model: The models referred to in this invention include models from disciplines such as machine learning, artificial intelligence, and neural networks. Models used for semantic encoding and decoding of business information are called semantic encoding models and semantic decoding models. Depending on the type of business information, they can be categorized as text models, audio models, image models, video models, point cloud models, one-dimensional waveform models, radar data models, etc. The type of model is related to the type of semantic encoding and the type of business information; it can be a model of all possible types of communication transmission.
[0038] Semantic communication prioritizes the accurate transmission of semantic information over the absolute accuracy of individual bits. In other words, semantic communication allows for some bit errors, as long as the semantic error is minimized. Even if the semantic information is affected by noise during transmission and some bits are corrupted, the trained model can still recover the correct semantic information. Therefore, semantic communication has strong noise resistance and can still transmit information accurately even with significant fluctuations in the signal-to-noise ratio.
[0039] In some related technologies, based on semantic communication, a semantic importance module is used to calculate the semantic importance of semantic features. Then, the parts with lower semantic importance are discarded, and only the parts with higher semantic importance are transmitted, which can further reduce the amount of data transmitted. However, in order to ensure that the receiver accurately recovers the semantic features, the index of the retained part in the original semantic features needs to be transmitted accurately. This not only increases the additional overhead, but also places higher demands on the channel conditions.
[0040] The semantic communication methods and systems, electronic devices, and computer-readable storage media provided in this disclosure are intended to solve at least one of the above-mentioned technical problems of the prior art.
[0041] The semantic communication method provided in this disclosure can be executed by electronic devices such as terminal devices or servers. Terminal devices can be in-vehicle devices, user equipment (UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.
[0042] Figure 1 A flowchart illustrating the semantic communication method provided in an embodiment of this disclosure is shown. Figure 1 As shown in the figure, the semantic communication method provided in this embodiment may include steps S110, S120, S130 and S140.
[0043] In step S110, the source data transmitted by the transmitter is input into a pre-trained semantic encoder for semantic encoding to obtain the source semantic features of the source data;
[0044] In step S120, the semantic features of the information source are filtered using a pre-acquired importance template, and the filtered semantic features of the information source are converted into a bit stream.
[0045] In step S130, the bit stream is transmitted to the receiving end through the communication channel;
[0046] In step S140, the bit stream received by the receiving end is restored to restored semantic features through the importance template, and the restored semantic features are input into the pre-trained semantic decoder for semantic decoding to obtain the restored information;
[0047] The importance template is obtained by training on the target dataset; the semantic encoder and semantic decoder are obtained by training on the importance template.
[0048] For example, the semantic communication method provided in this disclosure is used in a semantic communication system.
[0049] The semantic communication system includes a transmitter, a receiver, and a communication channel connecting the transmitter and the receiver.
[0050] In some possible implementations, in step S110, the source data can be image data, voice data, text data, point cloud data, or a mixture of any of the above.
[0051] In some possible implementations, a semantic encoder is a model used to semantically encode source data. Therefore, inputting source data into a semantic encoder can extract the source semantic features of the source data.
[0052] In some possible implementations, the semantic encoder can specifically be a CNN (Convolutional Neural Network), an RNN (Recurrent Neural Network), or the like.
[0053] This disclosure does not limit the specific type of semantic encoder; any model that can implement semantic encoding is within the protection scope of this disclosure.
[0054] In some possible implementations, in step S120, filtering the source semantic features obtained in step S110 using a pre-obtained importance template can be based on obtaining the importance of all source semantic features using the pre-obtained importance template, and filtering out source semantic features with higher importance based on the importance of the source semantic features.
[0055] Converting the selected new energy semantic features into a bitstream can be done by converting the selected high-importance source semantic features into a bitstream.
[0056] Bitstream refers to a sequence of binary numbers consisting of 0 and 1. Converting source semantic features into bitstream means converting source semantic features into binary data.
[0057] In some possible implementations, in step S130, since the bit stream can be transmitted via a network or other transmission medium, the bit stream can be transmitted via a communication channel.
[0058] Specifically, the transmitter sends a bit stream, and the receiver receives the bit stream sent by the transmitter.
[0059] In some possible implementations, in step S140, the bitstream received by the receiving end is restored to semantic features. The obtained semantic features are then recovered using an importance template to obtain restored semantic features. These restored semantic features are then input into a semantic decoder to obtain recovery information for recovering the source data. The obtained recovery information is the source data received by the receiving end.
[0060] The semantic decoder, as opposed to the semantic encoder, is a model used to semantically decode semantic features. Therefore, inputting the reconstructed semantic features into the semantic decoder can obtain the data corresponding to the semantic features.
[0061] In some possible implementations, the semantic decoder can specifically be a CNN (Convolutional Neural Network), an RNN (Recurrent Neural Network), or the like.
[0062] This disclosure does not limit the specific type of semantic decoder; any model that can implement semantic decoding is within the protection scope of this disclosure.
[0063] In this embodiment, the importance template used is obtained by training on the target dataset. The trained importance template is based on the importance of features in the target data. Since the target dataset and the source data are of the same type and have similar features, the importance template obtained by training on the target data is also applicable to the source data.
[0064] Meanwhile, the semantic encoder and semantic decoder are trained based on the importance template. Therefore, the semantic features acquired by the semantic encoder are consistent with the importance template. After the semantic encoding features are filtered by the importance template, important semantic features can be preserved, semantic feature loss can be minimized, and the similarity between the acquired restored information and the source data can be guaranteed, thereby ensuring the quality of the acquired restored information.
[0065] In some related semantic communication methods, a portion of the semantically encoded content transmitted from the transmitter is discarded based on semantic importance before retransmission. This discarded portion is unknown to the receiver, requiring the transmission of an additional index for correct decoding. Furthermore, this index must be transmitted losslessly; any errors will prevent decoding.
[0066] In the semantic communication method provided in this embodiment, the transmitting end and the receiving end use the same importance template. The transmitting end discards part of the semantically encoded content based on semantic importance and then retransmits it. The receiving end can recover the discarded part based on the importance template, so there is no need for an additional transmission index. This reduces the additional overhead of the transmission index and enables communication at a low signal-to-noise ratio.
[0067] The semantic communication method provided in the embodiments of this disclosure will be described in detail below.
[0068] Figure 2 This diagram illustrates a flowchart of an implementation method that filters source semantic features using an importance template and converts the filtered source semantic features into a bitstream. Figure 2 As shown, it may include steps S210 and S220.
[0069] In step S210, some elements in the semantic features of the information source are discarded according to a preset ratio using an importance template;
[0070] In step S220, the remaining source semantic features are discarded and converted into a bit stream.
[0071] In some possible implementations, in step S210, the size of the importance template is consistent with the size of the semantic features output by the semantic encoder, the number of elements included in the importance template is consistent with the number of elements included in the semantic features output by the semantic feature encoder, and they correspond one-to-one according to their positions.
[0072] Each element of the importance template is a number between 0 and 1. The larger the number, the greater the importance of the semantic feature element corresponding to that element.
[0073] In some possible implementations, the elements at each position of the importance template are sorted, and the positions corresponding to the elements at the top of the sorted sequence are selected as the filtering positions according to a preset ratio. The positions corresponding to the remaining data are selected as non-filtering positions, and the elements at the non-filtering positions of the source semantic features are discarded.
[0074] In some possible implementations, in step S220, the elements at the filtered positions are converted into binary data to convert the remaining source semantic features into a bitstream.
[0075] Figure 3 This diagram illustrates a flowchart of an implementation method for restoring semantic features from a received bitstream using an importance template, as shown below. Figure 3 As shown, it may include steps S310 and S320.
[0076] In step S310, the bit stream received by the receiving end is restored to feature data;
[0077] In step S320, zero-padding is performed on the feature data using the importance template to obtain restored semantic features.
[0078] In some possible implementations, in step S310, the binary data received in the bit stream at the receiving end is restored to other bases to obtain feature data.
[0079] In some possible implementations, in step S320, the elements at each position of the importance template are sorted, and the positions corresponding to the elements at the first preset ratio of the sorted elements are selected as the filtering positions according to a preset ratio. The positions corresponding to the remaining data are selected as the non-filtering positions. The feature data is written into the filtering positions in order, and zero-padding is performed in the non-filtering positions to generate restored semantic features.
[0080] In some possible implementations, the semantic communication method provided in this disclosure embodiment further includes: a training process for a semantic communication model and an importance template.
[0081] Figure 4 The flowchart illustrating the training process of the semantic communication model and importance template is shown, as follows: Figure 4 As shown, the training process of the semantic communication model and importance template may include steps S410, S420, S430, and S440.
[0082] In step S410, a semantic communication model is trained using a training dataset;
[0083] In step S420, the semantic importance module obtained from training is used to calculate the semantic importance of each sample in the target dataset;
[0084] In step S430, an importance template is generated based on the semantic importance of each sample in the target dataset;
[0085] In step S440, the semantic communication model is fine-tuned according to the importance template to obtain the trained semantic encoder and semantic decoder.
[0086] In some possible implementations, in step S410, the semantic communication model includes a semantic encoder, a semantic importance module, and a semantic decoder connected in sequence;
[0087] The semantic encoder is used to semantically encode the training data and obtain the semantic features of the training data; the semantic importance module is used to obtain the importance of each element of the semantic features; and the semantic decoder is used to semantically decode the semantic features.
[0088] Training a semantic communication model using training data can involve simultaneously training the semantic encoder, semantic importance module, and semantic decoder using the training data.
[0089] Specifically, the training dataset is input into the semantic encoder to obtain the semantic feature vector corresponding to the training dataset; the semantic feature vector is input into the semantic importance module to obtain the semantic importance of each element in the semantic feature vector; the semantic feature vector is input into the semantic decoder to obtain the decoder output; and the parameters of the semantic encoder, semantic importance module, and semantic decoder are adjusted through backpropagation based on the decoder output.
[0090] This disclosure does not limit the specific types of models used in the semantic encoder, semantic importance module, and semantic decoder; any model that can achieve the corresponding function is within the protection scope of this disclosure.
[0091] In some possible implementations, in step S420, the target dataset can be input into the semantic encoder to obtain the semantic features of each sample in the target dataset; the semantic features of each sample can be input into the semantic importance module to obtain the semantic importance of each element in the semantic features of each sample.
[0092] The process of inputting the target dataset into the semantic encoder can be as follows: each sample of the target dataset is input into the trained semantic encoder to obtain the semantic features of each sample. The semantic features of each sample are then input into the semantic importance module to obtain the semantic importance of each element in the semantic features of each sample, thereby obtaining the semantic importance of elements at the same position in different samples of the target dataset.
[0093] In some possible implementations, in step S430, the semantic importance of each element is sorted, and an importance template is generated based on the sorting result.
[0094] Specifically, after obtaining the semantic importance of elements at the same position in different samples of the target dataset, the mean of the semantic importance of elements at the same position in different samples can be obtained.
[0095] The mean of semantic importance of elements at the same position in different samples can be obtained by summing and averaging (i.e., summing the semantic importance of elements at the same position in different samples and dividing by the number of samples to obtain the mean), by calculating the median (i.e., sorting the semantic importance of elements at the same position in different samples and using the median as the mean), or by calculating the maximum value (i.e., sorting the semantic importance of elements at the same position in different samples and using the maximum value as the mean).
[0096] The mean values of semantic importance at different positions are sorted, and the numerical value corresponding to each position of the importance template is determined based on the sorting results.
[0097] Specifically, the mean semantic importance of elements at the same position in different samples can be normalized to a value between 0 and 1 using a normalization method. The normalized value is then filled into the corresponding position of the importance template to generate the importance template.
[0098] In some possible implementations, in step S440, the semantic encoder and semantic decoder of the semantic communication model are trained according to the importance template.
[0099] Figure 5 This diagram illustrates a flowchart of one implementation method for obtaining the trained semantic encoder and semantic decoder of a semantic communication model based on an importance template. Figure 5 As shown, it may include steps S510, S520, S530, and S540.
[0100] In step S510, the training dataset is input into the semantic encoder to obtain the semantic feature vector corresponding to the training dataset;
[0101] In step S520, some elements of the semantic feature vector are discarded according to the importance template to obtain the discarded semantic feature vector;
[0102] In step S530, the discarded semantic feature vector is input into the semantic decoder to obtain the decoder output;
[0103] In step S540, the parameters of the semantic encoder and semantic decoder are adjusted through backpropagation based on the decoder output.
[0104] In some possible implementations, in step S510, the training data may be different from the training data used to train the semantic communication model.
[0105] In some possible implementations, in step S520, similar to step S210, the elements at each position of the importance template are sorted, and the positions corresponding to the elements at the top preset ratio of the sorted elements are selected as the filtering positions according to a preset ratio. The positions corresponding to the remaining data are selected as non-filtering positions, and the elements at the non-filtering positions of the semantic feature vector are discarded.
[0106] In some possible implementations, in step S530, similar to step S320, the discarded semantic feature vectors are written sequentially to the filtering positions, and zero-padding is performed at the non-filtering positions to generate the semantic feature vectors of the input semantic decoder, and the output of the semantic decoder is obtained.
[0107] In some possible implementations, in step S540, a loss function value is determined based on the difference between the output of the semantic decoder and the training data, and the parameters of the semantic encoder and semantic decoder are adjusted by backpropagating the loss function value.
[0108] By adjusting the parameters of the semantic encoder and semantic decoder, the positions of important features extracted by the semantic encoder can be matched with the importance template. This ensures that important features of the source data are not discarded during data transmission, thereby guaranteeing the similarity between the recovered information and the source data and ensuring the quality of the data recovered by the receiving end.
[0109] The semantic communication method provided in this disclosure will be described in detail below with a specific embodiment.
[0110] This specific embodiment includes a training process and an inference process. Figure 6 A diagram illustrating the training and inference processes, such as Figure 6 The training process includes the following steps:
[0111] 1. Train a semantic communication model with semantic importance modules on other datasets. When the input is a 2048*1024 image, the semantic feature size of the encoder output is 1*96*128*64.
[0112] 2. Using the semantic importance module, calculate the semantic importance of each sample in the fine-tuned dataset. The semantic importance size is 1*96*128*64.
[0113] The fine-tuning dataset used consisted of 5000 images with a size of 3*2048*1024.
[0114] 3. Based on the preset algorithm, the importance template is calculated and saved from the semantic importance of 5000 samples, with a size of 1*96*128*64.
[0115] 4. Fine-tune the semantic communication network based on the saved importance templates.
[0116] The reasoning process includes the following steps:
[0117] 1. Deploy the fine-tuned model and importance template at both the sending and receiving ends;
[0118] 2. The transmitting end uses a semantic encoder to encode a 3*2048*1024 image to obtain 1*96*128*64 semantic features;
[0119] 3. Based on the importance template, the transmitter discards elements with lower importance according to the retention ratio α = 0.5, flattens the semantic features, converts them into binary bit rate, and then transmits them.
[0120] 4. The receiving end receives the binary bit rate and fills in the received semantic features in order based on the semantic importance template, filling the empty positions with zero elements;
[0121] 5. The receiving end inputs the restored semantic features into the semantic decoder to recover the image.
[0122] Based on and Figure 1 The method shown follows the same principle. Figure 7 A schematic diagram of the structure of a semantic communication device provided in an embodiment of this disclosure is shown, such as... Figure 7 As shown, the semantic communication device 70 may include:
[0123] Transmitter 710 is used to transmit source data, input the source data into a pre-trained semantic encoder for semantic encoding, and obtain the source semantic features of the source data; the source semantic features are filtered by a pre-obtained importance template, and the filtered source semantic features are converted into a bit stream;
[0124] Communication channel 720 is used to transmit bit streams from the transmitter to the receiver;
[0125] The receiver 730 is used to receive the bit stream, restore the bit stream to restored semantic features through the importance template, and input the restored semantic features into the pre-trained semantic decoder for semantic decoding to obtain the restored information;
[0126] The importance template is obtained by training on the target dataset; the semantic encoder and semantic decoder are obtained by training on the importance template.
[0127] In the semantic communication apparatus provided in this embodiment, the transmitting end and the receiving end use the same importance template. The transmitting end discards part of the semantically encoded content based on semantic importance and then retransmits it. The receiving end can recover the discarded part based on the importance template, so that no additional transmission index is needed. This reduces the additional overhead of transmission index and enables communication at a low signal-to-noise ratio.
[0128] In some possible implementations, the transmitter is also used to: discard some elements of the source semantic features according to a preset ratio using an importance template; and convert the remaining source semantic features into a bit stream.
[0129] In some possible implementations, the receiver is also used to: restore the bit stream received by the receiver to feature data; and pad the feature data with zeros using an importance template to obtain restored semantic features.
[0130] In some possible implementations, the semantic communication device further includes: a training module for training a semantic communication model using a training dataset; the semantic communication model includes a semantic encoder, a semantic importance module, and a semantic decoder connected in sequence; using the trained semantic importance module, the semantic importance of each sample in the target dataset is calculated; an importance template is generated based on the semantic importance of each sample in the target dataset; and the semantic communication model is fine-tuned based on the importance template to obtain the trained semantic encoder and semantic decoder.
[0131] In some possible implementations, the training module is also used to: input the training dataset into the semantic encoder to obtain the semantic feature vector corresponding to the training dataset; input the semantic feature vector into the semantic importance module to obtain the semantic importance of each element in the semantic feature vector; input the semantic feature vector into the semantic decoder to obtain the decoder output; and adjust the parameters of the semantic encoder, the semantic importance module, and the semantic decoder through backpropagation based on the decoder output.
[0132] In some possible implementations, the training module is also used to: input the target dataset into the semantic encoder to obtain the semantic features of each sample in the target dataset; input the semantic features of each sample into the semantic importance module to obtain the semantic importance of each element in the semantic features of each sample.
[0133] In some possible implementations, the training module is also used to: rank the semantic importance of each element and generate an importance template based on the ranking results.
[0134] In some possible implementations, the training module is also used to: input the training dataset into the semantic encoder to obtain the semantic feature vector corresponding to the training dataset; discard some elements of the semantic feature vector according to the importance template to obtain the discarded semantic feature vector; input the discarded semantic feature vector into the semantic decoder to obtain the decoder output; and adjust the parameters of the semantic encoder and semantic decoder through backpropagation based on the decoder output.
[0135] In some possible implementations, the source data can be any of the following: image data, voice data, text data, or point cloud data.
[0136] It is understood that the above-described modules of the semantic communication device in the embodiments of this disclosure have the ability to implement... Figure 1 The semantic communication method in the illustrated embodiment demonstrates the functionality of corresponding steps. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions. These modules can be software and / or hardware, and each module can be implemented individually or multiple modules can be integrated. For a detailed description of the functions of each module in the semantic communication device, please refer to [link to relevant documentation]. Figure 1 The corresponding descriptions of the semantic communication methods in the embodiments shown are not repeated here.
[0137] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0138] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0139] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the semantic communication method provided in the embodiments of this disclosure.
[0140] Compared with existing technologies, this electronic device uses the same importance template for both the transmitter and receiver. At the transmitter, the semantically encoded content is discarded and retransmitted based on semantic importance. The receiver can recover the discarded part based on the importance template, thus eliminating the need for an additional transmission index. This reduces the overhead of the transmission index and enables communication at a lower signal-to-noise ratio.
[0141] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the semantic communication method provided in the embodiments of this disclosure.
[0142] Compared with existing technologies, this readable storage medium uses the same importance template at both the transmitting and receiving ends. At the transmitting end, the semantically encoded content is discarded based on semantic importance and then retransmitted. The receiving end can recover the discarded part based on the importance template, thus eliminating the need for additional transmission indexes. This reduces the overhead of transmission indexes and enables communication at lower signal-to-noise ratios.
[0143] The computer program product includes a computer program that, when executed by a processor, implements the semantic communication method as provided in the embodiments of this disclosure.
[0144] Compared with existing technologies, this computer program product uses the same importance template for both the transmitter and receiver. At the transmitter, based on semantic importance, the semantically encoded content is discarded and then retransmitted. The receiver can recover the discarded part based on the importance template, thus eliminating the need for an additional transmission index. This reduces the overhead of the transmission index and enables communication at a lower signal-to-noise ratio.
[0145] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0146] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.
[0147] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0148] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as semantic communication methods. For example, in some embodiments, the semantic communication method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the semantic communication method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform semantic communication methods by any other suitable means (e.g., by means of firmware).
[0149] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0150] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0151] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0152] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0153] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0154] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0155] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0156] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. A semantic communication method, the method being used in a semantic communication system, the semantic communication system comprising a transmitter, a receiver, and a communication channel connecting the transmitter and the receiver, the method comprising: The source data transmitted by the transmitter is input into a pre-trained semantic encoder for semantic encoding to obtain the source semantic features of the source data; The semantic features of the information source are filtered by a pre-acquired importance template, and the filtered semantic features of the information source are converted into a bit stream. The bit stream is transmitted to the receiving end through the communication channel; The importance template is used to restore the bit stream received by the receiving end into restored semantic features, and the restored semantic features are input into a pre-trained semantic decoder for semantic decoding to obtain the restored information. The importance template is obtained by training on the target dataset; the semantic encoder and the semantic decoder are trained based on the importance template. The method further includes: A semantic communication model is trained using a training dataset; the semantic communication model includes a semantic encoder, a semantic importance module, and a semantic decoder connected in sequence. Using the trained semantic importance module, calculate the semantic importance of each sample in the target dataset; Based on the semantic importance of each sample in the target dataset, the importance template is generated; Based on the importance template, the semantic communication model is fine-tuned to obtain the trained semantic encoder and semantic decoder.
2. The method according to claim 1, wherein, The step of filtering the source semantic features using a pre-acquired importance template and converting the filtered source semantic features into a bitstream includes: By using the importance template, some elements in the semantic features of the information source are discarded according to a preset ratio; The remaining source semantic features are discarded and converted into a bitstream.
3. The method according to claim 1, wherein, The process of restoring the received bitstream to its original semantic features using the importance template includes: The received bit stream is restored to characteristic data; The feature data is padded with zeros using the importance template to obtain the restored semantic features.
4. The method according to claim 1, wherein, The step of training the semantic communication model using a training dataset includes: The training dataset is input into the semantic encoder to obtain the semantic feature vector corresponding to the training dataset; The semantic feature vector is input into the semantic importance module to obtain the semantic importance of each element in the semantic feature vector; Input the semantic feature vector into the semantic decoder to obtain the decoder output; Based on the decoder output, the parameters of the semantic encoder, semantic importance module, and semantic decoder are adjusted through backpropagation.
5. The method according to claim 1, wherein, The step of using the trained semantic importance module to calculate the semantic importance of each sample in the target dataset includes: The target dataset is input into the semantic encoder to obtain the semantic features of each sample in the target dataset; Input the semantic features of each sample into the semantic importance module to obtain the semantic importance of each element in the semantic features of each sample.
6. The method according to claim 5, wherein, The step of generating the importance template based on the semantic importance of each sample in the target dataset includes: The semantic importance of each element is sorted, and the importance template is generated based on the sorting results.
7. The method according to claim 1, wherein, The step of fine-tuning the semantic communication model according to the importance template to obtain the trained semantic encoder and semantic decoder includes: The training dataset is input into the semantic encoder to obtain the semantic feature vector corresponding to the training dataset; Based on the importance template, some elements of the semantic feature vector are discarded to obtain the discarded semantic feature vector; The discarded semantic feature vector is input into the semantic decoder to obtain the decoder output; The parameters of the semantic encoder and the semantic decoder are adjusted through backpropagation based on the decoder output.
8. The method according to claim 1, wherein, The source data can be any one of image data, voice data, text data, or point cloud data.
9. A semantic communication system, comprising: The transmitting end is used to transmit source data, input the source data into a pre-trained semantic encoder for semantic encoding, and obtain the source semantic features of the source data; The semantic features of the information source are filtered by a pre-acquired importance template, and the filtered semantic features of the information source are converted into a bit stream. A communication channel for transmitting the bit stream from the transmitting end to the receiving end; The receiving end is used to receive the bit stream, restore the bit stream to restored semantic features through the importance template, and input the restored semantic features into a pre-trained semantic decoder for semantic decoding to obtain the restored information; The importance template is obtained by training on the target dataset; the semantic encoder and the semantic decoder are trained based on the importance template. The training module is used to train a semantic communication model using a training dataset. The semantic communication model includes a semantic encoder, a semantic importance module, and a semantic decoder connected in sequence. Using the trained semantic importance module, the semantic importance of each sample in the target dataset is calculated. Based on the semantic importance of each sample in the target dataset, the importance template is generated. Based on the importance template, the semantic communication model is fine-tuned to obtain the trained semantic encoder and semantic decoder.
10. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.
11. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.
12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.
Citation Information
Patent Citations
Code rate adaptive video semantic communication method and related device
CN116896651A
