Semantic communication method and apparatus, and storage medium

Abstract

The embodiments of the present disclosure relate to the technical field of communications. Provided are a semantic communication method and apparatus, and a storage medium, which can realize matching between semantic features and channel features. The semantic communication method comprises: a coding end acquiring original information, and performing semantic coding on the original information on the basis of channel feature parameters to obtain first semantic feature information, wherein the first semantic feature information is used for representing semantic features of the original information, and the coding end sending the first semantic feature information; and a decoding end acquiring second semantic feature information obtained after the first semantic feature information is transmitted via a channel, and obtaining decoding information on the basis of the second semantic feature information.

Description

Semantic communication method, device and storage medium

[0001] This application claims priority to Chinese patent application No. 202410029099.1, filed on January 4, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of communication technology, and in particular to a semantic communication method, device, and storage medium. Background Art

[0003] With the continuous advancement of communication technology, classic communication has almost reached its limits, but it still cannot meet the high data volume requirements of various applications, such as mixed reality and immersive communication. Furthermore, with the continuous development of artificial intelligence (AI), the demand for communication data volume will further increase, and the demand for intelligent communication will become increasingly strong. To cope with this massive data volume requirement and to seamlessly integrate with various types of future AI applications, semantic communication has emerged.

[0004] Current semantic communication training typically uses an end-to-end training approach to develop a semantic communication codec. This approach fails to consider the impact of various factors on semantic communication during actual transmission, which can negatively impact the performance of semantic communication.

[0005] Summary of the Invention

[0006] The embodiments of the present disclosure provide a semantic communication method, apparatus, and storage medium, which can achieve matching of semantic features with channel features.

[0007] In one aspect, a semantic communication method is provided, applied to an encoding end. The semantic communication method includes: obtaining original information; semantically encoding the original information based on channel characteristic parameters to obtain first semantic feature information, the first semantic feature information being used to represent semantic features of the original information; and sending the first semantic feature information.

[0008] In another aspect, a semantic communication method is provided, applied to a decoding end. The semantic communication method includes: obtaining second semantic feature information, where the second semantic feature information is information obtained after first semantic feature information sent by an encoding end is transmitted through a channel, the first semantic feature information being matched with a channel characteristic parameter, and the first semantic feature information being used to represent semantic features of the original information; and obtaining decoded information based on the second semantic feature information.

[0009] In another aspect, a communication device is provided, comprising an acquisition module and a transmission module. The acquisition module is configured to acquire original information. The acquisition module is further configured to semantically encode the original information based on channel characteristic parameters to obtain first semantic feature information. The first semantic feature information is configured to represent semantic features of the original information. The transmission module is configured to transmit the first semantic feature information.

[0010] In another aspect, a communication device is provided, comprising an acquisition module. The acquisition module is configured to acquire second semantic feature information. The second semantic feature information is information obtained after first semantic feature information sent by an encoder is transmitted through a channel. The first semantic feature information matches a channel characteristic parameter and is used to represent semantic features of the original information. The acquisition module is further configured to obtain decoded information based on the second semantic feature information.

[0011] On the other hand, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the semantic communication method of any of the above aspects is implemented.

[0012] In yet another aspect, a computer program product is provided, comprising computer program instructions, which implement the semantic communication method according to any one of the above aspects when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the present disclosure, the following briefly introduces the drawings required for use in some embodiments of the present disclosure. Obviously, the drawings described below are only drawings of some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0014] FIG1 is a schematic structural diagram of a DeepJSCC model according to some embodiments of the present disclosure.

[0015] FIG2 is a schematic structural diagram of an NTSCC model according to some embodiments of the present disclosure.

[0016] FIG3 is a schematic structural diagram of a VSCC model according to some embodiments of the present disclosure.

[0017] FIG4 is a schematic structural diagram of a communication system according to some embodiments of the present disclosure.

[0018] FIG5 is a flowchart of a semantic communication method according to some embodiments of the present disclosure.

[0019] FIG6 is a flowchart of another semantic communication method according to some embodiments of the present disclosure.

[0020] FIG7 is a schematic structural diagram of a semantic communication method according to some embodiments of the present disclosure.

[0021] FIG8 is a schematic structural diagram of another semantic communication method according to some embodiments of the present disclosure.

[0022] FIG9 is a flowchart of yet another semantic communication method according to some embodiments of the present disclosure.

[0023] FIG10 is a flowchart of yet another semantic communication method according to some embodiments of the present disclosure.

[0024] FIG11 is a flowchart illustrating model fine-tuning in a semantic communication method according to some embodiments of the present disclosure.

[0025] FIG12 is a schematic diagram showing the composition of a communication device according to some embodiments of the present disclosure.

[0026] FIG13 is a schematic diagram showing the composition of another communication device according to some embodiments of the present disclosure.

[0027] FIG14 is a schematic structural diagram of a communication device according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] The following will clearly and completely describe the technical solutions of this disclosure in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of this disclosure, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0029] It should be noted that, in this disclosure, words such as "exemplary" or "for example" are used to describe examples, illustrations, or explanations. Any embodiment or design described in this disclosure using words such as "exemplary" or "for example" should not be interpreted as being more preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0030] In the following, the terms "first," "second," etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, a feature specified as "first," "second," etc. may explicitly or implicitly include one or more of the features.

[0031] In the description of this disclosure, unless otherwise specified, " / " means "or." For example, A / B can mean A or B. "And / or" herein is merely a description of an association relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: only A, only B, and both A and B. Furthermore, "at least one" means one or more, and "a plurality" means two or more.

[0032] First, the relevant terms involved in this disclosure are explained.

[0033] 1. A generative adversarial network (GAN) is a model that learns through a game of two neural networks. GANs can learn generative tasks without using labeled data. A GAN typically consists of a generator and a discriminator. The generator randomly samples from the latent space as input, and its output is required to closely mimic real samples from the training set. The discriminator takes real samples or the generator's output as input, and its goal is to distinguish the generator's output from real samples as much as possible. The generator and discriminator compete with each other and continuously learn, ultimately making it impossible for the discriminator to determine whether the generator's output is realistic.

[0034] 2. Variational autoencoders (VAEs), similar to generative adversarial networks, were developed to solve the problem of data generation. In autoencoder architectures, data is typically required as input, and the generated data is identical to the input data. However, it is often desirable for the generated data to have a certain degree of variation, which requires the input of a random vector and the ability for the model to learn the stylized characteristics of the generated image. Therefore, subsequent research has led to the development of generative adversarial network architectures that use random vectors as input to generate specific samples. Similarly, variational autoencoders use random samples from a specific distribution as input and can generate corresponding images. In this respect, variational autoencoders and generative adversarial networks have similar goals. However, variational autoencoders do not require a discriminator, but instead use an encoder to estimate a specific distribution.

[0035] 3. Semantic communication is a task-based communication technology that prioritizes understanding over transmission. In semantic communication, the original signal undergoes selective feature extraction, compression, and transmission, and then uses semantic information for communication. This technology can significantly improve the transmission efficiency of communication systems.

[0036] Semantic communication leverages the powerful nonlinear fitting capabilities of artificial neural networks (ANNs) to further compress data based on the semantic dimension of the source. This is typically achieved using a generative adversarial network or variational autoencoder architecture. Furthermore, due to the specific nature of ANN training, semantic communication encoders are currently typically trained end-to-end, employing joint source channel coding (JSCC).

[0037] On the one hand, semantic communication, based on the JSCC framework, combines source coding with channel coding to eliminate the correlation between the source and channel in the semantic dimension, even under limited code length or source distortion conditions. This achieves further communication gain, breaks the communication rate limitations of the classic separation framework, and increases the theoretical amount of data that can be transmitted by the channel. On the other hand, semantic communication technology, based on ANN, obtains similar structural information from the source data distribution and the channel transition probability distribution, summarizes this information into a knowledge base, and uses this knowledge base to further compress (encode) and expand (decode) the source, thereby achieving more information transmission in the semantic dimension.

[0038] However, despite adopting the JSCC communication architecture, the uninterpretability of ANNs makes it impossible to accurately model the gains that semantic communication can achieve from joint coding. In particular, current semantic communication implementations are all channel-independent, with no channel-related variables in their loss functions. That is, the channel is not considered within the training process, making it impossible to explicitly describe the relationship between the channel and the semantic codec, and unable to determine whether semantic communication can generate gains on the channel side. Furthermore, the lack of an explicit representation of the channel in the loss function results in poor generalization of the semantic codec, and it is also impossible to fine-tune it based on different channel characteristics to cope with varying channel variations.

[0039] The following is an example of the DeepJSCC (deep joint source channel coding) model, a communication architecture that complies with JSCC. Figure 1 is a schematic diagram of the structure of a DeepJSCC model according to an embodiment of the present disclosure. In this model, the joint source channel coding (semantic encoder) is the discriminator of the deep convolutional generative adversarial network (DCGAN), which is implemented using convolutional layers. The joint source channel decoding (semantic decoder) is the generator of the DCGAN, which is implemented using deconvolutional layers.

[0040] This model is essentially based on the auto encoder (AE) framework. Its source coding and channel coding adopt a joint coding method and are implemented based on the DCGAN discriminator. Its channel decoding and source decoding parts adopt a joint decoding method and are implemented based on the DCGAN generator.

[0041] The loss function during model training is the mean square error (MSE) between the input data and the output data:

[0042] In the formula, L represents the loss function, x represents the input data, Represents output data, It represents the expectation when x meets the probability distribution Px(x).

[0043] This model suffers from two problems. First, it relies on fixed-length coding, failing to consider that different sources may have semantic feature information of varying lengths, necessitating variable-length coding. Second, the model relies on an automated automatic recognition (AE) architecture to extract semantic feature information from the source, but fails to model the channel during the modeling process, nor does it incorporate channel feature parameters into the loss function. Consequently, it fails to effectively align source and channel feature parameters.

[0044] The nonlinear transform source channel coding (NTSCC) model has been proposed in related technologies to address the variable-length coding issue. Based on the DeepJSCC model, the NTSCC model adds a semantic feature extraction module and a semantic feature fusion module for the source, implemented using nonlinear source coding technology. Its structure is shown in Figure 2.

[0045] By adding a semantic feature extraction module to the NTSCC model, nonlinear semantic information fitting and modeling can be performed for different sources, thereby achieving a coding rate determined by the source semantics and ultimately achieving semantic encoding. The semantic feature extraction module uses nonlinear compression and can be implemented based on architectures such as Transformers and neural networks to achieve text or image transmission.

[0046] However, the modeling of the channel is still not considered in the NTSCC model. The loss function of this model is based on the loss function of the VAE model, that is, the KL divergence of the joint distribution. However, in the derivation process, the channel is regarded as a constant, and the posterior probability of the channel output value is also regarded as a constant, causing the VAE model to degenerate into an AE model. Its final loss function is:

[0047] Where η i , β M , β L , β D represents a constant (weight coefficient), d LPIPS , D represents two indicators for measuring image similarity, d LPIPS represents the distance of the learned perceptual image patch similarity (LPIPS) obtained by image learning, D represents the distance between the two images output by the discriminator of the cGAN (condition GAN, generating adversarial model), i represents the image pixel sequence number, and x represents the original image input. represents the received image output, y represents the semantic feature extraction output, represents the semantic features obtained after rate matching and channel transmission. Therefore, the NTSCC model does not truly incorporate the channel into semantic communication modeling, but only further fits the semantic feature information of the source.

[0048] In response to the problems existing in the above-mentioned model, in order to incorporate the channel into the consideration of semantic communication, better explain the role of the channel in JSCC, and also to eliminate the correlation between the source and the channel in the semantic dimension and further explore the potential of semantic communication, the present invention discloses a JSCC model based on variational inference, also known as variational source channel coding (VSCC) model, and proposes a semantic communication method. In this semantic communication method, the channel noise distribution becomes the core of variational inference and is a key consideration for deriving latent variables. In addition, the derivation of latent variables is based on the ANN training process, and the source distribution also needs to be considered. Therefore, in this semantic communication method, the source characteristics and channel characteristics are taken into account at the same time, so that the extracted source semantic characteristics match the channel noise, thereby effectively reducing the amount of data transmitted by the channel while ensuring the accuracy of semantic communication.

[0049] FIG3 is a schematic diagram of the structure of a VSCC model according to an embodiment of the present disclosure. As shown in FIG3, the input of the VSCC model is x. x first passes through a joint precoder f en (x; θ), we get the codeword y, where θ is the trainable parameter of the joint precoder. Then through the channel, the output is recorded as z (i.e., the hidden variable). Here The mapping function corresponding to the channel input and output is recorded as in Represents the relevant parameters of the mapping function. Finally, it passes through a joint decoder f de(z; φ), where φ is the trainable parameter of the joint decoder, and the final output is

[0050] In order to train the encoder / decoder in the above model, it is necessary to mathematically model it. As shown in Figure 3, in the model, the data flow undergoes two changes, one is joint source channel coding, and the other is joint source channel decoding. These two changes can be represented by the posterior probability p shown in the figure. z|x (z|x) and q x|z (x|z) to describe.

[0051] In the VSCC model, the purpose is to allow the decoding end (sink) to obtain information Distribution As close as possible to the distribution p of the message x sent by the encoding end (source) x (x) are the same to achieve the purpose of semantic communication. It should be noted that when With p x When (x) is close, the data sampled from both can be considered as x, so q can be used x (x) instead Conduct subsequent derivation.

[0052] In order to make the two distributions close, the KL divergence can be used to measure the distance between the distributions to minimize the KL divergence between the source and the destination, that is: min KL(p x (x)||q x (x)) (1)

[0053] Under the assumptions of the VSCC model, the above formula (1) can be further derived through the KL divergence of the joint probability density, because the KL divergence of the joint probability density is the upper bound of the KL divergence of the marginal distribution, that is: KL(p x,z (x,z)||q x,z (x,z)) = KL(p x (x)||q x (x))+∫p x (x)KL(p z|x (z|x)||q z|x (z|x))dx ≥KL(p x (x)||q x (x)) (2)

[0054] Where p x,z (x,z) and q x,z (x,z) denotes the joint distribution of x and z from the joint encoder / decoder perspective.

[0055] Therefore, the KL divergence of the marginal distribution (i.e., formula (1)) can be indirectly optimized by minimizing formula (2), and formula (2) can be further optimized based on the variational inference principle:

[0056] Where, Because p x (x) is a known fixed input message distribution, so it is a constant and does not play a role in the joint encoder / decoder training process of the VSCC model, so it can be discarded. Further derivation:

[0057] In formula (4), the first term corresponds to the joint encoder p z|x (z|x), the second term corresponds to the joint decoder q x|z (x|z),q z (z) represents the distribution function of z. Based on formula (4), the embodiment of the present disclosure proposes a semantic communication method that can fuse source features with channel features.

[0058] In the embodiments of the present disclosure, the network architecture of a mobile communication system (including but not limited to 5G and future 6G mobile communication systems) may include an encoding end and a decoding end. In some examples, the encoding end may be a base station, and the decoding end may be a terminal (UE). In other examples, the encoding end may be a UE, and the decoding end may be a base station. The embodiments of the present disclosure are not limited thereto.

[0059] The following description is made by taking the encoding end as a base station and the decoding end as a terminal as an example.

[0060] Figure 4 is a schematic diagram of the structure of a communication system according to an embodiment of the present disclosure. As shown in Figure 4, the communication system 40 includes a base station 41 and a terminal 42. The base station 41 and the terminal 42 can be connected via a wireless network.

[0061] In some embodiments, base station 41 is used to provide wireless access services to terminal 42. For example, a base station 41 provides a service coverage area (also called a cell). Terminal 42 entering this area can communicate with base station 41 via wireless signals to receive the wireless access services provided by base station 41.

[0062] In some embodiments, the base station 41 may be a millimeter wave base station, an evolution nodeB (eNB), a next generation nodeB (gNB), a transmission and reception point (TRP), a transmission point (TP), and some other access node. Depending on the size of the service coverage area provided, the base station 41 can be divided into a macro base station for providing macro cells (Macro cell), a micro base station for providing micro cells (Pico cell), and a femto base station for providing femto cells (Femto cell). With the continuous evolution of wireless communication technology, future base stations may also adopt other names. The signal coverage range of the base station 41 also includes near field and far field, and the terminal 42 can be within the near field range or the far field range.

[0063] In some embodiments, the terminal 42 may be a device with wireless transceiver capabilities, such as a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The embodiment of the present invention does not limit the type of the terminal 42.

[0064] It should be understood that Figure 4 is an exemplary structural diagram, and the number of devices included in the communication system shown in Figure 4 is not limited, for example, the number of base stations is not limited, and the number of terminals is not limited. Furthermore, in addition to the devices shown in Figure 4, the communication system shown in Figure 1 may also include other devices, which is not limited.

[0065] Figure 5 is a flow chart of a semantic communication method according to an embodiment of the present disclosure. The semantic communication method provided by the present disclosure can be applied to an encoding end, and exemplarily can be applied to a base station of the communication system shown in Figure 4 .

[0066] As shown in FIG5 , the semantic communication method provided by the present disclosure may include, for example, the following S501 to S503 .

[0067] In S501, original information is obtained.

[0068] In S502, semantic encoding is performed on the original information based on the channel characteristic parameters to obtain first semantic characteristic information.

[0069] In S503, first semantic characteristic information is sent.

[0070] The first semantic feature information is used to represent the semantic features of the original information.

[0071] In the embodiment of the present disclosure, the data volume of the first semantic feature information is smaller than the data volume of the original information, which is beneficial to reducing transmission overhead.

[0072] In some embodiments, the original information may be control signaling or service data, which is not limited.

[0073] In some embodiments, the channel characteristic parameter includes at least one of the following: channel noise variance, channel state information. The channel state information includes at least one of the following: signal-to-noise ratio (SNR), signal-to-interference plus noise ratio (SINR), channel matrix, channel quality indication (CQI), and reference signal received quality (RSRQ).

[0074] In some embodiments, the channel state information is obtained by measuring a pilot signal.

[0075] In some embodiments, the above S502 may be implemented as follows: the encoding end may perform semantic encoding on the original information based on the channel characteristic parameters to obtain first semantic feature information.

[0076] In some embodiments, the encoding end can semantically encode the original information based on the channel characteristic parameters to obtain first semantic feature information. This can be implemented, for example, as follows: the encoding end can input the original information into the encoder, and the encoder can semantically encode the original information based on the channel characteristic parameters to obtain the first semantic feature information.

[0077] In the embodiments of the present disclosure, the encoder may be referred to as a semantic encoder, or other names, without limitation.

[0078] It should be understood that semantic encoding, as discussed here, involves rearranging the original message data based on the channel transition probability distribution. The semantic encoding process encodes the original message data into first semantic feature information. The variables in each dimension of this first semantic feature information approximate the channel transition probability distribution. However, in the subsequent process, all dimensional variables pass through the channel to obtain second semantic feature information. This second semantic feature information is also a latent variable in variational inference, and its distribution approximates the channel transition probability distribution. This second semantic feature information then passes through a decoder that matches the channel, restoring the original message distribution.

[0079] The first semantic feature information is data that matches the channel transition probability before passing through the channel and is not affected by channel noise. Matching means that after transmission through the channel, the second semantic feature information that approximates the channel transition probability distribution can be obtained. In some embodiments, the encoder can be trained by the encoder itself or configured to the encoder by other nodes (such as core network elements or servers) after training.

[0080] In some embodiments, the encoder (also referred to as a semantic encoder) can be trained using a loss function derived from variational inference, where the loss function includes a regularization term and a reconstruction term. The details of the loss function can be found in the description of Example 1 or Example 2 below and are not detailed here.

[0081] In some embodiments, the regularization term can be determined based on source characteristic parameters and channel characteristic parameters. The source characteristic parameters are used to characterize the distribution characteristics of the information emitted by the source. The channel characteristic parameters are used to characterize the distribution characteristics of the channel transition probability. The details of the regularization term can be found in the relevant description of Example 1 or Example 2 below and are not detailed here.

[0082] In some embodiments, the distribution characteristics of the latent variables in variational inference are channel state transition probability distribution characteristics.

[0083] In some embodiments, the reconstruction item can be determined based on source characteristic parameters and sink characteristic parameters. The source characteristic parameters are used to characterize the distribution characteristics of information sent by the source. The sink characteristic parameters are used to characterize the distribution characteristics of information received by the sink. The details of the reconstruction item can be found in the relevant descriptions of Example 1 or Example 2 below and are not further elaborated here.

[0084] In some scenarios, the channel may undergo some slight changes during multiple communications. At this time, it is necessary to continuously transmit pilot signals to obtain the corresponding channel information, and then fine-tune the trained semantic codec according to the channel changes to re-achieve matching with the changed channel.

[0085] In the disclosed embodiment, since the first semantic feature information extracted from the original information matches the channel feature parameters, the stability of the first semantic feature information during channel transmission is ensured, which is beneficial to improving the communication efficiency of semantic communication.

[0086] Figure 6 is a flow chart of another semantic communication method according to an embodiment of the present disclosure. The semantic communication method provided by the present disclosure is applied to a decoding end, and can be exemplarily applied to the terminal of the communication system shown in Figure 4 .

[0087] As shown in FIG6 , the semantic communication method provided by the present disclosure may include, for example, the following steps S601 and S602 .

[0088] In S601, second semantic feature information is obtained.

[0089] In S602, decoding information is obtained based on the second semantic feature information.

[0090] In some embodiments, the second semantic feature information is information obtained after the first semantic feature information sent by the encoding end is transmitted through a channel, and the first semantic feature information matches the channel characteristic parameters.

[0091] In some embodiments, the above S602 may be implemented as: performing semantic decoding on the second semantic feature information based on the channel characteristic parameter to obtain decoding information.

[0092] In some embodiments, the decoding end can perform semantic decoding on the second semantic feature information based on the channel characteristic parameters to obtain decoding information, which can be implemented as follows: inputting the second semantic feature information into the decoder, and performing semantic decoding on the second semantic feature information based on the channel characteristic parameters by the decoder to obtain decoding information.

[0093] It should be understood that semantic decoding here refers to decoding the second semantic feature information based on the channel transition probability distribution. The semantic decoding process involves decoding the second semantic feature information using a channel-matched decoder to restore it to the original message distribution. The second semantic feature information approximates the channel transition probability distribution after passing through the channel. A channel-matched decoder is one trained using a loss function that incorporates channel characteristic parameters.

[0094] In the embodiments of the present disclosure, the decoder may be referred to as a semantic decoder or other names, which are not limited.

[0095] In some embodiments, the decoder may be trained by the decoding end itself, or may be configured to the decoding end after being trained by other nodes (eg, core network elements or servers).

[0096] In some embodiments, the decoder is trained using a loss function obtained based on variational inference, where the loss function includes a regularization term and a reconstruction term.

[0097] For the description of the regularization term and reconstruction term in the loss function, please refer to the description of the encoding end or the description below, which will not be repeated here.

[0098] In some embodiments, the decoder includes a latent variable calculation module, a reparameterization module, and a semantic feature information recovery module, which are connected in sequence. The latent variable calculation module is configured to extract latent variables from the second semantic feature information. The reparameterization module is configured to reparameterize the latent variables. The semantic feature information recovery module is configured to perform semantic decoding on the reparameterized latent variables to obtain decoded information.

[0099] The disclosed embodiments include a semantic communication method based on the VSCC model, which achieves matching between the source and the channel. By matching the distribution characteristics of the two, semantic feature information is extracted, ultimately resulting in a semantic encoder and semantic decoder that match the channel. For example, the semantic encoder uses the channel noise distribution as the latent variable distribution in variational inference, performs semantic segmentation on the source input, and obtains semantic feature information. The semantic decoder then recovers the received semantic feature information based on the corresponding semantic segmentation method, ultimately completing the communication.

[0100] The encoder and its corresponding decoder are described in detail below with reference to the embodiments and accompanying drawings. Embodiments 1 and 2 are described using Gaussian channels as an example, while embodiments 3 and 4 are described using general channels as an example.

[0101] Example 1

[0102] This embodiment proposes a semantic encoding method for Gaussian channels, and its model is shown in Figure 7. As shown in the structure in Figure 7, the semantic encoder is implemented in the form of a joint encoder, and the joint encoder can be implemented using an ANN module. The ANN module can be implemented by a Transformer layer, a CNN (convolutional neural network) layer, an RNN (recurrent neural network) layer, or a Dense (fully connected) layer, etc., so as to facilitate information transmission in different modalities. In the process of forward data transmission, the original information x is encoded by the semantic encoder to obtain the first semantic feature information y whose data volume is less than the original information x.

[0103] First, the original information x passes through the semantic encoder. After being trained with the loss function of variational inference, the encoder has two main encoding features, namely the regularization term and the reconstruction term.

[0104] On the one hand, the encoder semantically encodes the original information x in the direction of the channel noise distribution in order to obtain the first semantic feature information y that is consistent with the channel characteristic parameters. This capability is determined by the first term in the above formula (4), which can be called the regularization term. This term can make the encoded latent variable tend to the channel noise distribution. In other words, this term allows the encoder to use the channel noise distribution to divide and extract the latent variables of the input information, thereby making the encoder tend to use the channel characteristic parameters to extract semantic features from the source.

[0105] In one implementation, in order to achieve the above encoding effect, it is necessary to make assumptions about the posterior probability and the prior probability in the regularization term. z|x (z|x), for the convenience of calculation, it is assumed that the output y of the semantic encoder follows the Gaussian distribution N(μ1,σ1 2 ). If the channel is a Gaussian noise channel, the channel output is Among them, n~N(0,σ2 2 ) is the noise that conforms to Gaussian distribution, then there is Afterwards, receive the message are transmitted to the mean calculation module and the variance calculation module respectively, and the mean μ and variance σ of the latent variables calculated by the model are obtained: μ=μ1+0 (5) σ 2 =σ1 2 +σ2 2 (6)

[0106] Subsequently, the mean and variance are fed into the reparameterization module, which calculates the corresponding latent variable z: z=ε·σ+μ (7)

[0107] Among them, ε~N(0;1) is a random number that conforms to the standard Gaussian distribution.

[0108] It should be noted that this embodiment assumes a continuous Gaussian distribution. If the latent variable is a discrete distribution, the reparameterization needs to be performed using a discrete distribution method, such as Gumbel-Softmax reparameterization, which will not be described here. Finally, the calculated latent variable is restored through the semantic decoder to obtain the restored message

[0109] According to the properties of Gaussian distribution random variables, the posterior probability distribution of the latent variable z received by the decoder is: z|x (z|x)~N(μ=μ1+0,σ 2 =σ1 2 +σ2 2) (8)

[0110] For the prior probability q z (z), in order to include the influence of the channel, it can be assumed that the true latent variable z obeys the channel transition probability distribution N(0,σ2 2 ), so far all the assumptions in variational inference are completed, and the regularization term can be solved by bringing in:

[0111] Finally, the loss function used to train the encoder part can be written as:

[0112] It can be seen that the loss function not only includes the channel noise variance, but is also related to the source distribution. Therefore, the semantic encoder considers both the source and channel feature parameters during the encoding process, and can use the channel feature parameters to semantically encode the source.

[0113] On the other hand, the encoder considers the distribution characteristics of the original information x and the distribution characteristics of the destination information simultaneously to achieve a better semantic feature extraction method. This is determined by the second term in formula (4), which can be called the reconstruction term. The reconstruction term makes the data at the transmitting and receiving ends as similar as possible, thereby limiting the encoder to encoding based on the distribution characteristics of the original information x, thereby preserving the source semantic feature information.

[0114] In summary, the joint encoder with added channels can be modeled through variational inference. The final loss function for training the semantic encoder based on this algorithm model is:

[0115] When the data gradient is propagated backward, the channel noise variance σ2 needs to be known 2 , and then output μ, σ, Substitute it into formula (17) and continuously optimize to complete the training of the parameters θ of the ANN module in the semantic encoder.

[0116] Example 2

[0117] This embodiment proposes a semantic decoding method for Gaussian channels, and its model is shown in Figure 7. As shown in the structure in Figure 7, the semantic decoder is implemented in the form of a joint decoder, and the joint decoder can be implemented using an ANN module. The ANN module can be implemented by a Transformer layer, a CNN layer, an RNN layer, a Dense layer, and different activation functions, so as to realize information decoding of different modalities. In the process of forward data transmission, the encoded information is transmitted through the channel and becomes the second semantic feature information received by the decoding end. Finally, it is restored by the semantic decoder to obtain the semantic level decoded message

[0118] For the decoder, on the one hand, it is determined by the first term in formula (4) that it will perform decoding while taking into account the channel characteristic parameters. On the other hand, it is determined by the second term in formula (4) that the decoder is restricted to decoding based on the distribution characteristics of the original information x.

[0119] To achieve the above decoding effect, a reconstruction term needs to be determined. This reconstruction term can be solved based on whether the source distribution to be recovered is discrete or continuous, thereby obtaining semantic communication decoding methods for different information modalities. Here are two examples. For example, assume that the probability distribution of the information x to be decoded is a Bernoulli distribution, that is, x can only take the value of 1 or 0:

[0120] The decoder is responsible for calculating the probability ρ(z) of the above distribution, which is equivalent to making a classification prediction for the discrete x, and can be used as a semantic communication method for transmitting binary images. Assuming that the binary image contains D pixels, the second term of the above formula can be calculated as:

[0121] Furthermore, if it is assumed that the probability distribution of x is a discrete distribution with D variables, the semantic communication method of text transmission can be obtained according to a similar calculation process (assuming that the number of different characters contained in the text is D), which will not be repeated here.

[0122] If we assume that the probability distribution of x is Gaussian, we can also use the decoder to calculate the mean and variance of the distribution to obtain N(μ(z),σ 2 (z)), which is equivalent to generating continuous x, and can be used as a semantic communication method for image transmission with D continuous value pixels. The above formula (12) can be calculated as MSE:

[0123] The above formula (13) can be calculated as:

[0124] Assuming the variance in the input data is constant, we have:

[0125] It should be noted that in the above process of solving the reconstruction term, the hidden variable z is sampled once as an example, so the above omission of p z|x (z|x) is used to calculate the mean term. If the channel is complex, multiple sampling of z may be required.

[0126] In summary, the joint decoder with added channels can be modeled through variational inference. The final loss function for training the semantic decoder based on this algorithm model is:

[0127] When the data gradient is propagated backward, the channel noise variance σ2 needs to be known 2 , and then output μ, σ, Substitute it into formula (17) and continuously optimize to complete the training of the parameters θ of the ANN module in the semantic decoder.

[0128] Example 3

[0129] This embodiment proposes a semantic coding method for general channels, and the structure of its model is shown in FIG8 :

[0130] For general channels, it is assumed that the prior probability distribution p(z) in variational inference is the distribution F(α) obeyed by the channel transition probability, where α={v1,v2,...,v k} represents all unknown parameters in the distribution, and k represents the number of vectors v. This parameter can be estimated by the pilot signal: α={v1(x pilot ),v2(x pilot ),…,v k (x pilot )} (18)

[0131] In addition, it is assumed that the output y of the semantic encoder still follows the distribution N(μ1;σ1 2 ), and then the output after the channel Obey the distribution G(N(μ1;σ1 2 ),F(α(x pilot )), the distribution is consistent with F(α) and N(μ1;σ1 2 ) are all related, and the first term in the above formula (4) can be substituted into: min KL(G(N(μ1;σ1 2 ),F(α(x pilot )))||F(α(x pilot ))) (19)

[0132] The form of formula (19) needs to be solved according to different channel noise distributions.

[0133] Furthermore, the output distribution of the semantic encoder can be assumed to be a more general distribution, and its output y is assumed to conform to the distribution G(β), where β = {λ1,λ2,...,λ n} represents the unknown parameters in the distribution, and n represents the number of vectors λ. The actual β parameter is related to the distribution of y and F(α). The β parameter in the semantic communication method can be solved by the ANN module: β={λ1(x),λ2(x),...,λ n (x)} (20)

[0134] Then the first term in the above formula (4) can be derived as: min KL(G(β(x))||F(α(x pilot ))) (twenty one)

[0135] This results in the loss function for training the corresponding model parameters: L = min E x～p(x) [-logq(x|z)+KL(G(β(x))||F(α(x) pilot )))] (twenty two)

[0136] After the parameters in the semantic encoder are trained by the loss function in formula (22), the new channel feature parameter α={v1(x pilot ),v2(x pilot ),…,v k (x pilot )} performs corresponding semantic feature information extraction to obtain the semantic code y to be sent.

[0137] Example 4

[0138] This embodiment proposes a semantic decoding method for general channels, and the structure of its model is shown in FIG8 .

[0139] In the semantic decoder, for the generalized channel, based on Example 2, it is necessary to use the latent variable parameter calculation module to obtain all the parameters that can represent the latent variable distribution β = {λ1,λ2,...,λ n In addition, it is necessary to improve the reparameterization module so that the reparameterization module can use the parameters β={λ1,λ2,...,λ n}, based on the distribution G(β), sample the latent variable z, and finally input the semantic feature information recovery module to obtain the recovery message

[0140] Example 5

[0141] In some application scenarios, the channel may undergo some slight changes during multiple communications. At this time, it is necessary to continuously transmit pilot signals to obtain the corresponding channel information, and then fine-tune the trained semantic encoder according to the channel changes to re-match the changed channel.

[0142] As shown in FIG9 , another semantic communication method according to an embodiment of the present disclosure is applied to an encoding end. The semantic communication method includes S301 to S303 .

[0143] S301: Acquire new channel characteristic parameters.

[0144] As an implementation method, new channel characteristic parameters are obtained by measuring the pilot signal.

[0145] As another implementation manner, new channel characteristic parameters fed back by the decoding end are received.

[0146] In some embodiments, the encoder may periodically measure the pilot signal to obtain channel characteristic parameters. The encoder determines whether the channel has changed by comparing the channel characteristic parameters at different periods. After determining that the channel has changed, the method shown in FIG9 is executed to obtain an encoder suitable for the changed channel.

[0147] S302: Re-determine the loss function based on the new channel characteristic parameters.

[0148] S303: Fine-tune the encoder based on the re-determined loss function to obtain an updated encoder.

[0149] In the embodiment of the present disclosure, the loss function is readjusted, that is, the channel feature parameters involved in the regularization term are readjusted, and then the encoder is fine-tuned to obtain an encoder suitable for the changed channel (that is, updated) to ensure the accuracy of semantic communication in the changed channel.

[0150] In one implementation, the encoder may be fine-tuned based on the overall loss function. In another implementation, the encoder may be fine-tuned based on the regularization term in the loss function.

[0151] This is explained in conjunction with Figure 11. Figure 11 is a flow chart of model fine-tuning in a semantic communication method according to an embodiment of the present disclosure. As shown in Figure 11, the pre-trained model in the figure is the encoder trained by the semantic communication method in the above-mentioned embodiment 3. When the channel conditions change, according to the description of embodiment 3, the regularization term in the loss function of the above formula (4) will also change. Therefore, in the case of channel changes, the device for fine-tuning the model (which can be the encoding end or the decoding end, or other nodes) can obtain new channel characteristic parameters by measuring the pilot signal. On the one hand, the regularization term is adjusted according to the new channel parameters, and then the relevant encoding modules are fine-tuned according to the adjusted regularization term. Since the regularization term can be calculated based on the channel parameters, the semantic encoder latent variable parameter calculation module can be fine-tuned according to the regularization term. On the other hand, the reparameterization module is adjusted according to the new channel parameters. Finally, a new model after fine-tuning is obtained.

[0152] Example 6

[0153] In some application scenarios, the channel may undergo some slight changes during multiple communications. At this time, it is necessary to continuously transmit pilot signals to obtain the corresponding channel information, and then fine-tune the trained semantic decoder according to the channel changes to re-achieve matching with the changed channel.

[0154] As shown in FIG10 , another semantic communication method according to an embodiment of the present disclosure is applied to a decoding end. The semantic communication method includes S401 to S403 .

[0155] In S401, new channel characteristic parameters are obtained.

[0156] As an implementation method, new channel characteristic parameters are obtained by measuring the pilot signal.

[0157] As another implementation manner, the changed channel characteristic parameters of the channel fed back by the encoding end are received.

[0158] In some embodiments, the decoding end may periodically obtain channel characteristic parameters and determine whether the channel has changed by comparing channel characteristic parameters of different periods. After determining that the channel has changed, the method shown in FIG10 is executed to obtain a new channel decoding end.

[0159] In S402, the loss function is re-determined based on the new channel characteristic parameters.

[0160] In S403 , the decoder is fine-tuned based on the re-determined loss function to obtain an updated decoder.

[0161] In some embodiments, S402 may be implemented, for example, as: re-adjusting the loss function based on new channel characteristic parameters.

[0162] In some embodiments, the semantic communication method further includes: adjusting the parameters of the latent variable calculation module and the reparameterization module based on the new channel characteristic parameters to obtain the latent variable calculation module and the reparameterization module suitable for the changed channel.

[0163] In the disclosed embodiment, the decoder is fine-tuned to obtain a decoder suitable for the changed channel, thereby ensuring the accuracy of semantic communication in the changed channel.

[0164] In one implementation, the decoder is fine-tuned based on the modified loss function. In another implementation, the encoder is fine-tuned based on the regularization term in the loss function.

[0165] Continuing with the above explanation in conjunction with Figure 11, similarly, when the channel changes, the device used to fine-tune the model (which can be the encoder or decoder, or other node) can obtain new channel characteristic parameters by measuring pilot signals. First, the regularization term is adjusted based on the new channel parameters, and then the relevant encoding modules are fine-tuned based on the adjusted regularization term. Second, the reparameterization module is adjusted based on the new channel parameters. Finally, a new, fine-tuned model is obtained.

[0166] It should be understood that during modeling, the channel is considered part of the joint encoder, so the latent variable parameter calculation module is essentially part of the joint encoder, but it is located within the semantic decoder. By fine-tuning these modules, the matching between the source code and the channel can be re-achieved. In addition, the parameters of the reparameterization module need to be readjusted based on the new channel parameters.

[0167] It should be noted that in response to changing channels, both the encoder and decoder can be fine-tuned to better ensure the accuracy of semantic communication. Alternatively, only the encoder can be fine-tuned without fine-tuning the decoder to reduce the complexity of the adjustment.

[0168] Since the first term in formula (4) is mainly used to train the semantic encoder so that the source transmission semantics can match the channel characteristic parameters, the semantic encoder can be fine-tuned through the first term in formula (4), i.e., the regularization term, to achieve semantic communication performance recovery for the changing channel without having to retrain the entire model.

[0169] FIG11 is a flow chart of model fine-tuning in a semantic communication method according to an embodiment of the present disclosure. As shown in FIG11 , the pre-trained model in the figure is the codec obtained by training the semantic communication method in the above-mentioned embodiment 4. When the channel conditions change, according to the description of embodiment 4, the regularization term in the loss function of the above-mentioned formula (4) will also change. Therefore, when the channel changes, the device used to fine-tune the model (which can be the encoding end or the decoding end, or other nodes) can obtain new channel parameters by measuring the pilot signal. On the one hand, the regularization term is adjusted according to the new channel parameters to calculate the KL divergence term, and then the relevant encoding module is fine-tuned according to the regularization term KL divergence. Since the regularization term KL divergence can be calculated based on the channel parameters, the semantic encoder and the latent variable parameter calculation module in the semantic decoder can be fine-tuned according to the regularization term. On the other hand, the reparameterization module is adjusted according to the new channel parameters. Finally, a new model after fine-tuning is obtained.

[0170] During modeling, the channel is considered part of the joint encoder, so the latent variable parameter calculation module is also essentially part of the joint encoder. Fine-tuning these modules can re-align the source and channel. Furthermore, the parameters of the reparameterization module must be readjusted based on the new channel parameters.

[0171] It should be noted that one implementation of fine-tuning the relevant modules is to fine-tune all relevant modules. Another implementation is to fine-tune only the relevant modules in the semantic encoder, or to achieve re-matching with the channel.

[0172] In the semantic communication method provided by the embodiment of the present disclosure, by obtaining the original information, the original information is semantically encoded based on the channel characteristic parameters to obtain the first semantic feature information and send it. The first semantic feature information provided for transmission by the present disclosure includes the original information to be transmitted and matches the channel characteristic parameters. In other words, the present disclosure takes into account the characteristic factors of the channel used to transmit data during the actual transmission process in the encoding and decoding process of semantic communication, so that the source characteristics of the channel transmission in the semantic communication are more consistent with the channel characteristics, thereby effectively reducing the amount of data transmitted through the channel while ensuring the accuracy of the semantic communication.

[0173] It is understandable that, in order to implement the above functions, the encoding end and the decoding end include hardware structures and / or software modules corresponding to the execution of each function. It should be readily apparent to those skilled in the art that, in conjunction with the algorithmic steps of each example described in the embodiments of the present disclosure, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present disclosure.

[0174] The embodiments of the present disclosure can divide the functional modules of the communication device according to the above-mentioned method embodiments. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one functional module. The above-mentioned integrated modules can be implemented in the form of hardware or software. It should be noted that the division of modules in the embodiments of the present disclosure is schematic and is only a logical functional division. In actual implementation, there may be other division methods. The following is an example of dividing each functional module corresponding to each function.

[0175] FIG12 is a schematic diagram of the composition of a communication device according to an embodiment of the present disclosure, which can execute the semantic communication method provided by the above method embodiment. As shown in FIG12 , the communication device includes an acquisition module 1201 and a sending module 1202 .

[0176] The acquisition module 1201 is used to acquire original information.

[0177] The acquisition module 1201 is further configured to perform semantic encoding on the original information based on the channel characteristic parameters to obtain first semantic feature information, where the first semantic feature information is used to represent the semantic features of the original information.

[0178] The sending module 1202 is used to send the first semantic feature information.

[0179] In some embodiments, the channel characteristic parameter includes at least one of the following: channel noise variance, channel state information. The channel state information includes at least one of the following: signal-to-noise ratio (SNR), signal-to-interference ratio (SINR), channel matrix, channel quality indicator (CQI), and reference signal received quality (RSRQ).

[0180] In some embodiments, the channel state information is obtained by measuring a pilot signal.

[0181] In some embodiments, the acquisition module 1201 is used, for example, to perform semantic encoding on the original information based on the channel characteristic parameters to obtain first semantic feature information.

[0182] In some embodiments, the acquisition module 1201 is used, for example, to input the original information into an encoder, and the encoder performs semantic encoding on the original information based on channel characteristic parameters to obtain first semantic feature information.

[0183] In some embodiments, the encoder and the decoder matched with the encoder are trained by a loss function obtained based on variational inference, where the loss function includes a regularization term and a reconstruction term.

[0184] In some embodiments, the regularization term is determined based on a source characteristic parameter and a channel characteristic parameter. The source characteristic parameter is used to characterize the distribution characteristics of information sent by the source. The channel characteristic parameter is used to characterize the distribution characteristics of channel transition probabilities.

[0185] In some embodiments, the communication device further includes a determination module 1203. Acquisition module 1201 is further configured to acquire new channel characteristic parameters. Determination module 1203 is configured to redetermine the loss function based on the new channel characteristic parameters. Acquisition module 1201 is further configured to fine-tune the encoder based on the redetermined loss function to obtain an updated encoder.

[0186] In some embodiments, the reconstruction item is determined based on a source characteristic parameter and a sink characteristic parameter. The source characteristic parameter is used to characterize the distribution characteristics of information sent by the source and the sink characteristic parameter is used to characterize the distribution characteristics of information received by the sink.

[0187] In some embodiments, the distribution characteristics of the latent variables in variational inference are channel state transition probability distribution characteristics.

[0188] FIG13 is a schematic diagram of another communication device according to an embodiment of the present disclosure, which can execute the semantic communication method provided by the above method embodiment. As shown in FIG13 , the communication device includes an acquisition module 1301 .

[0189] The acquisition module 1301 is used to acquire second semantic feature information. The second semantic feature information is information obtained after the first semantic feature information sent by the encoder is transmitted through the channel. The first semantic feature information matches the channel characteristic parameters and is used to represent the semantic features of the original information.

[0190] The acquisition module 1301 is further configured to obtain decoding information based on the second semantic feature information.

[0191] In some embodiments, the acquisition module 1301 is configured to, for example, perform semantic decoding on the second semantic feature information based on a channel characteristic parameter to obtain decoded information. In some embodiments, the channel characteristic parameter includes at least one of the following: channel noise variance and channel state information. The channel state information includes at least one of the following: signal-to-noise ratio (SNR), signal-to-interference ratio (SINR), channel matrix, channel quality indicator (CQI), and reference signal received quality (RSRQ).

[0192] In some embodiments, the channel state information is obtained by measuring a pilot signal.

[0193] In some embodiments, the acquisition module 1301 is used, for example, to input the second semantic feature information into a decoder, and the decoder performs semantic decoding on the second semantic feature information based on channel characteristic parameters to obtain decoding information.

[0194] In some embodiments, the decoder is trained using a loss function obtained based on variational inference, where the loss function includes a regularization term and a reconstruction term.

[0195] In some embodiments, the regularization term is determined based on a source characteristic parameter and a channel characteristic parameter. The source characteristic parameter is used to characterize the distribution characteristics of information sent by the source. The channel characteristic parameter is used to characterize the distribution characteristics of channel transition probabilities.

[0196] In some embodiments, the reconstruction item is determined based on a source characteristic parameter and a sink characteristic parameter. The source characteristic parameter is used to characterize the distribution characteristics of information sent by the source and the sink characteristic parameter is used to characterize the distribution characteristics of information received by the sink.

[0197] In some embodiments, the acquisition module 1301 is further used to acquire new channel characteristic parameters; redetermine the loss function based on the new channel characteristic parameters; and fine-tune the decoder based on the redetermined loss function to obtain an updated decoder.

[0198] In some embodiments, the decoder includes a latent variable calculation module, a reparameterization module, and a semantic feature information recovery module connected in sequence. The latent variable calculation module is configured to extract latent variables from the second semantic feature information, the reparameterization module is configured to reparameterize the latent variables, and the semantic feature information recovery module is configured to perform semantic decoding on the reparameterized latent variables to obtain decoded information.

[0199] In some embodiments, the acquisition module 1301 is used, for example, to readjust the loss function based on new channel characteristic parameters.

[0200] In some embodiments, the acquisition module 1301 is further used, for example, to adjust the parameters of the latent variable calculation module and the reparameterization module based on the new channel characteristic parameters to obtain updated latent variable calculation module and reparameterization module.

[0201] In some embodiments, the distribution characteristics of the latent variables in variational inference are channel state transition probability distribution characteristics.

[0202] In the case of implementing the functions of the above-mentioned integrated modules in hardware, the embodiments of the present disclosure provide another structure of the communication device involved in the above-mentioned embodiments. As shown in Figure 14, the communication device 140 includes: a processor 1402 and a bus 1404. In some embodiments, the communication device may also include a memory 1401. In some embodiments, the communication device may also include a communication interface 1403.

[0203] Processor 1402 may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present disclosure. Processor 1402 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the embodiments of the present disclosure. Processor 1402 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP (digital signal processor) and a microprocessor, and the like.

[0204] The communication interface 1403 is used to connect to other devices via a communication network, such as Ethernet, wireless access network, or wireless local area network (WLAN).

[0205] The memory 1401 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0206] As an implementation, memory 1401 may exist independently of processor 1402. Memory 1401 may be connected to processor 1402 via bus 1404 to store instructions or program codes. When processor 1402 calls and executes the instructions or program codes stored in memory 1401, the semantic communication method provided in the embodiments of the present disclosure can be implemented.

[0207] In another implementation, the memory 1401 may also be integrated with the processor 1402 .

[0208] Bus 1404 can be an Extended Industry Standard Architecture (EISA) bus, etc. Bus 1404 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, FIG14 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0209] In some embodiments, the memory 1401 stores executable instructions. When the processor 1402 executes the executable instructions, the communication device executes the semantic communication method as described in any of the above embodiments.

[0210] Some embodiments of the present disclosure provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium), which stores computer program instructions. When the computer program instructions are executed on a computer, the computer executes the semantic communication method described in any of the above embodiments.

[0211] Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes, etc.), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memories (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in the present disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0212] An embodiment of the present disclosure provides a computer program product comprising instructions. When the computer program product is run on a computer, the computer is enabled to execute the semantic communication method described in any one of the above embodiments.

[0213] The semantic communication technology solution provided by the embodiment of the present disclosure obtains the original information, semantically encodes the original information based on the channel characteristic parameters to obtain the first semantic feature information and sends it. The first semantic feature information provided for transmission by the present disclosure includes the original information to be transmitted and matches the channel characteristic parameters. In other words, the present disclosure takes the characteristic factors of the channel used to transmit data into consideration in the encoding and decoding process of semantic communication during the actual transmission process, so that the source characteristics of the channel transmission in the semantic communication are more consistent with the channel characteristics, thereby effectively reducing the amount of data transmitted through the channel while ensuring the accuracy of the semantic communication.

[0214] The above is only a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or replacements within the technical scope disclosed in the present disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A semantic communication method, applied to an encoding end, includes: Obtain the original information; Perform semantic encoding on the original information based on channel characteristic parameters to obtain first semantic feature information, where the first semantic feature information is used to characterize the semantic features of the original information; Transmit the first semantic feature information.

2. The method according to claim 1, wherein The channel characteristic parameters include at least one of the following: channel noise variance, channel state information; the channel state information includes at least one of the following: signal-to-noise ratio SNR, signal-to-interference-plus-noise ratio SINR, channel matrix, channel quality indicator CQI, reference signal received quality RSRQ.

3. The method according to claim 2, wherein, The channel state information is obtained by measuring a pilot signal.

4. The method according to claim 1, wherein The performing semantic encoding on the original information based on channel characteristic parameters to obtain the first semantic feature information includes: Input the original information into an encoder, and perform semantic encoding on the original information based on the channel characteristic parameters through the encoder to obtain the first semantic feature information; Among them, the encoder is trained through a loss function obtained based on variational inference, and the loss function includes a regularization term and a reconstruction term.

5. The method according to claim 4, wherein, The regularization term is determined based on source characteristic parameters and the channel characteristic parameters; where the source characteristic parameters are used to characterize the distribution characteristics of the information sent by the source; the channel characteristic parameters are used to characterize the channel transition probability distribution characteristics.

6. The method according to claim 4, further includes: Obtain new channel characteristic parameters; Based on the new channel characteristic parameters, re-determine the loss function; Based on the re-determined loss function, fine-tune the encoder to obtain an updated encoder.

7. The method according to claim 4, wherein The reconstruction term is determined based on source characteristic parameters and destination characteristic parameters; where the source characteristic parameters are used to characterize the distribution characteristics of the information sent by the source; the destination characteristic parameters are used to characterize the distribution characteristics of the information received by the destination.

8. A semantic communication method, applied to a decoding end, includes: Obtain second semantic feature information, where the second semantic feature information is the information obtained after the first semantic feature information sent by the encoding end is transmitted through the channel, the first semantic feature information matches the channel characteristic parameters, and the first semantic feature information is used to characterize the semantic features of the original information; Based on the second semantic feature information, obtain decoded information.

9. The method according to claim 8, wherein, The obtaining the decoded information based on the second semantic feature information includes: Perform semantic decoding on the second semantic feature information based on the channel characteristic parameters to obtain the decoded information.

10. The method according to claim 9, wherein, The channel characteristic parameters include at least one of the following: channel noise variance, channel state information; the channel state information includes at least one of the following: signal-to-noise ratio SNR, signal-to-interference-plus-noise ratio SINR, channel matrix, channel quality indicator CQI, reference signal received quality RSRQ.

11. The method according to claim 10, wherein, The channel state information is obtained by measuring a pilot signal.

12. The method according to claim 9, wherein, The performing semantic decoding on the second semantic feature information based on the channel characteristic parameters to obtain the decoded information includes: Input the second semantic feature information into a decoder, and through the decoder, based on the channel characteristic parameters, for the second semantic feature information Perform semantic decoding on the information to obtain the decoded information; Among them, the decoder is trained by a loss function obtained based on variational inference, and the loss function includes a regularization term and a reconstruction term.

13. The method according to claim 12, wherein, The regularization term is determined based on the source feature parameters and the channel feature parameters; among them, the source feature parameters are used to characterize the distribution characteristics of the information sent by the source; the channel feature parameters are used to characterize the channel transition probability distribution characteristics.

14. The method according to claim 12, wherein, The reconstruction term is determined based on the source feature parameters and the destination feature parameters; among them, the source feature parameters are used to characterize the distribution characteristics of the information sent by the source; the destination feature parameters are used to characterize the distribution characteristics of the information received by the destination.

15. The method according to claim 12, further comprising: Obtain new channel feature parameters; Based on the new channel feature parameters, re-determine the loss function; Based on the re-determined loss function, fine-tune the decoder to obtain an updated decoder.

16. The method according to claim 12, wherein, The decoder includes a latent variable calculation module, a reparameterization module, and a semantic feature information recovery module connected in sequence; the latent variable calculation module is used to extract the latent variables in the second semantic feature information, the reparameterization module is used to perform reparameterization processing on the latent variables, and the semantic feature information recovery module is used to perform semantic decoding processing on the reparameterized latent variables to obtain decoded information.

17. The method according to claim 15, wherein, The re-determining the loss function based on the new channel feature parameters includes: Based on the new channel feature parameters, re-adjust the loss function.

18. The method according to claim 16, further comprising: Based on the new channel feature parameters, adjust the parameters in the latent variable calculation module and the reparameterization module to obtain an updated latent variable calculation module and reparameterization module.

19. A communication device, wherein, Includes a memory, a processor, and computer program instructions stored on the memory and executable on the processor, and when the processor executes the computer program instructions, it implements the method according to any one of claims 1 to 18.

20. A computer-readable storage medium, wherein, The computer-readable storage medium includes computer program instructions; among them, when the computer program instructions run on a computer, the computer is caused to execute the method according to any one of claims 1 to 18.

Images