Semantic communication method and device and storage medium

Abstract

The embodiment of the invention provides a semantic communication method and device and a storage medium, relates to the technical field of communication, and can realize matching of semantic features and channel features. The method comprises the following steps: a coding end obtains an original message; performing semantic coding on the original information based on the channel feature parameters to obtain first semantic feature information; the first semantic feature information is used for representing semantic features of the original information; the coding end sends the first semantic feature information; the decoding end obtains second semantic feature information after the first semantic feature information is transmitted through the channel; and obtaining decoding information based on the second semantic feature information.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technologies, and in particular, to a semantic communication method, apparatus, and storage medium. Background Art

[0002] With the continuous development of communication technologies, classical communication has almost reached its limit. However, it still does not meet the requirements of various applications, such as high data volume requirements in scenarios like mixed reality and immersive communication. In addition, with the continuous development of artificial intelligence (AI) technologies, the requirements for communication data volume will further increase, and the demand for intelligence in the communication process will become more and more intense. To cope with the huge data volume requirements and to seamlessly connect with various future types of AI applications, semantic communication has emerged.

[0003] Currently, in the training process of semantic communication, an end-to-end training method is generally used to obtain the encoder and decoder of semantic communication. Such a training method does not consider the influence of various factors in the actual transmission process on semantic communication, which will affect the performance of semantic communication. Summary of the Invention

[0004] Embodiments of the present disclosure provide a semantic communication method, apparatus, and storage medium, which can achieve the matching of semantic features and channel features.

[0005] On the one hand, a semantic communication method is provided, which is applied to an encoding end and includes: obtaining original information; performing semantic encoding on the original information based on channel feature parameters to obtain first semantic feature information, where the first semantic feature information is used to represent the semantic features of the original information; and sending the first semantic feature information.

[0006] On the other hand, a semantic communication method is provided, which is applied to a decoding end and includes: obtaining second semantic feature information, where the second semantic feature information is the information obtained after the first semantic feature information sent by the encoding end is transmitted through a channel, the first semantic feature information matches the channel feature parameters, and the first semantic feature information is used to represent the semantic features of the original information; and obtaining decoded information based on the second semantic feature information.

[0007] On still another hand, a communication apparatus is provided, which includes an obtaining module and a sending module. The obtaining module is used to obtain original information; the obtaining module is further used to perform semantic encoding on the original information based on channel feature parameters to obtain first semantic feature information, where the first semantic feature information is used to represent the semantic features of the original information; and the sending module is used to send the first semantic feature information.

[0008] In another aspect, a communication device is provided, including an acquisition module. The acquisition module is configured to acquire second semantic feature information, where the second semantic feature information is the information obtained after the first semantic feature information sent by the encoding end is transmitted through a channel. The first semantic feature information matches the channel feature parameters, and the first semantic feature information is used to represent the semantic features of the original information. The acquisition module is further configured to obtain decoding information based on the second semantic feature information.

[0009] In another aspect, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the semantic communication method described in any of the above embodiments is implemented.

[0010] In another aspect, a computer program product is provided, which includes computer program instructions. When the computer program instructions are executed by a processor, the semantic communication method described in any of the above embodiments is implemented.

[0011] An embodiment of the present disclosure provides a semantic communication method, which obtains an original information, performs semantic encoding on the original information based on channel feature parameters to obtain first semantic feature information and sends it. The first semantic feature information transmitted in the present disclosure includes the original information to be transmitted and matches the channel feature parameters. That is to say, the present disclosure considers the characteristic factors of the channel used for data transmission in the semantic communication encoding and decoding process, so that the source characteristics transmitted through the channel in semantic communication are more in line with the channel characteristics, thereby effectively reducing the amount of data transmitted through the channel on the premise of ensuring the accuracy of semantic communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] To more clearly illustrate the technical solutions in the present disclosure, the following briefly introduces the drawings required to be used in some embodiments of the present disclosure. Obviously, the drawings in the following description are only the drawings of some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings.

[0013] Figure 1 FIG. 18 is a schematic structural diagram of a DeepJSCC model provided in some embodiments of the present disclosure;

[0014] Figure 2 FIG. 22 is a schematic structural diagram of an NTSCC model provided in some embodiments of the present disclosure;

[0015] Figure 3 FIG. 26 is a schematic structural diagram of a VSCC model provided in some embodiments of the present disclosure;

[0016] Figure 4 FIG. 30 is a schematic structural diagram of a communication system provided in some embodiments of the present disclosure;

[0017] Figure 5 Flow diagram of a semantic communication method provided by some embodiments of the present disclosure;

[0018] Figure 6 Flow diagram of another semantic communication method provided by some embodiments of the present disclosure;

[0019] Figure 7 Structural diagram related to a semantic communication method provided by some embodiments of the present disclosure;

[0020] Figure 8 Structural diagram related to another semantic communication method provided by some embodiments of the present disclosure;

[0021] Figure 9 Flow diagram of yet another semantic communication method provided by some embodiments of the present disclosure;

[0022] Figure 10 Flow diagram of yet another semantic communication method provided by some embodiments of the present disclosure;

[0023] Figure 11 Flow diagram of model fine-tuning in a semantic communication method provided by some embodiments of the present disclosure;

[0024] Figure 12 Composition diagram of a communication device provided by some embodiments of the present disclosure;

[0025] Figure 13 Composition diagram of another communication device provided by some embodiments of the present disclosure;

[0026] Figure 14 Structural diagram of a communication device provided by some embodiments of the present disclosure. Detailed implementation manners

[0027] Next, the technical solutions in the present disclosure will be clearly and completely described in conjunction with the accompanying drawings in the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0028] It should be noted that in the present disclosure, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present disclosure should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly, using words such as "exemplary" or "for example" aims to present relevant concepts in a specific manner.

[0029] Hereinafter, the terms "first" and "second" are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features.

[0030] In the description of the present disclosure, unless otherwise specified, " / " means "or". For example, A / B may represent A or B. The "and / or" herein is merely a description of the relationship between associated objects, indicating that there can be three relationships. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, "at least one" means one or more, and "a plurality" means two or more.

[0031] First, relevant terms related to the present disclosure are explained.

[0032] 1. A generative adversarial network (GAN) is a generative model that learns through a game between two neural networks. The generative adversarial network can learn to perform a generation task without using labeled data. The generative adversarial network generally consists of a generator and a discriminator. The generator randomly samples from the latent space as input, and its output result needs to mimic the real samples in the training set as much as possible. The input of the discriminator is either the real sample or the output of the generator, and its purpose is to distinguish the output of the generator from the real samples as much as possible. The generator and the discriminator confront each other and continuously learn, and the ultimate goal is to make the discriminator unable to determine whether the output result of the generator is real.

[0033] 2. A variational auto encoder (VAE), similar to the generative adversarial network, is also designed to solve the data generation problem. In the auto encoder structure, usually an input data is required, and the generated data is the same as the input data. However, it is usually desired that the generated data has a certain degree of difference, which requires an input random vector and the model can learn the stylized characteristics of the generated image. Therefore, in subsequent research, a generative adversarial network structure that uses a randomized vector as input to generate specific samples emerged. The variational auto encoder also uses a random sample with a specific distribution as input and can generate corresponding images. In this regard, its goal is similar to that of the generative adversarial network. However, the variational auto encoder does not require a discriminator but uses an encoder to estimate a specific distribution.

[0034] 3. Semantic communication. Semantic communication is a communication technology that takes tasks as the main body, first understands and then transmits. It first performs selective feature extraction, compression, and transmission on the original signal, and then uses semantic-level information for communication. This technology can significantly improve the transmission efficiency of communication systems.

[0035] Semantic communication is based on the powerful non-linear fitting ability of artificial neural networks (ANNs). For the semantic dimension of the source, further data compression is performed. The implementation of semantic communication generally adopts the architecture of generative adversarial networks or variational autoencoders. In addition, due to the particularity of ANN training, currently, the end-to-end training method is generally used to obtain the encoder and decoder of semantic communication, that is, the joint source channel coding (JSCC) method is used to implement semantic communication.

[0036] On the one hand, based on the JSCC framework, semantic communication combines source coding and channel coding to eliminate the correlation between the source and the channel in the semantic dimension in the case of finite code length or source distortion, thereby obtaining further communication gains, breaking the communication rate limit of the classical separation framework, and increasing the theoretical data transmission capacity of the channel. On the other hand, semantic communication technology is based on ANNs to obtain similar structural information from the source data distribution and the channel transition probability distribution, which is summarized into a knowledge base, and the source is further compressed (encoded) and expanded (decoded) through the knowledge base, thereby realizing more information transmission in the semantic dimension.

[0037] However, although the communication architecture of JSCC is adopted, due to the non-interpretability of ANNs, it is impossible to accurately model and represent the gains obtained by semantic communication from joint coding. Especially, the current implementations of semantic communication are all independent of the channel. There is no variable related to the channel in its loss function, that is, the channel is not considered in the training process, so it is impossible to explicitly explain the relationship between the channel and semantic encoding and decoding, and it is impossible to determine whether semantic communication can generate gains at the channel end. Furthermore, due to the lack of explicit representation of the channel in the loss function, the generalization of the semantic encoder and decoder is poor, and it is also impossible to perform fine-tuning according to different channel characteristics to cope with different channel changes.

[0038] The following takes the DeepJSCC model that conforms to the JSCC communication architecture as an example for illustration. Figure 1Schematic diagram of the structure of a DeepJSCC model provided by an embodiment of the present disclosure. In this model, the joint source-channel coding (semantic encoder) is the discriminator of a deep convolutional generative adversarial network (DCGAN), which is implemented using convolutional layers. The joint source-channel decoding (semantic decoder) is the generator of the DCGAN, which is implemented using transposed convolutional layers.

[0039] Essentially, this model is based on the auto encoder (AE) framework. Its source coding and channel coding adopt a joint coding method and are implemented based on the discriminator of the DCGAN. Its channel decoding and source decoding parts adopt a joint decoding method and are implemented based on the generator of the DCGAN.

[0040] The loss function during the training of this model is the mean square error (MSE) between the input data and the output data:

[0041]

[0042] This model has the following problems: On the one hand, it is based on the idea of fixed-length coding and does not consider that different sources may have semantic feature information of different lengths. Therefore, the problem of variable-length coding needs to be solved. On the other hand, it is based on the AE architecture to extract semantic feature information from the source, but it does not model the channel during the modeling process, and the channel feature parameters are not specifically reflected in the loss function. Therefore, it is impossible to well adapt the source features to the channel feature parameters.

[0043] The nonlinear transform source-channel coding (NTSCC) model is proposed in the related technology to solve the problem of variable-length coding. Based on the DeepJSCC model, the NTSCC model additionally adds a semantic feature extraction module and a semantic feature fusion module for the source, which are implemented using nonlinear source coding technology. Its structure is shown in Figure 2 as follows.

[0044] In the NTSCC model, by adding a semantic feature extraction module, nonlinear semantic information fitting and modeling can be completed for different sources, so as to obtain the coding rate determined by the source semantics and finally obtain semantic coding. The semantic feature extraction module is implemented in a nonlinear compression manner and can be implemented based on architectures such as Transformer and neural networks, so as to realize the transmission of text or images.

[0045] However, during the modeling process of the NTSCC model, the channel modeling is still not considered. The loss function of this model is based on the loss function of the VAE model, that is, obtained from the KL divergence of the joint distribution. However, during the derivation process, the channel is regarded as a constant, thus treating the posterior probability of the channel output value as a constant, which makes the entire VAE model degenerate into an AE model. Its final loss function is:

[0046]

[0047] In the formula, η and β represent constants (weight coefficients), and d LPIPS , D represent two metrics for measuring image similarity. Therefore, the NTSCC model also does not truly achieve incorporating the channel into semantic communication for modeling, but only makes a further fitting of the semantic feature information of the source.

[0048] To address the problems existing in the above model, in order to incorporate the channel into the consideration scope of semantic communication, better explain the role of the channel in JSCC, and at the same time eliminate the correlation between the source and the channel in the semantic dimension and further explore the potential of semantic communication, this disclosure proposes a semantic communication method based on the variational inference-based JSCC model, also known as the variational source channel coding (VSCC) model. In this method, the channel noise distribution becomes the core of variational inference and is a key consideration factor for deriving the latent variable. In addition, since the derivation of the latent variable is based on the ANN training process and the source distribution also needs to be considered, both the source characteristics and the channel characteristics are considered in this semantic communication method, making the extracted source semantic features match the channel noise, thereby effectively reducing the amount of data transmitted through the channel while ensuring the accuracy of semantic communication.

[0049] Figure 3 FIG. is a schematic structural diagram of a VSCC model provided by an embodiment of this disclosure. As Figure 3 shown, the input of the above VSCC model is x, which first passes through a joint pre-encoder f en (x; θ) to obtain a codeword y, where θ is the trainable parameter of the joint pre-encoder. Then it passes through the channel, and the output is denoted as z. Here, the mapping function corresponding to the channel input and output is denoted as where represents the relevant parameters of this mapping function. Finally, it passes through a joint decoder f de (z; φ), where φ is the trainable parameter of the joint decoder, to obtain the final output

[0050] To train the encoder and decoder in the above model, mathematical modeling needs to be carried out on them. As Figure 3As shown, in the model, the data flow undergoes two changes. One is joint source-channel coding, and the other is joint source-channel decoding. These two changes can be described by the posterior probabilities p z|x (z|x) and q x|z (x|z) shown in the figure.

[0051] In the VSCC model, the purpose is to make the distribution of the information obtained by the decoder (destination) as close as possible to the distribution p(x) of the message x sent by the encoder (source), so as to achieve the purpose of semantic transmission. It should be noted that when is close to p(x), the data sampled from both can be considered as x. Therefore, q x (x) can be used to replace for subsequent derivations.

[0052] To achieve the purpose of making the two distributions close, the KL divergence can be used to measure the distance between the distributions, so as to minimize the KL divergence between the source and the destination, that is:

[0053] min KL(p x (x)||q x (x)) (1)

[0054] Under the assumptions of the VSCC model, the above formula (1) can be further derived through the KL divergence of the joint probability density, because the KL divergence of the joint probability density is the upper bound of the KL divergence of the marginal distributions, that is:

[0055]

[0056] Therefore, the KL divergence of the marginal distributions, that is, formula (1), can be indirectly optimized by minimizing formula (2). Based on the variational inference principle, formula (2) can be further optimized:

[0057]

[0058] where Since p x (x) is the known fixed input message distribution and is therefore a constant, it does not play a role in the training process of the joint encoder-decoder in the VSCC model and can thus be discarded. Further derivation:

[0059]

[0060] In formula (4), the first term corresponds to the joint encoder p z|x (z|x). The second term corresponds to the joint decoder q x|z(x|z). Based on formula (4), an embodiment of the present disclosure proposes a semantic communication method that can fuse source features and channel features.

[0061] In the embodiment of the present disclosure, the network architecture of a mobile communication system (including but not limited to 5G and future 6G mobile communication systems) may include an encoding end and a decoding end. In some examples, the encoding end may be a base station, and the decoding end may be a terminal / UE. In other examples, the encoding end may be a UE, and the decoding end may be a base station. The embodiment of the present disclosure is not limited thereto.

[0062] Hereinafter, an example in which the encoding end is a base station and the decoding end is a terminal will be described.

[0063] Figure 4 It is a schematic structural diagram of a communication system provided by an embodiment of the present disclosure. As Figure 4 shown, the communication system 40 includes a base station 41 and a terminal 42. Among them, the base station 41 and the terminal 42 can be connected through a wireless network.

[0064] In some embodiments, the base station 41 is used to provide wireless access services for the terminal 42. Specifically, one base station 41 provides a service coverage area (also called a cell). The terminal 42 entering this area can communicate with the base station 41 through a wireless signal to receive the wireless access service provided by the base station 41.

[0065] In some embodiments, the base station 41 may be a millimeter wave base station, an evolved Node B (eNB), a next-generation Node B (gNB), a transmission receive point (TRP), a transmission point (TP), and some other access nodes. According to the size of the provided service coverage area, the base station 41 can be further divided into a macro base station for providing a macro cell, a micro base station for providing a picocell, and a femto base station for providing a femto cell. With the continuous evolution of wireless communication technology, future base stations may also use other names. The signal coverage range of the base station 41 also includes the near field and the far field, and the terminal 42 can be within the near field range or within the far field range.

[0066] In some embodiments, the terminal 42 may be a device with wireless transceiver functions, such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The specific type of the terminal 42 is not limited in the embodiments of the present invention.

[0067] It should be understood that Figure 4 is an exemplary structural diagram, Figure 4 the number of devices included in the illustrated communication system is not limited. For example, the number of base stations is not limited and the number of terminals is not limited. And, in addition to Figure 4 the devices shown, Figure 1 the illustrated communication system may further include other devices, which are not limited herein.

[0068] Figure 5 The flowchart of a semantic communication method provided by an embodiment of the present disclosure is shown. The semantic communication method provided by the present disclosure may be applied to an encoding end, and may be exemplarily applied to Figure 4 the base station of the illustrated communication system.

[0069] As Figure 5 shown, the semantic communication method provided by the present disclosure may specifically include the following steps:

[0070] S501. Obtain the original information.

[0071] S502. Perform semantic encoding on the original information based on the channel characteristic parameters to obtain the first semantic feature information.

[0072] S503. Transmit the first semantic characteristic information.

[0073] Wherein, the first semantic feature information is used to characterize the semantic features of the original information.

[0074] In the embodiments of the present disclosure, the data volume of the first semantic feature information is smaller than that of the original information, which is beneficial to reducing the transmission overhead.

[0075] In some embodiments, the original information may be control signaling or service data, which is not limited herein.

[0076] In some embodiments, the channel characteristic parameters include at least one of the following: channel noise variance, channel state information; the channel state information includes at least one of the following: signal-to-noise ratio (SNR), signal to interference plus noise ratio (SINR), channel matrix, channel quality indication (CQI), reference signal received quality (RSRQ).

[0077] In some embodiments, the channel state information is obtained by measuring a pilot signal.

[0078] In some embodiments, the above S502 can be implemented as: the encoding end can perform semantic encoding on the original information based on the channel characteristic parameters to obtain first semantic feature information.

[0079] In some embodiments, the encoding end can perform semantic encoding on the original information based on the channel characteristic parameters to obtain first semantic feature information, which can be specifically implemented as: the encoding end can input the original information into an encoder, and the encoder performs semantic encoding on the original information based on the channel characteristic parameters to obtain first semantic feature information.

[0080] In the embodiments of the present disclosure, the above encoder can be referred to as a semantic encoder, or other names, which are not limited herein.

[0081] It should be understood that the semantic encoding here is to re-arrange the original message according to the channel transition probability distribution. The semantic encoding process is to encode the original message data into first semantic feature information. The variables in each dimension of the first semantic feature information are approximately the channel transition probability distribution. However, in the subsequent process, all the dimensional variables pass through the channel to obtain second semantic feature information. The second semantic feature information is also a latent variable in variational inference, and its distribution is approximately the channel transition probability distribution. The second semantic feature information can be restored to the original message distribution after passing through a decoder that matches the channel.

[0082] The first semantic feature information is the data that has not been affected by channel noise before passing through the channel and matches the channel transition probability. The matching means that after passing through the channel transmission, second semantic feature information approximately the channel transition probability distribution can be obtained. In some embodiments, the encoder can be trained by the encoding end itself, or can be configured for the encoding end after being trained by other nodes (such as core network elements or servers).

[0083] In some embodiments, the above-mentioned encoder (which may also be referred to as a semantic encoder) can be trained by a loss function obtained based on variational inference. The loss function includes a regularization term and a reconstruction term. For the specific content of the loss function, reference can be made to the relevant descriptions in Embodiment 1 or Embodiment 2 below, and details will not be elaborated here.

[0084] In some embodiments, the above-mentioned regularization term can be determined based on source feature parameters and channel feature parameters. Among them, the source feature parameters are used to characterize the distribution characteristics of the information sent by the source. The channel feature parameters are used to characterize the channel transition probability distribution characteristics. For the specific content of the regularization term, reference can be made to the relevant descriptions in Embodiment 1 or Embodiment 2 below, and details will not be elaborated here.

[0085] In some embodiments, the distribution characteristics of the latent variables in variational inference are the channel state transition probability distribution characteristics.

[0086] In some embodiments, the above-mentioned reconstruction term can be determined based on source feature parameters and destination feature parameters. Among them, the source feature parameters are used to characterize the distribution characteristics of the information sent by the source. The destination feature parameters are used to characterize the distribution characteristics of the information received by the destination. For the specific content of the reconstruction term, reference can be made to the relevant descriptions in Embodiment 1 or Embodiment 2 below, and details will not be elaborated here.

[0087] In some scenarios, during multiple communication processes, the channel may undergo some minor changes. At this time, it is necessary to continuously transmit pilot signals to obtain the corresponding channel information, and then, according to the change of the channel, finely adjust the trained semantic codec to re-match the changing channel.

[0088] In the embodiments of the present disclosure, since the first semantic feature information extracted from the original information matches the channel feature parameters, the stability of the first semantic feature information during channel transmission is ensured, which is beneficial to improving the communication efficiency of semantic communication.

[0089] Figure 6 It is a schematic flowchart of a semantic communication method provided by an embodiment of the present disclosure. The semantic communication method provided by the present disclosure is applied to the decoding end and can be exemplarily applied to Figure 4 the terminal of the communication system shown.

[0090] As Figure 6 shown, the semantic communication method provided by the present disclosure may specifically include the following steps:

[0091] S601. Obtain the second semantic feature information.

[0092] S602. Obtain the decoded information based on the second semantic feature information.

[0093] In some embodiments, the second preset feature information is the information obtained after the first semantic feature information sent by the encoding end is transmitted through a channel, and the first semantic feature information matches the channel feature parameters.

[0094] In some embodiments, the above S602 can be implemented as: performing semantic decoding on the second semantic feature information based on the channel feature parameters to obtain decoded information.

[0095] In some embodiments, the decoding end can perform semantic decoding on the original information based on the channel feature parameters to obtain decoded information, which can be specifically implemented as: inputting the second semantic feature information into a decoder, and the decoder performs semantic decoding on the second semantic feature information based on the channel feature parameters to obtain decoded information.

[0096] It should be understood that the semantic decoding mentioned here is to decode the second semantic feature information according to the channel transition probability distribution. The semantic decoding process is to restore the second semantic feature information to the original message distribution after decoding by a decoder matching the channel. Among them, the second semantic feature information is data that approximates the channel transition probability distribution after passing through the channel. The decoder matching the channel refers to a decoder trained by a loss function including channel feature parameters.

[0097] In the embodiments of the present disclosure, the above decoder may be referred to as a semantic decoder, or other names, which are not limited herein.

[0098] In some embodiments, the decoder can be trained by the decoding end itself, or can be configured for the decoding end after being trained by other nodes (such as core network elements or servers).

[0099] In some embodiments, the decoder is trained by a loss function obtained based on variational inference, and the loss function includes a regularization term and a reconstruction term.

[0100] Among them, the relevant descriptions of the regularization term and the reconstruction term in the loss function can refer to the relevant descriptions of the encoding end, or the relevant descriptions below, and will not be repeated here.

[0101] In some embodiments, the decoder includes a latent variable calculation module, a reparameterization module, and a semantic feature information recovery module connected in sequence. Among them, the latent variable calculation module is used to extract the latent variable in the second semantic feature information. The reparameterization module is used to perform reparameterization processing on the latent variable. The semantic feature information recovery module is used to perform semantic decoding processing on the reparameterized latent variable to obtain decoded information.

[0102] An embodiment of the present disclosure includes a semantic communication method based on a VSCC model. In this method, the matching between the source and the channel is realized, and through the matching of the distribution characteristics of the two, the semantic feature information is extracted, and finally a semantic encoder and a semantic decoder that match the channel are obtained. Specifically, the semantic encoder uses the channel noise distribution as the latent variable distribution in variational inference to perform semantic segmentation on the source input to obtain semantic feature information, and its decoder restores the received semantic feature information according to the corresponding semantic segmentation method, and finally completes the communication.

[0103] The above encoder and its corresponding decoder will be described in detail below in conjunction with specific embodiments and the accompanying drawings of the specification. Embodiments 1 and 2 are described by taking the Gaussian channel as an example, and Embodiments 3 and 4 are described by taking the general channel as an example.

[0104] Embodiment 1

[0105] This embodiment proposes a semantic coding method for the Gaussian channel, and its model is as Figure 7 shown. As shown in the structure of Figure 7 , the semantic encoder is implemented in the form of a joint encoder, and the joint encoder can be implemented by an ANN module. The ANN module can be implemented by a Transformer layer, a CNN layer, an RNN layer, or a Dense layer, etc., so as to facilitate the transmission of information in different modalities. In the process of data forward transmission, the original information x is encoded by the semantic encoder to obtain the first semantic feature information y with a data volume smaller than the original information x.

[0106] First of all, the original information x passes through the semantic encoder. After being trained by the loss function of variational inference, the encoder mainly has two coding features, namely the regularization term and the reconstruction term.

[0107] On the one hand, the encoder performs semantic encoding on the original information x in the direction of the channel noise distribution, so as to obtain the first semantic feature information y that can fit the channel characteristic parameters. This ability is determined by the first term in the above formula (4) and can be called the regularization term. This term can make the coding latent variable tend to the channel noise distribution. In other words, this term enables the encoder to use the channel noise distribution to divide and extract the latent variable of the input information, so that the encoder tends to use the channel characteristic parameters to extract the semantic features of the source.

[0108] In one implementation, in order to achieve the above coding effect, it is necessary to make assumptions about the posterior probability and the prior probability in the regularization term. For the posterior probability p z|x (z|x), for the convenience of calculation, it is assumed here that the output y of the semantic encoder follows a Gaussian distribution N(μ1,σ1 2 ). The channel is an additive Gaussian noise channel, so the channel output is Among them, n~N(0,σ2 2 ) is the noise that conforms to Gaussian distribution, then there is After receiving the message They are respectively transmitted to the mean calculation module and the variance calculation module to obtain the hidden variable mean μ and variance σ calculated by the model:

[0109] μ=μ1+0 (5)

[0110] σ 2 =σ1 2 +σ2 2 (6)

[0111] The mean and variance are then fed into the reparameterization module, which calculates the corresponding latent variable z:

[0112] z=ε·σ+μ (7)

[0113] Among them, ε~N(0;1) is a random number that conforms to the standard Gaussian distribution.

[0114] It should be noted that this embodiment assumes a continuous Gaussian distribution. If the latent variable is a discrete distribution, the reparameterization needs to be performed in a discrete distribution manner, such as Gumbel-Softmax reparameterization, which will not be described here. The latent variable calculated is finally restored through the semantic decoder to obtain the restored message

[0115] According to the properties of Gaussian distribution random variables, the posterior probability distribution of the hidden variable z received by the decoder is:

[0116] p z|x (z|x)~N(μ=μ1+0,σ 2 =σ1 2 +σ2 2 ) (8)

[0117] For the prior probability q z (z), in order to include the influence of the channel, it can be assumed that the true hidden variable z obeys the channel transition probability distribution N(0,σ2 2 ), so far all the assumptions in variational inference are completed, and the regularization term can be solved:

[0118]

[0119] Finally, the loss function used to train the encoder part can be written as:

[0120]

[0121] It can be seen that the loss function contains both the channel noise variance and is related to the source distribution. Therefore, the semantic encoder considers both the source and channel characteristic parameters during the encoding process and can use the channel characteristic parameters to perform semantic encoding on the source.

[0122] On the other hand, the encoder simultaneously considers the distribution characteristics of the original information x and the distribution characteristics of the destination information in order to achieve the best semantic feature extraction method, which is determined by the second term in formula (4) and can be called the reconstruction term. The reconstruction term makes the data at the sending end and the receiving end as close as possible, thereby restricting the encoder to encode based on the distribution characteristics of the original information x, so as to retain the semantic feature information of the source.

[0123] As shown above, the joint encoder with a channel added can be modeled through variational inference, and the final loss function for training the semantic encoder based on this algorithm model is:

[0124]

[0125] When the data gradient is propagated backward, the channel noise variance σ2 needs to be known 2 , and then each output is brought into formula (17) for continuous optimization to complete the training of the parameters θ of the ANN module in the semantic encoder.

[0126] Embodiment 2

[0127] This embodiment proposes a semantic decoding method for a Gaussian channel, and its model is as Figure 7 shown. As shown in the structure in Figure 7 , the semantic decoder is implemented in the form of a joint decoder, and the joint decoder can be implemented by an ANN module. The ANN module can be implemented by a Transformer layer, a CNN layer, an RNN layer, a Dense layer, and different activation functions, etc., in order to achieve the decoding of different modalities of information. During the forward transmission of data, the encoded information is transmitted through the channel and becomes the second semantic feature information received at the decoding end Finally, it is restored by the semantic decoder to obtain the decoding message at the semantic level

[0128] For the decoder, on the one hand, it is determined by the first term in formula (4), and it will perform decoding considering the channel characteristic parameters. On the other hand, it is determined by the second term in formula (4), which restricts the decoder to perform decoding based on the distribution characteristics of the original information x.

[0129] To achieve the above decoding effect, it is necessary to determine the reconstruction term. This reconstruction term can be solved by referring to whether the source distribution to be restored is discrete or continuous, so as to obtain semantic communication decoding methods for different information modalities. Two examples are given here. For instance, assume that the probability distribution of the information x to be decoded follows a Bernoulli distribution, that is, x can only take 1 or 0:

[0130]

[0131] The decoder is responsible for calculating the specific probability ρ(z) of the above distribution. At this time, it is equivalent to making a classification prediction for discrete x, which can be used as a semantic communication method for transmitting binary images. Assume that the binary image contains D pixels. The second term in the above formula can be calculated as:

[0132]

[0133] Furthermore, if it is assumed that the probability distribution of x is a discrete distribution with D variables, a semantic communication method for text transmission can be obtained according to a similar calculation process (assuming that the number of different characters contained in the text is D), which will not be elaborated here.

[0134] If it is assumed that the probability distribution of x is a Gaussian distribution, the decoder can also be used to calculate the mean and variance of this distribution to obtain N(μ(z),σ 2 (z)). At this time, it is equivalent to generating for continuous x, which can be used as a semantic communication method for transmitting images with D continuous-valued pixel points. The above formula (12) can be calculated as MSE:

[0135]

[0136] The above formula (13) can be calculated as:

[0137]

[0138] Assuming that the variance in the input data is a constant, we can obtain:

[0139]

[0140] It should be noted that in the above process of solving the reconstruction term, taking the example of sampling the latent variable z once, the mean term of p z|x (z|x) is omitted before. In a complex channel situation, it may be necessary to sample z multiple times.

[0141] In summary, through variational inference, the joint decoder with the channel can be modeled. Based on this algorithm model, the final loss function for training the semantic decoder is:

[0142]

[0143] When the data gradient is propagated backward, the channel noise variance σ2 needs to be known. 2 , and then each output is substituted into formula (17) for continuous optimization to complete the training of the parameters θ of the ANN module in the semantic decoder.

[0144] Embodiment 3

[0145] This embodiment proposes a semantic coding method for a general channel, and the structure of its model is as Figure 8 shown:

[0146] For a general channel, assume that the prior probability distribution p(z) in variational inference is the distribution F(α) followed by the channel transition probability, where α = {v1, v2,..., v k} represents all the unknown parameters in this distribution, and these parameters can be estimated through pilot signals:

[0147]

[0148] In addition, assume that the output y of the semantic encoder still follows the distribution N(μ1; σ1 2 ), and the output after passing through the channel 2 follows the distribution G(N(μ1; σ1 pilot ), F(α(x 2 ))), and this distribution is related to both F(α) and N(μ1; σ1

[0149] min KL(G(N(μ1; σ1 2 ), F(α(x pilot )))||F(α(x pilot ))) (19)

[0150] The specific form of this formula (19) needs to be solved according to different channel noise distributions.

[0151] Furthermore, the output distribution of the semantic encoder can be assumed to be a more general distribution. Let its output y conform to the distribution G(β), where β = {λ1, λ2,..., λ n} represents the unknown parameters in this distribution, and the true β parameters are related to the distribution followed by y and F(α). The β parameters in the semantic communication method can be solved through the ANN module:

[0152] β = {λ1(x), λ2(x),..., λ n (x)} (20)

[0153] Then the first term in the above formula (4) can be derived:

[0154] min KL(G(β(x))||F(α(x pilot ))) (21)

[0155] Thus, the loss function for training the corresponding model parameters is obtained as follows:

[0156] L = min E x～p(x) [-logq(x|z)+KL(G(β(x))||F(α(x pilot )))] (22)

[0157] After the parameters in the semantic encoder are trained by the loss function in formula (22), based on the distribution characteristics of the original information x, the new channel feature parameters α = {v1(x pilot ), v2(x pilot ),..., v3(x pilot )} are used to extract the corresponding semantic feature information, so as to obtain the semantic code y to be sent.

[0158] Example 4

[0159] This example proposes a semantic decoding method for a general channel, and the structure of its model is as Figure 8 shown:

[0160] In the semantic decoder, for a general channel, based on Example 2, it is necessary to use the latent variable parameter calculation module to obtain all the parameters β = {λ1, λ2,..., λ n} that can represent the latent variable distribution. In addition, it is also necessary to improve the reparameterization module so that the reparameterization module can use the parameters β = {λ1, λ2,..., λ n}, based on the distribution G(β), sample the latent variable z, and finally input it into the semantic feature information recovery module to obtain the recovered message

[0161] Example 5

[0162] In some application scenarios, during multiple communication processes, the channel may undergo some minor changes. At this time, it is necessary to continuously transmit pilot signals to obtain the corresponding channel information, and then fine-tune the trained semantic encoder according to the channel changes to re-match the changing channel.

[0163] As Figure 9 shown, an embodiment of the present disclosure provides a semantic communication method, which is applied to the encoding end. The method includes:

[0164] S301. Obtain new channel feature parameters.

[0165] As a possible implementation, new channel characteristic parameters are obtained by measuring pilot signals.

[0166] As another possible implementation, new channel characteristic parameters fed back by the receiving and decoding end are received.

[0167] In some embodiments, the encoding end may periodically measure pilot signals to obtain channel characteristic parameters. By comparing the channel characteristic parameters of different periods, it is determined whether the channel has changed. Then, after determining that the channel has changed, the Figure 9 shown method is executed to obtain an encoder applicable to the changed channel.

[0168] S302. Based on the new channel characteristic parameters, re-determine the loss function.

[0169] S303. Based on the re-determined loss function, fine-tune the encoder to obtain an updated encoder.

[0170] In the embodiments of the present disclosure, the loss function is re-adjusted, that is, the channel characteristic parameters involved in the regularization term are re-adjusted, and then the encoder is fine-tuned to obtain an encoder applicable to the changed channel (i.e., an updated encoder) to ensure the accuracy of semantic communication in a changing channel.

[0171] In one implementation, when fine-tuning the encoder, it can be fine-tuned according to the overall loss function. In another implementation, when fine-tuning the encoder, it can be fine-tuned according to the regularization term in the loss function.

[0172] Combined with Figure 11 for illustration. Figure 11 is a schematic flowchart of model fine-tuning in a semantic communication method provided by an embodiment of the present disclosure. As Figure 11 shown, the pre-trained model in the figure is the encoder obtained by training the semantic communication method in the above-mentioned Embodiment 3. When the channel condition changes, according to the description of Embodiment 3, the regularization term in the loss function of the above formula (4) also changes. Therefore, in the case of channel change, the device for fine-tuning the model (which can be the encoding end or the decoding end, or other nodes) can obtain new channel characteristic parameters by measuring pilot signals. On the one hand, the regularization term is adjusted according to the new channel parameters, and then the relevant encoding module is fine-tuned. Since the regularization term can be calculated according to the channel parameters, the latent variable parameter calculation module of the semantic encoder can be fine-tuned according to this regularization term. On the other hand, the reparameterization module is adjusted according to the new channel parameters. Finally, a fine-tuned new model is obtained.

[0173] Embodiment 6

[0174] In some application scenarios, during multiple communication processes, the channel may undergo some minor changes. At this time, it is necessary to continuously transmit pilot signals to obtain the corresponding channel information, and then, according to the changes in the channel, finely adjust the trained semantic decoder to re-match the changing channel.

[0175] As Figure 10 shown, an embodiment of the present disclosure provides a semantic communication method applied to a decoding end. The method includes:

[0176] S401. Obtain new channel characteristic parameters.

[0177] As a possible implementation, new channel characteristic parameters are obtained by measuring pilot signals.

[0178] As another possible implementation, receive the channel characteristic parameters after the change of the channel fed back by the encoding end.

[0179] In some embodiments, the decoding end can periodically obtain channel characteristic parameters. By comparing the channel characteristic parameters of different periods to determine whether the channel has changed, and then, after determining that the channel has changed, execute the Figure 10 shown method to obtain the decoding end of the new channel.

[0180] S402. Based on the new channel characteristic parameters, re-determine the loss function.

[0181] S403. Based on the re-determined loss function, finely adjust the decoder to obtain an updated decoder.

[0182] In some embodiments, step S402 can be specifically implemented as: based on the new channel characteristic parameters, adjust the latent variable calculation module and the reparameterization module to obtain the latent variable calculation module and the reparameterization module applicable to the changed channel.

[0183] In the embodiments of the present disclosure, the decoder is finely adjusted to obtain a decoder applicable to the changed channel to ensure the accuracy of semantic communication in the changing channel.

[0184] In one implementation, when finely adjusting the decoder, it can be finely adjusted according to the corrected loss function. In another implementation, when finely adjusting the encoder, it can be finely adjusted according to the regularization term in the loss function.

[0185] Continue to combine the above Figure 11A description will be given. Similarly, in the case of channel variation, the device for fine-tuning the model (which can be the encoding end or the decoding end, or other nodes) can obtain new channel characteristic parameters by measuring pilot signals. On the one hand, adjust the regularization term according to the new channel parameters, and then fine-tune the relevant encoding module accordingly. On the other hand, adjust the reparameterization module according to the new channel parameters. Finally, a new fine-tuned model is obtained.

[0186] It should be understood that during the modeling process, the channel is regarded as part of the joint encoder. Therefore, the latent variable parameter calculation module is essentially part of the joint encoder, but it is located within the semantic decoder. By fine-tuning these modules, the matching between source decoding and the channel can be re-implemented. In addition, the parameters of the reparameterization module also need to be readjusted according to the new channel parameters.

[0187] It should be noted that for a changing channel, both the encoder and the decoder can be fine-tuned to better ensure the accuracy of semantic communication. Or, only the encoder can be fine-tuned without fine-tuning the decoder to reduce the complexity of the adjustment.

[0188] Since the first term in formula (4) is mainly used to train the semantic encoder to make the source transmission semantics match the channel characteristic parameters, the semantic encoder can be fine-tuned through the first term in formula (4), that is, the regularization term, so as to restore the semantic communication performance for a changing channel without having to retrain the entire model.

[0189] Figure 11 This is a schematic diagram of the process of model fine-tuning in a semantic communication method provided by an embodiment of the present disclosure. As Figure 11 shown, the pre-trained model in the figure is the encoder and decoder obtained by training the semantic communication method in the above-mentioned embodiment 4. When the channel condition changes, according to the description of embodiment 4, the regularization term in the above formula (4) loss function also changes. Therefore, in the case of channel variation, the device for fine-tuning the model (which can be the encoding end or the decoding end, or other nodes) can obtain new channel parameters by measuring pilot signals. On the one hand, adjust the regularization term to calculate the KL divergence term according to the new channel parameters, and then fine-tune the relevant encoding module according to the KL divergence of the regularization term. Since the KL divergence of this regularization term can be calculated based on the channel parameters, the semantic encoder and the latent variable parameter calculation module in the semantic decoder can be fine-tuned according to this regularization term. On the other hand, adjust the reparameterization module according to the new channel parameters. Finally, a new fine-tuned model is obtained.

[0190] During the modeling process, the channel is regarded as part of the joint encoder. Therefore, the latent variable parameter calculation module is also essentially part of the joint encoder. By fine-tuning these modules, the matching between the source and the channel can be re-implemented. In addition, the parameters of the reparameterization module need to be readjusted according to the new channel parameters.

[0191] It should be noted that for fine-tuning the relevant modules, one implementation method is to fine-tune all relevant modules. In another implementation method, only the relevant modules in the semantic encoder can be fine-tuned, and the re-matching with the channel can also be achieved.

[0192] The semantic communication method provided by the embodiments of the present disclosure provides a semantic communication method. By obtaining the original information, semantic encoding is performed on the original information based on the channel characteristic parameters to obtain the first semantic feature information and then transmitted. The first semantic feature information transmitted by the present disclosure includes the original information to be transmitted and matches the channel characteristic parameters. That is to say, the present disclosure takes into account the characteristic factors of the channel used for data transmission in the encoding and decoding process of semantic communication, so that the source characteristics transmitted by the channel in semantic communication are more in line with the channel characteristics, thereby effectively reducing the amount of data transmitted by the channel while ensuring the accuracy of semantic communication.

[0193] It can be understood that in order to implement the above functions, the encoding end and the decoding end include the corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should easily realize that, combined with the algorithm steps of each example described in the embodiments of the present disclosure, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0194] The embodiments of the present disclosure can divide the communication device into functional modules according to the above method embodiments. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one functional module. The above integrated module can be implemented in the form of hardware or software. It should be noted that the division of modules in the embodiments of the present disclosure is illustrative, only a logical function division, and there may be other division methods in actual implementation. The following takes the example of dividing each functional module corresponding to each function for illustration.

[0195] Figure 12 It is a schematic diagram of the composition of a communication device provided by the embodiments of the present disclosure. The communication device can execute the semantic communication method provided by the above method embodiments. As Figure 12As shown, an acquisition module 1201 and a transmission module 1202.

[0196] The acquisition module 1201 is configured to acquire original information;

[0197] The acquisition module 1201 is further configured to perform semantic encoding on the original information based on channel characteristic parameters to obtain first semantic feature information, where the first semantic feature information is used to characterize the semantic features of the original information;

[0198] The transmission module 1202 is configured to transmit the first semantic feature information.

[0199] In some embodiments, the channel characteristic parameters include at least one of the following: channel noise variance, channel state information; the channel state information includes at least one of the following: signal-to-noise ratio SNR, signal-to-interference-plus-noise ratio SINR, channel matrix, channel quality indicator CQI, reference signal received quality RSRQ.

[0200] In some embodiments, the channel state information is obtained by measuring pilot signals.

[0201] In some embodiments, the acquisition module 1201 is specifically configured to perform semantic encoding on the original information based on channel characteristic parameters to obtain first semantic feature information.

[0202] In some embodiments, the acquisition module 1201 is specifically configured to input the original information into an encoder, and the encoder performs semantic encoding on the original information based on channel characteristic parameters to obtain first semantic feature information.

[0203] In some embodiments, the encoder and the decoder matched with the encoder are trained by a loss function obtained based on variational inference, and the loss function includes a regularization term and a reconstruction term.

[0204] In some embodiments, the regularization term is determined based on source characteristic parameters and channel characteristic parameters; wherein, the source characteristic parameters are used to characterize the distribution characteristics of the information sent by the source; the channel characteristic parameters are used to characterize the channel transition probability distribution characteristics.

[0205] In some embodiments, the above device further includes a determination module 1203. The acquisition module 1201 is further configured to acquire new channel characteristic parameters; the determination module 1203 is configured to re-determine the loss function based on the new channel characteristic parameters; the acquisition module 1201 is further configured to fine-tune the encoder based on the re-determined loss function to obtain an updated encoder.

[0206] In some embodiments, the reconstruction term is determined based on source characteristic parameters and destination characteristic parameters; wherein, the source characteristic parameters are used to characterize the distribution characteristics of the information sent by the source; the destination characteristic parameters are used to characterize the distribution characteristics of the information received by the destination.

[0207] In some embodiments, the distribution feature of the latent variable in variational inference is the channel state transition probability distribution feature.

[0208] Figure 13 It is a schematic diagram of the composition of another communication device provided by an embodiment of the present disclosure. The communication device can execute the semantic communication method provided by the above method embodiment. As Figure 13 shown, acquisition module 1301.

[0209] The acquisition module 1301 is configured to acquire second semantic feature information. The second semantic feature information is the information obtained after the first semantic feature information sent by the encoding end is transmitted through the channel. The first semantic feature information matches the channel feature parameters, and the first semantic feature information is used to characterize the semantic features of the original information;

[0210] The acquisition module 1301 is further configured to obtain decoding information based on the second semantic feature information.

[0211] In some embodiments, the acquisition module 1301 is specifically configured to perform semantic decoding on the second semantic feature information based on the channel feature parameters to obtain decoding information. In some embodiments, the channel feature parameters include at least one of the following: channel noise variance, channel state information; the channel state information includes at least one of the following: signal-to-noise ratio SNR, signal-to-interference-plus-noise ratio SINR, channel matrix, channel quality indicator CQI, reference signal received quality RSRQ.

[0212] In some embodiments, the channel state information is obtained by measuring pilot signals.

[0213] In some embodiments, the acquisition module 1301 is specifically configured to input the second semantic feature information into a decoder, and the decoder performs semantic decoding on the second semantic feature information based on the channel feature parameters to obtain decoding information.

[0214] In some embodiments, the decoder is trained through a loss function obtained based on variational inference, and the loss function includes a regularization term and a reconstruction term.

[0215] In some embodiments, the regularization term is determined based on source feature parameters and channel feature parameters; wherein, the source feature parameters are used to characterize the distribution feature of the information sent by the source; the channel feature parameters are used to characterize the channel transition probability distribution feature.

[0216] In some embodiments, the reconstruction term is determined based on source feature parameters and destination feature parameters; wherein, the source feature parameters are used to characterize the distribution feature of the information sent by the source; the destination feature parameters are used to characterize the distribution feature of the information received by the destination.

[0217] In some embodiments, the acquisition module 1301 is further configured to acquire new channel feature parameters; based on the new channel feature parameters, re-determine a loss function, and fine-tune the decoder based on the re-determined loss function to obtain an updated decoder.

[0218] In some embodiments, the decoder includes a latent variable calculation module, a reparameterization module, and a semantic feature information recovery module that are connected in sequence; the latent variable calculation module is configured to extract latent variables in the second semantic feature information, the reparameterization module is configured to perform reparameterization processing on the latent variables, and the semantic feature information recovery module is configured to perform semantic decoding processing on the reparameterized latent variables to obtain decoded information.

[0219] In some embodiments, the acquisition module 1301 is specifically configured to adjust the latent variable calculation module and the reparameterization module based on the new channel feature parameters to obtain an updated latent variable calculation module and reparameterization module.

[0220] In some embodiments, the distribution feature of the latent variable in variational inference is the channel state transition probability distribution feature.

[0221] In the case where the functions of the above integrated modules are implemented in the form of hardware, the embodiments of the present disclosure provide another possible structure of the communication device involved in the above embodiments. As Figure 14 shown, the communication device 140 includes: a processor 1402, a bus 1404. Optionally, the communication device may further include a memory 1401; optionally, the communication device may further include a communication interface 1403.

[0222] The processor 1402 may be used to implement or execute various exemplary logic blocks, modules, and circuits described in connection with the embodiments of the present disclosure. The processor 1402 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in connection with the embodiments of the present disclosure. The processor 1402 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0223] The communication interface 1403 is used to connect to other devices through a communication network. The communication network may be an Ethernet, a radio access network, a wireless local area network (WLAN), etc.

[0224] The memory 1401 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a magnetic disk storage medium, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0225] As a possible implementation, the memory 1401 can exist independently of the processor 1402. The memory 1401 can be connected to the processor 1402 via the bus 1404 for storing instructions or program code. When the processor 1402 calls and executes the instructions or program code stored in the memory 1401, the method for sending and receiving reference signal configuration information provided by the embodiments of the present disclosure can be implemented.

[0226] In another possible implementation, the memory 1401 can also be integrated with the processor 1402.

[0227] The bus 1404 can be an extended industry standard architecture (EISA) bus, etc. The bus 1404 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 14 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0228] In some embodiments, executable instructions are stored in the memory 1401. When the processor 1402 executes the executable instructions, the communication device is caused to execute the semantic communication method described in any one of the above embodiments.

[0229] Some embodiments of the present disclosure provide a computer-readable storage medium (for example, a non-transitory computer-readable storage medium). Computer program instructions are stored in the computer-readable storage medium. When the computer program instructions run on a computer, the computer is caused to execute the semantic communication method described in any one of the above embodiments.

[0230] Exemplarily, the above computer-readable storage medium may include, but is not limited to: magnetic storage devices (such as hard disks, floppy disks, or magnetic tapes, etc.), optical disks (such as Compact Disks (CDs), Digital Versatile Disks (DVDs), etc.), smart cards, and flash memory devices (such as Erasable Programmable Read-Only Memories (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this disclosure may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data).

[0231] An embodiment of the present disclosure provides a computer program product containing instructions. When the computer program product runs on a computer, it causes the computer to execute the semantic communication method described in any one of the above embodiments.

[0232] As described above, the above is only the specific implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present disclosure should be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A semantic communication method, characterized in that, Applied to the encoding end, the method includes: Obtain the original information; Perform semantic encoding on the original information based on channel characteristic parameters to obtain first semantic feature information, where the first semantic feature information is used to characterize the semantic features of the original information; Transmit the first semantic feature information.

2. The method according to claim 1, wherein The channel characteristic parameters include at least one of the following: channel noise variance, channel state information; the channel state information includes at least one of the following: signal-to-noise ratio SNR, signal-to-interference-plus-noise ratio SINR, channel matrix, channel quality indicator CQI, reference signal received quality RSRQ.

3. The method according to claim 2, characterized in that, The channel state information is obtained by measuring pilot signals.

4. The method according to claim 1, wherein The performing semantic encoding on the original information based on channel characteristic parameters to obtain the first semantic feature information includes: Input the original information into an encoder, and perform semantic encoding on the original information by the encoder based on channel characteristic parameters to obtain the first semantic feature information; Among them, the encoder is trained by a loss function obtained based on variational inference, and the loss function includes a regularization term and a reconstruction term.

5. The method according to claim 4, characterized in that, The regularization term is determined based on source characteristic parameters and channel characteristic parameters; where the source characteristic parameters are used to characterize the distribution characteristics of the information sent by the source; the channel characteristic parameters are used to characterize the channel transition probability distribution characteristics.

6. The method according to claim 4, characterized in that The method further includes: Obtain new channel characteristic parameters; Based on the new channel characteristic parameters, re-determine the loss function; Based on the re-determined loss function, fine-tune the encoder to obtain an updated encoder.

7. The method according to claim 4, characterized in that, The reconstruction term is determined based on source characteristic parameters and destination characteristic parameters; where the source characteristic parameters are used to characterize the distribution characteristics of the information sent by the source; the destination characteristic parameters are used to characterize the distribution characteristics of the information received by the destination.

8. A semantic communication method, characterized in that, Applied to the decoding end, the method includes: Obtain second semantic feature information, where the second semantic feature information is the information obtained after the first semantic feature information sent by the encoding end is transmitted through the channel, the first semantic feature information matches the channel characteristic parameters, and the first semantic feature information is used to characterize the semantic features of the original information; Based on the second semantic feature information, obtain decoded information.

9. The method according to claim 8, wherein The obtaining decoded information based on the second semantic feature information includes: Perform semantic decoding on the second semantic feature information based on the channel characteristic parameters to obtain the decoded information.

10. The method according to claim 9, characterized in that, The channel characteristic parameters include at least one of the following: channel noise variance, channel state information; the channel state information includes at least one of the following: signal-to-noise ratio SNR, signal-to-interference-plus-noise ratio SINR, channel matrix, channel quality indicator CQI, reference signal received quality RSRQ.

11. The method according to claim 10, wherein The channel state information is obtained by measuring pilot signals.

12. The method according to claim 9, wherein The performing semantic decoding on the second semantic feature information based on the channel characteristic parameters to obtain the decoded information includes: Input the second semantic feature information into a decoder, and perform semantic decoding on the second semantic feature information by the decoder based on the channel characteristic parameters to obtain the decoded information; Among them, the decoder is trained by a loss function obtained based on variational inference, and the loss function includes a regularization term and a reconstruction term.

13. The method according to claim 12, wherein The regularization term is determined based on a source feature parameter and a channel feature parameter; wherein, the source feature parameter is used to characterize the distribution feature of the information sent by the source; the channel feature parameter is used to characterize the channel transition probability distribution feature.

14. The method according to claim 12, wherein The reconstruction term is determined based on a source feature parameter and a destination feature parameter; wherein, the source feature parameter is used to characterize the distribution feature of the information sent by the source; the destination feature parameter is used to characterize the distribution feature of the information received by the destination.

15. The method according to claim 12, wherein The method further includes: Obtaining a new channel feature parameter; Based on the new channel feature parameter, re-determining the loss function; Based on the re-determined loss function, fine-tuning the decoder to obtain an updated decoder.

16. The method according to claim 12, wherein The decoder includes a latent variable calculation module, a reparameterization module, and a semantic feature information recovery module connected in sequence; the latent variable calculation module is used to extract the latent variable in the second semantic feature information, the reparameterization module is used to perform reparameterization processing on the latent variable, and the semantic feature information recovery module is used to perform semantic decoding processing on the reparameterized latent variable to obtain decoded information.

17. The method according to claim 15, wherein The re-training the decoder based on the new channel feature parameter to obtain an updated decoder for the channel includes: Based on the new channel feature parameter, adjusting the latent variable calculation module and the reparameterization module to obtain the updated latent variable calculation module and the reparameterization module.

18. A communication device, characterized in that, It includes a memory, a processor, and computer program instructions stored on the memory and executable on the processor, and when the processor executes the computer program instructions, the method described in any one of claims 1 to 17 is implemented.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer program instructions; wherein, when the computer program instructions run on a computer, the computer is caused to execute the method described in any one of claims 1 to 17.

Images