A semantic communication method, system and terminal based on multi-objective contrastive learning

Abstract

The present invention relates to the field of artificial intelligence and semantic communication technology, and proposes a semantic communication method, system and terminal based on multi-objective contrastive learning. The semantic communication method is completed based on a trained autoencoder model, including: configuring several texts of different lengths, inputting several texts of different lengths into a semantic communication encoder in sequence to obtain a semantic coding sequence; the semantic coding sequence is transmitted through a channel to become a fuzzy semantic coding sequence, and the fuzzy semantic coding sequence is input into a semantic communication decoder for semantic decoding; the autoencoder model is trained based on a multi-objective contrastive learning loss function, and the training is completed when the loss value does not decrease in a preset number of consecutive trainings; the trained autoencoder model is deployed to a channel for semantic communication. The present invention not only takes into account the accurate reconstruction of text information, but also noise robustness, feature uniformity and text authenticity, and has stronger anti-noise robustness.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and semantic communication technology, and in particular to a semantic communication method, system and terminal based on multi-objective contrast learning. Background Art

[0002] In recent years, with the development of deep learning (DL) and artificial intelligence (AI), machines have gradually acquired semantic understanding capabilities comparable to humans. An important example is the emergence of GPT. Under the development trend of deep learning, deep learning-based source-channel joint coding (Deep JSCC) has become a promising method and an effective solution for semantic communication. As a generative model based on deep learning, the autoencoder (AE) jointly learns the encoding and decoding processes and reconstructs the original data as much as possible based on the compressed low-dimensional features. Its optimization goal is widely used in Deep JSCC. The categories of autoencoders include: the most primitive autoencoder (AE), variational autoencoder (VAE) and adversarial autoencoder (AAE). For example, the first DeepJSCC image transmission method based on the autoencoder architecture proposed in the prior art is superior to the digital transmission scheme based on source-channel separation under low signal-to-noise ratio conditions compared with the traditional JPEG and JPEG2000 source coding. From a variational perspective, Deep JSCC can be regarded as a variational autoencoder (VAE). Based on this conclusion, some people take the objective function of VAE as the optimization target and propose nonlinear transform source channel coding (NTSCC), which can closely adapt to the source distribution under nonlinear transformation. Others have proposed Deep JSCC based on SwinTransformer, called Swin JSCC, which also takes the objective function of VAE as the optimization target, is suitable for semantic communication, and surpasses the most advanced BPG + 5G LDPC coded transmission system.

[0003] Although the above-mentioned autoencoder-based Deep JSCC scheme has shown impressive performance, the impact of semantic noise and the robustness of the system when applying autoencoders to semantic communication still need further study. Similar to the noise in traditional systems, semantic noise can cause misunderstanding of semantic information and decoding errors, resulting in misleading between the expected semantics and the reconstructed semantics at the receiver. Semantic noise can be generated at different stages such as semantic encoding, data transmission, and decoding. In the semantic encoding stage, semantic noise corresponds to the mismatch between the original signal after semantic encoding and the encoded signal, which is related to the representation ability of the encoder. In the data transmission stage, signal distortion caused by channel fading or some carefully designed signals sent by malicious attackers introduce semantic noise. In the decoding stage, misunderstanding, incorrect expression, and confusion of meaning bring semantic noise to the receiver. In view of this, it is urgent to propose a semantic communication method that can accurately reconstruct text information. Summary of the invention

[0004] The purpose of the present invention is to provide a semantic communication method, system and terminal based on multi-objective contrastive learning. The method closely aligns similar samples and keeps a distance between dissimilar samples through contrastive learning, thereby obtaining a semantic communication encoder and decoder with strong anti-noise ability. The present invention not only considers the accurate reconstruction of text information, but also considers noise robustness, feature uniformity and text authenticity. Under different signal-to-noise ratios (SNRs), especially under low signal-to-noise ratios, the present invention demonstrates stronger anti-noise robustness.

[0005] In order to achieve the above object, the present invention adopts the following technical scheme:

[0006] In a first aspect, the present invention provides a semantic communication method based on multi-objective contrastive learning, the method is implemented based on a trained autoencoder model, the autoencoder model includes: a semantic communication encoder, a semantic communication decoder and a channel, the semantic communication method includes the following steps:

[0007] S1. Configuring several texts of different lengths, inputting several texts of different lengths into the semantic communication encoder in sequence to obtain a semantic coding sequence;

[0008] S2. The semantic coding sequence is transmitted through the channel to become a fuzzy semantic coding sequence, and the fuzzy semantic coding sequence is input into the semantic communication decoder for semantic decoding;

[0009] S3. Train the autoencoder model based on the multi-objective contrastive learning loss function. When the loss value does not decrease after a preset number of consecutive trainings, the training is completed.

[0010] The multi-objective contrastive learning loss function is as follows:

[0011] ;

[0012] In the formula, represents the loss value, represents the reconstruction loss, represents the KL divergence, represents the discriminator loss, represents the generator loss, represents the contrast loss, , , and They are the weight hyperparameters of KL divergence, discriminator loss, generator loss, and contrast loss respectively;

[0013] S4. Deploy the autoencoder model trained in S3 to the channel in S2 for semantic communication.

[0014] As a possible implementation method, the reconstruction loss is calculated by the following formula:

[0015]

[0016] in, Represents text Middle Words The true distribution of Represents the semantic decoder output reconstructed text Middle Words The probability distribution of For a given set of texts, for The text collection after being decoded by the semantic decoder.

[0017] As a possible implementation method, KL divergence is introduced as a regular term of the multi-objective contrastive learning loss function:

[0018]

[0019] represents the KL divergence, is the mean of each encoding, is the variance of each encoding, represents a semantic coding sequence, represents the distribution of semantic coding sequences, represents the distribution of semantic encoding sequences predicted by the model, represents the dimension of semantic encoding, It means it follows the standard normal distribution.

[0020] As a possible implementation, the discriminator loss is calculated as follows:

[0021]

[0022] In the formula, represents the discriminator loss, represents the cross entropy loss, represents a semantic coding sequence, Indicates noise;

[0023] The generator loss is calculated as follows:

[0024]

[0025] In the formula, represents the generator loss, represents the cross entropy loss, Represents a semantic encoding sequence.

[0026] As a possible implementation, the contrast loss is calculated as follows:

[0027]

[0028] In the formula, B express B training batches, b represents the mini-batch size, represents the cosine similarity, that is , represents the temperature hyperparameter and , Represents semantic encoding, express Enhanced coding, Indicates that the same semantic coding sequence is Other semantic encodings besides .

[0029] As a possible implementation method, S1 specifically includes the following sub-steps:

[0030] S10. dividing each text into word sequences, and normalizing the lengths of the word sequences to obtain text sequences of the same length;

[0031] S11. For each text in the text sequence of the same length, one-hot encode the words in the text, and obtain an embedded representation of the text based on the one-hot encoding;

[0032] S12. Using a bidirectional long short-term memory network to obtain a forward encoding representation and a backward encoding representation of the embedded representation, and concatenating the forward encoding representation and the backward encoding representation to obtain an encoding representation of the text;

[0033] S13. Extract the mean and logarithmic variance of the encoded representation, reparameterize the mean and logarithmic variance, and obtain the semantic encoding of the text.

[0034] As a possible implementation method, S2 specifically includes the following sub-steps:

[0035] S20. Initialize the embedding representation in S11 to obtain the text sequence embedding representation;

[0036] S21. After adding the fuzzy semantic encoding sequence to the text sequence embedding representation, a long short-term memory network is used to obtain the decoding representation of the first word and the hidden state of the first word;

[0037] S22. Based on the hidden state of the first word, a decoded representation of the second word and a hidden state of the second word are obtained, and the decoded representation of the third word and a hidden state of the third word are obtained based on the hidden state of the second word, until the decoded representations of all words are obtained, and semantic decoding is completed.

[0038] In a second aspect, the present invention provides a semantic communication system based on multi-objective contrastive learning, which is used to execute the semantic communication method based on multi-objective contrastive learning provided in the first aspect, including: a semantic communication encoder, a semantic communication decoder and a channel;

[0039] The semantic communication encoder includes a word segmenter, an embedding layer, a bidirectional long short-term memory network, a first linear layer, and a second linear layer;

[0040] The channel is one of an AWGN channel, a uniform fading channel, a Rayleigh fading channel or a Rice fading channel;

[0041] The semantic communication decoder comprises: a first dropout layer, a second dropout layer, a third linear layer, a fourth linear layer, a long short-term memory network, and an embedding layer shared with the semantic communication encoder;

[0042] The word segmenter is used to segment each text into word sequences, standardize the length of the word sequences, and obtain text sequences of the same length; the embedding layer is used to obtain the embedded representation of the text; the bidirectional long short-term memory network is used to obtain the forward encoding representation and the backward encoding representation of the embedded representation and concatenate them into the encoding representation of the text; the first linear layer is used to extract the mean of the encoding representation, and the second linear layer is used to extract the logarithmic variance of the encoding representation;

[0043] The first and second Dropout layers are used to close the connections of some neurons during training; the third linear layer is used to map the fuzzy semantic coding sequence to the same dimension as the text sequence embedding representation; the long short-term memory network is used to obtain the decoding representation and hidden state of the word; the fourth linear layer is used to change the dimension of the coding representation.

[0044] As a possible implementation, the word segmenter also includes a vocabulary, which is used to map each word in the word sequence to a unique identifier, and to standardize the length of the word sequence based on the identifier of each word.

[0045] In a third aspect, the present invention provides a terminal, comprising a processor and a communication interface coupled to the processor, wherein the processor is used to run a computer program or instruction to implement the semantic communication method based on multi-objective contrastive learning provided in the first aspect.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] 1. The semantic communication method based on multi-objective contrastive learning provided by the present invention adopts a multi-objective contrastive learning loss function to train an autoencoder model. Through contrastive learning, similar samples are closely aligned and dissimilar samples are kept apart, thereby obtaining a semantic communication encoder and decoder with strong anti-noise ability, which can be well applied to semantic communication.

[0048] 2. The semantic communication method based on multi-objective contrastive learning provided by the present invention trains three types of autoencoders, namely AE, VAE and AAE, based on multi-objectives, and trains AE to achieve the goal of accurately reconstructing text. The uniformity of semantic coding features is guaranteed by calculating the KL divergence loss in VAE, and the discriminator and discriminant loss calculation are introduced when training AAE to improve the authenticity of the text. Therefore, the present invention can well ensure the uniformity of semantic features and the authenticity of text.

[0049] 3. The semantic communication method based on multi-objective contrastive learning provided by the present invention comprehensively considers reconstruction loss, KL divergence, discriminator loss, generator loss, contrast loss, etc. to construct a multi-objective contrastive learning loss function, which not only considers the accurate reconstruction of text information, but also considers noise robustness, feature uniformity and text authenticity, and has stronger anti-noise robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0051] Figure 1is a flow chart of a semantic communication method based on multi-objective contrastive learning in an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of training three types of autoencoders based on a multi-objective contrastive learning loss function in an embodiment of the present invention;

[0053] Figure 3 Schematic diagram of a semantic communication system based on multi-objective contrastive learning in an embodiment of the present invention.

[0054] Reference numerals

[0055] 1-semantic communication encoder, 10-word segmenter, 11-embedding layer, 12-bidirectional long short-term memory network, 13-first linear layer, 14-second linear layer;

[0056] 2-semantic communication decoder, 20-first Dropout layer, 21-second Dropout layer, 22-third linear layer, 23-fourth linear layer, 24-long short-term memory network;

[0057] 3-Channel. DETAILED DESCRIPTION

[0058] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. For example, the first threshold and the second threshold are only used to distinguish different thresholds, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0059] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0060] In the present invention, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The following at least one item (items) or similar expressions thereof refer to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one item (items) of a, b or c can mean: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0061] When existing autoencoders are used in semantic communication, semantic noise can cause misunderstanding of semantic information and decoding errors, resulting in misleading between the semantics expected by the receiving end and the reconstructed semantics. To solve this problem, the present invention proposes a semantic communication method, system and terminal based on multi-objective contrastive learning. Through contrastive learning, similar samples are closely aligned, while dissimilar samples are kept apart, thereby obtaining a semantic communication encoder and decoder with strong anti-noise ability. The present invention not only considers the accurate reconstruction of text information, but also considers noise robustness, feature uniformity and text authenticity. Under different signal-to-noise ratios (SNRs), especially under low signal-to-noise ratios, the present invention demonstrates stronger anti-noise robustness.

[0062] In a first aspect, the present invention provides a semantic communication method based on multi-objective contrastive learning, which is implemented based on a preset and trained autoencoder model, wherein the autoencoder model includes: a semantic communication encoder, a semantic communication decoder, and a channel, see Figure 1 ,The semantic communication method includes the following steps:

[0063] S1. Configuring several texts of different lengths, inputting several texts of different lengths into the semantic communication encoder in sequence to obtain a semantic coding sequence;

[0064] As a possible implementation method, S1 specifically includes the following sub-steps:

[0065] S10. dividing each text into word sequences, and normalizing the lengths of the word sequences to obtain text sequences of the same length;

[0066] As an example, suppose the number of texts of different lengths is b Each text is divided into several words, and the word with the longest length is found. Assume that the length of the word is , taking the word with the longest length as a reference, add padding marks to other words so that the length of all words becomes , concatenate the words of uniform length according to their positions in the original text, and get b Sequences of text with the same length.

[0067] S11. For each text in a text sequence of the same length, one-hot encode the words in the text, i.e., assign a unique identifier to each word, and obtain an embedded representation of the text based on the one-hot encoding; illustratively, the one-hot encoding is input into an embedding layer to obtain an embedded representation of the text, and the embedded representation contains the semantic information of each word in the text.

[0068] S12. A bidirectional long short-term memory network is used to obtain a forward encoding representation and a backward encoding representation of the embedded representation, and the forward encoding representation and the backward encoding representation are concatenated to obtain the encoding representation of the text; the bidirectional long short-term memory network is used to scan the entire text forward and backward respectively, similar to humans reading articles, so as to obtain the contextual information of the text.

[0069] S13. Extract the mean and logarithmic variance of the encoded representation, reparameterize the mean and logarithmic variance, and obtain the semantic encoding of the text.

[0070] As an example, the mean of the encoded representation is expressed as: , the logarithmic variance is expressed as: ,in, b Indicates the number of texts. represents the dimension of semantic encoding, is a real number set, and the semantic encoding obtained by reparameterization is expressed as:

[0071]

[0072] in, represents the noise randomly sampled from a standard normal distribution, that is, , represents the Hadamard product.

[0073] S2. The semantic coding sequence is transmitted through the channel to become a fuzzy semantic coding sequence, and the fuzzy semantic coding sequence is input into the semantic communication decoder for semantic decoding;

[0074] As a possible implementation method, S2 specifically includes the following sub-steps:

[0075] Taking the model transmitted on the AWGN channel as an example, the fuzzy semantic coding sequence is expressed as:

[0076]

[0077] in, represents the channel model, represents the channel factor matrix, is random Gaussian noise, and , and Independent of each other, Represents a fuzzy semantic coding sequence.

[0078] As a possible implementation method, S2 specifically includes the following sub-steps:

[0079] S20. Initialize the embedding representation in S11 to obtain the text sequence embedding representation;

[0080] As an example, in the preset autoencoder model, the semantic communication encoder and the semantic communication decoder share the embedding layer, and semantic decoding is performed by two branches. The first branch first defines a special tag, such as , this special tag represents the decoded text set The start tag of ,after, Enter the embedding layer to obtain the text sequence embedding representation. The second branch is the fuzzy semantic coding sequence through a linear layer, the purpose of which is to map the fuzzy semantic coding sequence to the embedding dimension.

[0081] S21. After adding the fuzzy semantic encoding sequence to the text sequence embedding representation, a long short-term memory network is used to obtain the decoding representation of the first word and the hidden state of the first word;

[0082] Exemplarily, the text sequence embedding representation output by the first branch is added to the fuzzy semantic coding sequence output by the second branch as the input of the long short-term memory network to obtain the decoded representation of the first word and the hidden state of the first word. The decoded representation of the first word is then passed through a Dropout layer and a linear layer to obtain the probability distribution of the next word. Through different decoding strategies, such as greedy, sampling, and Top-k strategies, the predicted next word, i.e., the second word, can be obtained, and the second word is filled into the decoded text set. The next iteration is performed at the end of .

[0083] S22. Based on the hidden state of the first word, a decoded representation of the second word and a hidden state of the second word are obtained, and the decoded representation of the third word and a hidden state of the third word are obtained based on the hidden state of the second word, until the decoded representations of all words are obtained, and semantic decoding is completed.

[0084] As an example, the hidden state of the first word will be used as the input of the LSTM network at the next iteration until the entire semantic encoding sequence is decoded.

[0085] S3. Train the autoencoder model based on the multi-objective contrastive learning loss function. When the loss value does not decrease after a preset number of consecutive trainings, the training is completed.

[0086] The multi-objective contrastive learning loss function is as follows:

[0087] ;

[0088] In the formula, represents the loss value, represents the reconstruction loss, represents the KL divergence, represents the discriminator loss, represents the generator loss, represents the contrast loss, , , and They are the weight hyperparameters of KL divergence, discriminator loss, generator loss, and contrast loss respectively;

[0089] Autoencoders are widely used generative models that also have an encoder-decoder architecture, making them very suitable for semantic communication. However, when existing technologies apply autoencoders to semantic communication, the impact of semantic noise and the robustness of the system are still large. Autoencoders include: original autoencoders (AE), variational autoencoders (VAE), and adversarial autoencoders (AAE), see Figure 2 The present invention trains three types of autoencoders based on a multi-objective contrastive learning loss function, trains the original autoencoder to achieve the goal of accurately reconstructing the text, trains the variational autoencoder to achieve the goal of making the semantic coding sequence obey the standard normal distribution, and trains the adversarial autoencoder to generate a text that is indistinguishable from the real thing, so as to achieve the goal of ensuring the authenticity of the text. Exemplarily, a discriminator is introduced in the process of training the adversarial autoencoder to determine whether the semantic coding sequence is from the output of the semantic communication encoder or from the sampled standard normal distribution, and the discriminator, the semantic communication encoder and the semantic communication decoder are alternately optimized during the training process.

[0090] After multi-objective training, the autoencoder model is trained with the contrastive learning loss function, that is, a semantic code in a training batch is And its corresponding enhanced coding is considered as a positive sample pair, With other encodings in the same training batch The purpose of contrastive learning is to bring similar samples (positive sample pairs) closer in feature space and push dissimilar samples (negative sample pairs) apart in feature space, so that the autoencoder model can learn the similarity and diversity representation of semantic features, and thus more effectively resist noise perturbations.

[0091] Exemplarily, the preset number of training times is 8 to 12 times, for example: 8 times, 9 times, 10 times, 11 times, 12 times.

[0092] S4. Deploy the autoencoder model trained in S3 to the channel in S2 for semantic communication.

[0093] As a possible implementation method, the reconstruction loss is calculated as follows:

[0094]

[0095] in, Represents text Middle Words The true distribution of Represents the semantic decoder output reconstructed text Middle Words The probability distribution of For a given set of texts, for The text collection after being decoded by the semantic decoder.

[0096] As a possible implementation method, KL divergence is introduced as a regular term of the multi-objective contrastive learning loss function:

[0097]

[0098] represents the KL divergence, is the mean of each encoding, is the variance of each encoding, represents a semantic coding sequence, represents the distribution of semantic coding sequences, represents the distribution of semantic encoding sequences predicted by the model, represents the distribution of semantic encoding sequences predicted by the model, represents the dimension of semantic encoding, It follows the standard normal distribution. represents the mathematical expectation, Represents the identity matrix.

[0099] As a possible implementation, the discriminator loss is calculated as follows:

[0100]

[0101] In the formula, represents the discriminator loss, represents the cross entropy loss, represents a semantic coding sequence, Indicates noise;

[0102] The generator loss is calculated as follows:

[0103]

[0104] In the formula, represents the generator loss, represents the cross entropy loss, Represents a semantic encoding sequence.

[0105] As a possible implementation, the contrast loss is calculated as follows:

[0106]

[0107] In the formula, B express B training batches, b represents the mini-batch size, represents the cosine similarity, that is , usually, the semantic encoding is normalized, so the cosine similarity can be calculated as the dot product between two semantic encodings, that is, , represents the temperature hyperparameter and , Represents semantic encoding, express Enhanced coding, Indicates that the same semantic coding sequence is Other semantic encodings besides .

[0108] In a second aspect, the present invention provides a semantic communication system based on multi-objective contrastive learning, which is used to execute the semantic communication method based on multi-objective contrastive learning provided in the first aspect, see Figure 3 , comprising: a semantic communication encoder 1, a semantic communication decoder 2 and a channel 3;

[0109] See also Figure 3 , the semantic communication encoder 1 includes a word segmenter 10, an embedding layer 11, a bidirectional long short-term memory network 12, a first linear layer 13 and a second linear layer 14;

[0110] Channel 3 is one of an AWGN channel, a uniform fading channel, a Rayleigh fading channel or a Rice fading channel;

[0111] See also Figure 3 , the semantic communication decoder 2 includes: a first Dropout layer 20, a second Dropout layer 21, a third linear layer 22, a fourth linear layer 23, a long short-term memory network 24, and an embedding layer 11 shared with the semantic communication encoder;

[0112] The word segmenter 10 is used to segment each text into a word sequence, standardize the length of the word sequence, and obtain a text sequence of the same length; the embedding layer 11 is used to obtain the embedded representation of the text; the bidirectional long short-term memory network 12 is used to obtain the forward encoding representation and the backward encoding representation of the embedded representation and concatenate them into the encoding representation of the text; the first linear layer 13 is used to extract the mean of the encoding representation, and the second linear layer 14 is used to extract the logarithmic variance of the encoding representation;

[0113] The first Dropout layer 20 and the second Dropout layer 21 are used to close the connections of some neurons during training; the third linear layer 22 is used to map the fuzzy semantic coding sequence to the same dimension as the text sequence embedding representation; the long short-term memory network 24 is used to obtain the decoding representation and hidden state of the word; the fourth linear layer 23 is used to change the dimension of the coding representation.

[0114] As a possible implementation, the word segmenter 10 further includes a word table, which is used to map each word in the word sequence to a unique identifier, and to standardize the length of the word sequence based on the identifier of each word.

[0115] In a third aspect, the present invention provides a terminal, comprising a processor and a communication interface coupled to the processor, wherein the processor is used to run a computer program or instruction to implement the semantic communication method based on multi-objective contrastive learning provided in the first aspect.

[0116] The semantic communication system and semantic communication method based on multi-objective contrastive learning provided by the present invention are described in detail below in conjunction with specific embodiments.

[0117] This embodiment is carried out on a server with the following specific configurations: Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz, two NVIDIA GeForce RTX 4090 24GB GPUs, and an operating system of Ubuntu 20.04.6 LTS. The simulation experiment is carried out on an AWGN channel with SNR=3dB.

[0118] First, two datasets are constructed, namely the Yelp and Yahoo datasets. For each dataset, it is divided into training set, development set and test set. The training set is used for autoencoder model training. During the training process, the development set is evaluated after each round of training, and the autoencoder model with the smallest loss value on the development set is saved. The Yelp dataset includes 444K / 63K / 127K sentences as training / development / test sets respectively, and the sentence length is less than 16 words. The Yahoo dataset was originally document-level, and the sentence length was between 2 and 50 words through sentence segmentation, including 495K / 49K / 50K sentences as training / development / test sets respectively.

[0119] First, a word segmenter is used to unify the length of the sentences. The method of this embodiment is to In a sentence of , for length less than Sentences with special markers in the missing parts <pad>, for length greater than After processing, we get a batch of sentences:

[0120] .

[0121] In addition, the tokenizer also contains a vocabulary (the vocabulary size is denoted by ), which is used to map each token to a unique identifier (id). This mapping result can be called one-hot encoding. Then, the one-hot encoding is input into the embedding layer to obtain the embedding representation of the sentence. Assume that the dimension of the embedding layer is ,but Generally speaking, the embedding representation contains the semantic information of each word in the sentence, but it cannot fully represent the semantic information of the entire sentence because it only contains the independent information of each word and has no context-related temporal information. The embedding representation is input into the bidirectional long short-term memory network to obtain a forward encoding representation of a batch of sentences. and the backward encoding representation ,in Represents the hidden layer dimension. Concatenate the forward and backward encoding representations to get the encoding representation:

[0122] .

[0123] At this point, the encoded representation can represent the semantic information contained in the entire sentence, because the bidirectional long short-term memory network scans the entire sentence through forward and backward respectively, which is similar to the process of human reading sentences and articles, thereby obtaining the contextual information of the sentence. However, in order to obtain richer sentence features, referring to the variational autoencoder (VAE), two linear layers are designed to extract the mean of sentence features respectively. and log variance ,in represents the semantic encoding dimension in order to achieve better sentence reconstruction. Finally, using the reparameterization technique, the semantic encoding ,in, represents the noise randomly sampled from a standard normal distribution, that is, , represents the Hadamard product.

[0124] Abstract the entire processing process as a semantic encoder ,in Represents the parameters of all neural networks in the encoder. The semantic encoder takes as input Encoded as ,Right now . Generally speaking, the encoding represents Dimensions Much smaller than the vocabulary size , thereby realizing the dimensionality reduction transmission of data.

[0125] Next, we take the transmission of the autoencoder model on the AWGN channel as an example. The channel model is expressed as

[0126]

[0127] in, represents the channel model, represents the channel factor matrix, is random Gaussian noise, and , and are independent of each other. For the Gaussian additive white noise (AWGN) channel, is the identity matrix , so the above formula can also be expressed as , which indicates that the AWGN channel contains only additive noise.

[0128] Next, we enter the decoding operation. There are two branches at the beginning of decoding. The first branch first defines a special mark , which means the output sentence The start tag of Then, output the sentence Entering the embedding layer, we get , and then enter the first dropout layer. Note that the role of the first dropout layer here is to close the connection of some neurons during training, so as to avoid overfitting of model training. Since the embedding layer is shared with the semantic communication encoder, and the dropout layer does not change the embedding dimension, therefore: ; The second branch encodes the semantics after channel transmission Enter the third linear layer to map the semantic encoding to the embedding dimension, i.e. , and then expands to The outputs of the two branches are added together as the input of the long short-term memory network, and two outputs are obtained: one is the decoded representation , the second is the hidden state, including the cell state of the long short-term memory network and hidden state .

[0129] Decoded representation After the second dropout layer and the fourth linear layer again, the probability distribution of predicting the next word is obtained, that is, , through different decoding strategies, such as greedy, sampling and Top-k strategies, the predicted next word can be obtained , and fill the predicted words into the sentence At the end of the next iteration, the hidden state will also be used as the input of the long short-term memory network in the next iteration. Until the semantic communication decoder decodes the end tag When , stop the iteration and get the final output sentence:

[0130] .

[0131] Abstract the entire processing process as a semantic decoder ,in represents the parameters of the decoder neural network. The decoder receives After that, use And the decoded part of the current sentence to reconstruct the input sentence, decoding to get ,Right now Generally speaking, if two input sentences If their semantics are similar, then their semantic encodings and should be as close as possible to ensure that their reconstructed sentences and The semantics are also similar.

[0132] Next, the autoencoder model is trained end-to-end with multiple targets, see Figure 2 AE aims to accurately reconstruct sentences. The original text is segmented and then input into the encoder to obtain the encoded representation. The encoded representation will be damaged after passing through the noisy channel and will be decoded and reconstructed at the decoder. and the set of sentences reconstructed from the dataset , jointly optimize the parameters of the semantic communication encoder and semantic communication decoder, namely:

[0133]

[0134] in, represents an original input sentence, Represents the reconstructed sentence output by the original input sentence after passing through the semantic communication encoder, channel, and semantic communication decoder.

[0135] The cross entropy loss between the decoded reconstructed sentence and each token in the original sentence is:

[0136]

[0137] in, Expressing sentences Middle Words The true distribution of (each word is one-hot encoded, which is a Bernoulli distribution), Represents the decoder output reconstructed sentence Middle Words The probability distribution of .

[0138] The goal of VAE is to make the encoding Follow the standard normal distribution as much as possible , to ensure feature uniformity, KL divergence is introduced as the regularization term of the loss function based on the reconstruction loss. ,Right now:

[0139] ;

[0140] in, and denote the mean and variance of each encoded feature, respectively.

[0141] The goal of AAE is to make the encoder generate realistic text, thereby ensuring the authenticity of the text; by introducing a discriminator Determine whether the encoding comes from the output of the encoder or from the sampled standard normal distribution, and alternately optimize the discriminator, encoder (regarded as a generator), and decoder during training. The loss function of the discriminator is calculated as

[0142] ,

[0143] The encoder additionally calculates a generation loss function

[0144] ;

[0145] Combined with the contrastive learning loss function to train the autoencoder model, one encoding in a training batch And the corresponding enhanced coding is considered as a positive sample pair, With other encodings in the same training batch The purpose of contrastive learning is to bring similar samples (positive sample pairs) closer together in the feature space and push dissimilar samples (negative sample pairs) apart in the feature space, so that the model can learn the similarity and diversity representation of semantic features, and thus more effectively resist noise perturbations. The contrastive loss is calculated as follows:

[0146]

[0147] Where b represents the mini-batch size, represents the cosine similarity, that is , represents the temperature hyperparameter and , Represents semantic encoding, express Enhanced coding, Indicates that the same semantic coding sequence is Other semantic encodings besides . Generally speaking, the encoding features are normalized, so the cosine similarity can be calculated as the dot product between two encoding features, that is, .

[0148] In summary, the overall multi-objective loss function is calculated as follows:

[0149]

[0150] Finally, the trained autoencoder model is deployed on the channel for semantic communication. The text to be transmitted is semantically encoded at the sender to obtain a hidden code; then, the hidden code is transmitted through the channel, which will introduce semantic noise; finally, the semantic code reaches the receiver and is semantically decoded to obtain reconstructed semantic information. The decoding results are shown in Table 1 below:

[0151] Table 1 Simulation results of semantic communication in this embodiment

[0152]

[0153] It can be seen from the above table that the present invention takes into account multi-objective contrastive learning loss, can obtain more accurate and realistic semantic communication results, and has strong anti-noise robustness.

[0154] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the drawings, etc. In the specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the specification. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0155] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the present invention and its equivalents.< / pad>

Claims

1. A semantic communication method based on multi-objective contrastive learning, characterized in that: The semantic communication method is implemented based on a trained autoencoder model, wherein the autoencoder model includes: a semantic communication encoder, a semantic communication decoder, and a channel. The semantic communication method includes the following steps: S1. Configuring several texts of different lengths, inputting the several texts of different lengths into the semantic communication encoder in sequence, and obtaining a semantic coding sequence, specifically comprising the following sub-steps: S10. Each text is divided into word sequences, and the length of the word sequences is standardized to obtain text sequences of the same length; S11. For each text in the text sequence of the same length, one-hot encode the words in the text, and obtain an embedded representation of the text based on the one-hot encoding; S12. Using a bidirectional long short-term memory network to obtain a forward encoding representation and a backward encoding representation of the embedded representation, and concatenating the forward encoding representation and the backward encoding representation to obtain an encoding representation of the text; S13. extracting the mean and logarithmic variance of the encoded representation of the text, re-parameterizing the mean and logarithmic variance, and obtaining the semantic encoding of the text; S2. The semantic coding sequence is transmitted through the channel to become a fuzzy semantic coding sequence, and the fuzzy semantic coding sequence is input into the semantic communication decoder for semantic decoding; S3. Training the autoencoder model based on a multi-objective contrast learning loss function, when the loss value does not decrease in a preset number of consecutive trainings, the training is completed; The multi-objective contrastive learning loss function is as follows: ； In the formula, represents the loss value, represents the reconstruction loss, represents the KL divergence, represents the discriminator loss, represents the generator loss, represents the contrast loss, , , and They are the weight hyperparameters of KL divergence, discriminator loss, generator loss, and contrast loss respectively; S4. Deploy the autoencoder model trained in S3 to the channel described in S2 for semantic communication.

2. The semantic communication method based on multi-objective contrastive learning according to claim 1, characterized in that: The reconstruction loss is calculated by the following formula: in, represents the reconstruction loss, represents the cross entropy loss, Represents text Middle Words The true distribution of Represents the semantic decoder output reconstructed text Middle Words The probability distribution of For a given set of texts, for The text collection after being decoded by the semantic decoder.

3. The semantic communication method based on multi-objective contrastive learning according to claim 1 is characterized in that: The KL divergence is introduced as the regularization term of the multi-objective contrastive learning loss function: represents the KL divergence, is the mean of each encoding, is the variance of each encoding, represents a semantic coding sequence, Represents text, represents the distribution of semantic coding sequences, represents the distribution of semantic encoding sequences predicted by the model, represents the dimension of semantic encoding, It follows the standard normal distribution. represents the mathematical expectation, Represents the identity matrix.

4. The semantic communication method based on multi-objective contrastive learning according to claim 1, characterized in that: The discriminator loss is calculated as follows: In the formula, represents the discriminator loss, represents the cross entropy loss, represents a semantic coding sequence, Indicates noise; The generator loss is calculated as follows: In the formula, represents the generator loss, represents the cross entropy loss, Represents a semantic encoding sequence.

5. The semantic communication method based on multi-objective contrastive learning according to claim 1, characterized in that: The contrast loss is calculated as follows: In the formula, B express B training batches, b represents the mini-batch size, represents the cosine similarity, that is , represents the temperature hyperparameter and , Represents semantic encoding, express Enhanced coding, Indicates that the same semantic coding sequence is Other semantic encodings besides .

6. The semantic communication method based on multi-objective contrastive learning according to claim 1, characterized in that: The S2 specifically includes the following sub-steps: S20. Initialize the embedding representation described in S11 to obtain the text sequence embedding representation; S21. After adding the fuzzy semantic coding sequence to the text sequence embedding representation, a long short-term memory network is used to obtain a decoding representation of the first word and a hidden state of the first word; S22. Based on the hidden state of the first word, a decoded representation and a hidden state of the second word are obtained, and the decoded representation and the hidden state of the third word are obtained based on the hidden state of the second word, until the decoded representations of all words are obtained, and semantic decoding is completed.

7. A semantic communication system based on multi-objective contrastive learning, used to execute the semantic communication method based on multi-objective contrastive learning according to any one of claims 1 to 6, characterized in that: include: semantic communication encoder, semantic communication decoder and channel; The semantic communication encoder includes a word segmenter, an embedding layer, a bidirectional long short-term memory network, a first linear layer and a second linear layer; The channel is one of an AWGN channel, a uniform fading channel, a Rayleigh fading channel or a Rice fading channel; The semantic communication decoder comprises: a first Dropout layer, a second Dropout layer, a third linear layer, a fourth linear layer, a long short-term memory network, and an embedding layer shared with the semantic communication encoder; The word segmenter is used to segment each text into a word sequence, and the length of the word sequence is standardized to obtain a text sequence of the same length; the embedding layer is used to obtain an embedded representation of the text; the bidirectional long short-term memory network is used to obtain a forward encoding representation and a backward encoding representation of the embedded representation of the text and splice them into an encoding representation of the text; the first linear layer is used to extract the mean of the encoding representation, and the second linear layer is used to extract the logarithmic variance of the encoding representation; The first Dropout layer and the second Dropout layer are used to close the connections of some neurons during training; the third linear layer is used to map the fuzzy semantic coding sequence to the same dimension as the text sequence embedding representation; the long short-term memory network is used to obtain the decoding representation and hidden state of the word; the fourth linear layer is used to change the dimension of the coding representation.

8. The semantic communication system based on multi-objective contrastive learning according to claim 7, characterized in that: The word segmenter also includes a word table, which is used to map each word in the word sequence to a unique identifier, and to standardize the length of the word sequence based on the identifier of each word.

9. A terminal, characterized in that: It includes a processor and a communication interface coupled to the processor, and the processor is used to run a computer program or instruction to implement the semantic communication method based on multi-objective contrastive learning as described in any one of claims 1 to 6.

Images