A semantic communication method, system and terminal based on improved contrastive learning

Abstract

The present invention relates to the fields of artificial intelligence, contrastive learning and semantic communication technology, and proposes a semantic communication method, system and terminal based on improved contrastive learning. The method is completed based on a trained autoencoder model, including: configuring a number of texts of different lengths, inputting the texts of different lengths into a semantic communication encoder in sequence, and obtaining a semantic coding sequence; the semantic coding sequence is transmitted through a channel to become a fuzzy semantic coding sequence, and the fuzzy semantic coding sequence is input into a semantic communication decoder for semantic decoding; constructing contrastive learning positive sample pairs and negative sample pairs; based on the positive sample pairs and negative sample pairs, using an improved contrastive learning loss function to train the autoencoder model, when the loss value does not decrease in a preset number of consecutive trainings, the training is completed; the trained autoencoder model is deployed to the channel for semantic communication. The present invention can ensure the alignment and uniformity of contrastive learning, so that semantic communication has better anti-noise robustness.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence, contrastive learning and semantic communication technology, and in particular to a semantic communication method, system and terminal based on improved contrastive learning. Background Art

[0002] The idea of ​​contrastive learning was first proposed by Hadsell et al. as an effective method for dimensionality reduction. Wang and Isola emphasized two key factors of contrastive loss: alignment and uniformity. Alignment loss ensures a close mapping of similar data, while uniformity loss promotes a uniformly distributed output feature space, retaining maximum information. Many existing contrastive learning methods can be viewed as specific implementations of these two types of losses, such as InfoNCE, which aims to maximize the probability of selecting similar sample pairs in a batch of sample pairs and can be interpreted as a lower bound of the mutual information (MI) between two sample views, which is consistent with the InfoMax principle.

[0003] Chaccour et al. proposed to use contrastive learning to preprocess the data to be transmitted, decomposing the original source data into deep semantic clusters, which can separate learnable data from memorable data and data points belonging to different semantic content elements. Subsequently, the formed semantic deep clusters are sorted according to their confidence, where high-confidence deep semantic clusters are considered to be learnable and semantically rich data, that is, data that can be used to build language in semantic communication systems. Hua et al. used image class labels as an additional source of semantic information and further enhanced contrastive learning by encouraging samples with the same label to be clustered through supervised contrastive learning (SCL). Tang et al. combined contrastive learning with semantic communication, regarded the image damage that occurred during transmission over limited channels as a form of data augmentation operation, and proposed semantic contrastive coding to reduce the semantic similarity between the original image and the reconstructed image, while maintaining the semantic similarity between unrelated images for better recognition. Tian et al. used contrastive learning to independently train a deep learning (DL)-based image encoder, so that the learned encoding is expressive and can be applied to various downstream tasks. To improve the reasoning performance, Wu et al. proposed a collaborative semantic communication scheme based on contrastive learning, which optimizes the extracted features in a coarse-to-fine manner based on the correlation between the extracted features.

[0004] Contrastive learning is a self-supervised representation learning method, which usually obtains positive samples through data enhancement and negative samples through negative sampling, and uses an objective function to increase the distance between positive sample pairs in the encoding space and reduce the distance between negative sample pairs in the encoding space, thereby obtaining a discriminative encoding representation; in terms of semantic communication based on image transmission, different views of an image sample are usually obtained by rotation, cropping, noise addition, etc., and the different views of the image sample are used as positive samples; the construction method of this positive sample relies on prior knowledge, that is, data enhancement will not affect the label of the image; specifically, for an image of a puppy, the image is transformed by rotation, cropping or noise addition (generally the degree of transformation is relatively small), and the image is difficult to become an image of other categories (such as a car image); therefore, the image after data enhancement can be used as a positive sample of the original image; images are usually continuous information representations (although the pixel values ​​are discretized), and the change of a pixel or the overall change of pixels usually does not lead to a change in the semantics of the image.

[0005] However, currently, semantic communication based on contrastive learning is mainly applied to image transmission, while research on semantic communication for text transmission has not been covered in domestic and foreign papers and patents. For semantic communication based on text transmission, the data enhancement method is more complicated. One of the reasons is that text is a discrete information representation. The change of a word or the adjustment of the order is likely to cause the semantics of the sentence to change, especially when the text sequence is very short, which can easily lead to semantic inconsistency between positive sample pairs. In view of this, it is urgent to propose a semantic communication method with noise resistance. Summary of the invention

[0006] The purpose of the present invention is to provide a semantic communication method based on improved contrastive learning, which can ensure the alignment and uniformity of contrastive learning and make the semantic communication have better anti-noise robustness.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] In a first aspect, the present invention provides a semantic communication method based on improved contrastive learning, which is completed based on a trained autoencoder model, the autoencoder model includes: a semantic communication encoder, a semantic communication decoder and a channel, and the semantic communication method includes the following steps:

[0009] S1. Configuring several texts of different lengths, inputting several texts of different lengths into the semantic communication encoder in sequence to obtain a semantic coding sequence;

[0010] S2. The semantic coding sequence is transmitted through the channel to become a fuzzy semantic coding sequence, and the fuzzy semantic coding sequence is input into the semantic communication decoder for semantic decoding;

[0011] S3. Construct positive and negative sample pairs for contrastive learning;

[0012] S4. Based on the positive sample pairs and the negative sample pairs, the autoencoder model is trained using the improved contrastive learning loss function. When the loss value does not decrease after a preset number of consecutive trainings, the training is completed;

[0013] The improved contrastive learning loss function is as follows:

[0014] ;

[0015] In the formula, represents the loss value, Represents semantic encoding, express Enhanced coding, Indicates that the same semantic coding sequence is Other semantic codes besides represents the similarity measure of two encoded features in contrastive learning, represents a class of functions that can be parameterized, represents the distribution of positive sample pairs, represents the data distribution, represents the product of the data itself, that is, the marginal product between any two samples, is a convex function, express The convex conjugate function of , and is monotonically increasing, represents mathematical expectation;

[0016] S5. Deploy the autoencoder model trained in S4 to the channel in S2 for semantic communication.

[0017] As a possible implementation, the positive sample pairs are encoded by a semantic code in a training batch And its corresponding enhanced coding Composition; negative sample pairs are composed of a semantic encoding in a training batch Compared with other encodings in the training batch constitute.

[0018] As a possible implementation method, S1 specifically includes the following sub-steps:

[0019] S10. dividing each text into word sequences, and normalizing the lengths of the word sequences to obtain text sequences of the same length;

[0020] S11. For each text in the text sequence of the same length, one-hot encode the words in the text, and obtain an embedded representation of the text based on the one-hot encoding;

[0021] S12. using a bidirectional long short-term memory network to obtain a forward encoding representation and a backward encoding representation of the embedded representation, and concatenating the forward encoding representation and the backward encoding representation to obtain an encoding representation of the text;

[0022] S13. Extract the mean and logarithmic variance of the encoded representation, reparameterize the mean and logarithmic variance, and obtain the semantic encoding of the text.

[0023] As a possible implementation method, S2 specifically includes the following sub-steps:

[0024] S20. Initialize the embedding representation in S11 to obtain the text sequence embedding representation;

[0025] S21. After adding the fuzzy semantic encoding sequence to the text sequence embedding representation, a long short-term memory network is used to obtain the decoding representation of the first word and the hidden state of the first word;

[0026] S22. Based on the hidden state of the first word, a decoded representation of the second word and a hidden state of the second word are obtained, and the decoded representation of the third word and a hidden state of the third word are obtained based on the hidden state of the second word, until the decoded representations of all words are obtained, and semantic decoding is completed.

[0027] As a possible implementation manner, the preset number of times ranges from 8 to 15 times.

[0028] In a second aspect, the present invention provides a semantic communication system based on improved contrastive learning, which is used to execute the semantic communication method based on improved contrastive learning provided in the first aspect, wherein the semantic communication system is obtained by deploying an autoencoder model trained by an improved contrastive learning loss function to a channel, and includes: a semantic communication encoder, a semantic communication decoder, and a channel;

[0029] The semantic communication encoder includes a word segmenter, an embedding layer, a bidirectional long short-term memory network, a first linear layer, a second linear layer, and a reparameterizer;

[0030] The word segmenter is used to segment each of several texts into word sequences, standardize the length of the word sequences, and obtain text sequences of the same length; the embedding layer is used to obtain the embedded representation of the text; the bidirectional long short-term memory network is used to obtain the forward encoding representation and the backward encoding representation of the embedded representation; the first linear layer is used to calculate the mean of the forward encoding representation and the backward encoding representation; the second linear layer is used to calculate the logarithmic variance of the forward encoding representation and the backward encoding representation; the reparameterizer is used to sample parameters from the standard Gaussian distribution, and use the parameters, mean and logarithmic variance to calculate the semantic encoding;

[0031] The semantic communication decoder comprises: a first dropout layer, a second dropout layer, a third linear layer, a fourth linear layer, a long short-term memory network, and an embedding layer shared with the semantic communication encoder;

[0032] The first Dropout layer and the second Dropout layer are used to close the connections of some neurons during training; the third linear layer is used to map the fuzzy semantic coding sequence to the same dimension as the text sequence embedding representation; the long short-term memory network is used to obtain the decoding representation and hidden state of the word; the fourth linear layer is used to change the dimension of the coding representation.

[0033] As a possible implementation manner, the channel is one of an AWGN channel, a uniform fading channel, a Rayleigh fading channel or a Ricean fading channel.

[0034] As a possible implementation, the word segmenter also includes a vocabulary, which is used to map each word in the word sequence to a unique identifier, and to standardize the length of the word sequence based on the identifier of each word.

[0035] In a third aspect, the present invention provides a terminal, comprising a processor and a communication interface coupled to the processor, wherein the processor is used to run a computer program or instruction to implement the semantic communication method based on improved contrastive learning provided in the first aspect.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] 1. The semantic communication method based on improved contrastive learning proposed in the present invention introduces an improved contrastive learning loss function, generalizes the contrastive learning based on KL divergence through f-divergence, ensures the alignment and uniformity of contrastive learning, and achieves better training effect.

[0038] 2. Compared with semantic communication based on image transmission, semantic communication based on text transmission is more difficult to construct reasonable positive samples in terms of data enhancement. The present invention designs two levels of enhancement methods, one is the enhancement at the word segmentation level, and the other is the enhancement at the encoding feature level. Experiments show that the semantic communication method based on improved contrastive learning has better noise robustness than the baseline method.

[0039] 3. The semantic communication method based on improved contrastive learning proposed in this invention considers two Dropout methods to obtain enhanced positive sample features. The first is the traditional Dropout, which uses probability to represent the features of a certain layer output in the neural network. Closed, obeying Bernoulli distribution; the second is Gaussian Dropout, which multiplies the output with random Gaussian noise to obtain an output with a small amount of noise; for negative samples, the present invention ensures that there are no two semantically similar sentences in the same training batch, and regards a sample in a training batch and other samples as negative sample pairs. The construction of such reasonable samples well guarantees the accurate transmission and effective reconstruction of semantic information. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0041] Figure 1 is a flow chart of a semantic communication method based on improved contrastive learning in an embodiment of the present invention;

[0042] Figure 2 The experimental comparison results of the word segmentation level enhancement method and the semantic coding level enhancement method in the embodiment of the present invention are shown;

[0043] Figure 3 Schematic diagram of a semantic communication system based on improved contrastive learning in an embodiment of the present invention;

[0044] Figure 4 Comparison of semantic communication results between the embodiment of the present invention using improved contrastive learning and not using improved contrastive learning under the BLEU-4 evaluation index;

[0045] Figure 5 The results of semantic communication using improved contrastive learning and not using improved contrastive learning under the ROUGE-L evaluation index in the embodiment of the present invention are compared;

[0046] Figure 6 The results of semantic communication using improved contrastive learning and not using improved contrastive learning under the METEOR evaluation index in an embodiment of the present invention are compared.

[0047] Reference numerals

[0048] 1-semantic communication encoder, 10-word segmenter, 11-embedding layer, 12-bidirectional long short-term memory network, 13-first linear layer, 14-second linear layer, 15-reparameterizer;

[0049] 2-semantic communication decoder, 20-first Dropout layer, 21-second Dropout layer, 22-third linear layer, 23-fourth linear layer, 24-long short-term memory network;

[0050] 3-Channel. DETAILED DESCRIPTION

[0051] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. For example, the first threshold and the second threshold are only used to distinguish different thresholds, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0052] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0053] In the present invention, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The following at least one item (items) or similar expressions thereof refer to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one item (items) of a, b or c can mean: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0054] The present invention provides a semantic communication method based on improved contrastive learning, which can ensure the alignment and uniformity of contrastive learning and make the semantic communication have better anti-noise robustness.

[0055] In a first aspect, the present invention provides a semantic communication method based on improved contrastive learning, see Figure 1 The method is based on a trained autoencoder model, the autoencoder model includes: a semantic communication encoder, a semantic communication decoder and a channel, and the semantic communication method includes the following steps:

[0056] S1. Configuring several texts of different lengths, inputting several texts of different lengths into the semantic communication encoder in sequence to obtain a semantic coding sequence;

[0057] As a possible implementation method, S1 specifically includes the following sub-steps:

[0058] S10. dividing each text into word sequences, and normalizing the lengths of the word sequences to obtain text sequences of the same length;

[0059] As an example, assume that there are 16 texts of different lengths, and each text is divided into several words. The number of words contained in each text is preset, for example, 256. For texts with less than 256 words, special tags are used to fill them; for texts with more than 256 words, the text is truncated into 256-word part and the excess part. For the excess part, it is further determined whether it exceeds 256 words. If it is less, special tags are still used to fill it. If it exceeds, the truncation process continues to be performed to ensure that the number of words contained in the 16 texts is the same; then find the word with the largest length, assuming that the length of the word is , taking the word with the longest length as a reference, add padding marks to other words so that the length of all words becomes , concatenate the words of uniform length according to their positions in the original text, and get 16 text sequences with the same number of words and the same length:

[0060] .

[0061] S11. For each text in the text sequence of the same length, one-hot encode the words in the text, and obtain an embedded representation of the text based on the one-hot encoding;

[0062] As an example, one-hot encoding is done using a vocabulary of 20,000 words, mapping each word to a unique identifier in the vocabulary, thus providing a unique vector representation for each word; these vector representations are then converted into higher-dimensional embedding representations via an embedding layer: , the embedding representation contains the semantic information of each word in the text.

[0063] S12. using a bidirectional long short-term memory network to obtain a forward encoding representation and a backward encoding representation of the embedded representation, and concatenating the forward encoding representation and the backward encoding representation to obtain an encoding representation of the text;

[0064] The Bi-directional Long Short-Term Memory (BiLSTM) network scans the entire text forward and backward, similar to how humans read articles, to obtain the contextual information of the text. For example, the forward encoding obtained by BiLSTM is represented as: ; The backward encoding obtained by BiLSTM is expressed as: , where 64 represents the dimension of the hidden layer. The forward encoding representation and the backward encoding representation are combined to obtain an encoding representation that can capture the semantics of the entire text. , providing a solid foundation for subsequent information transmission and analysis.

[0065] S13. Extract the mean and logarithmic variance of the encoded representation, reparameterize the mean and logarithmic variance, and obtain the semantic encoding of the text.

[0066] As an example, the mean of the encoded representation is expressed as: , the logarithmic variance is expressed as: ,in, b Indicates the number of texts. Represents the dimension of semantic coding, and the semantic coding is obtained by reparameterization as follows:

[0067]

[0068] in, represents the noise randomly sampled from a standard normal distribution, that is, , Represents the Hadamard product.

[0069] S2. The semantic coding sequence is transmitted through the channel to become a fuzzy semantic coding sequence, and the fuzzy semantic coding sequence is input into the semantic communication decoder for semantic decoding;

[0070] As a possible implementation method, S2 specifically includes the following sub-steps:

[0071] Taking the model transmitted on the AWGN channel as an example, the fuzzy semantic coding sequence is expressed as:

[0072]

[0073] in, represents the channel model, represents the channel factor matrix, is random Gaussian noise, and , and Independent of each other, Represents a fuzzy semantic coding sequence.

[0074] As a possible implementation method, S2 specifically includes the following sub-steps:

[0075] S20. Initialize the embedding representation in S11 to obtain the text sequence embedding representation;

[0076] As an example, in the preset autoencoder model, the semantic communication encoder and the semantic communication decoder share the embedding layer, and semantic decoding is performed by two branches. The first branch first defines a starting tag, such as , which is the start marker used to initialize the output sequence: ;after, Enter the embedding layer and obtain the text sequence embedding representation: In this invention, the text sequence embedding representation is processed by the first Dropout layer. The Dropout layer randomly disables the connections of some neurons during the training process. This is to prevent the model from overfitting and maintain its generalization ability. The Dropout layer does not change the dimension of the embedding, so the output Keep in in the space.

[0077] The second branch is the fuzzy semantic coding sequence After a linear layer, the purpose is to map the semantic encoding to the same dimensional space as the embedding layer. , and then expands to to match the dimensions of the embedding representation.

[0078] S21. After adding the fuzzy semantic encoding sequence to the text sequence embedding representation, a long short-term memory network is used to obtain the decoding representation of the first word and the hidden state of the first word;

[0079] For example, the text sequence embedding representation output by the first branch is added to the fuzzy semantic encoding sequence output by the second branch as the input of the Long Short-Term Memory (LSTM) network, which produces two outputs: the decoded representation and the internal state of the LSTM, which includes the cell state and hidden state The decoded representation After the second Dropout layer, another linear layer is used to generate the probability distribution of the next word. Through different decoding strategies, such as greedy strategy, sampling strategy and Top-k strategy, the predicted next word can be obtained. , fill this word into the decoded text set The next iteration is performed at the end of .

[0080] S22. Based on the hidden state of the first word, a decoded representation of the second word and a hidden state of the second word are obtained, and the decoded representation of the third word and a hidden state of the third word are obtained based on the hidden state of the second word, until the decoded representations of all words are obtained, and semantic decoding is completed.

[0081] As an example, the hidden state of the first word will be used as the input of the LSTM network in the next iteration, and this process will be repeated until a special marker indicating the end of the text is decoded. , and get the complete semantic encoding sequence:

[0082] .

[0083] The entire decoding process can be abstracted as a semantic decoder ,in Represents the parameter set of the semantic decoder neural network; when receiving the semantic code transmitted through the channel When , the semantic decoder uses these codes and the partially decoded text information to reconstruct the complete representation of the input text, and obtains ,Right now , in Represents the length of the output text; usually, if two input texts are semantically similar, their semantic encodings should be as close as possible to ensure that the reconstructed text is also semantically consistent; this design ensures accurate transmission of text information and effective reconstruction.

[0084] S3. Construct positive and negative sample pairs for contrastive learning;

[0085] As a possible implementation, the positive sample pairs are encoded by a semantic code in a training batch And its corresponding enhanced coding Composition; negative sample pairs are composed of a semantic encoding in a training batch Compared with other encodings in the training batch constitute.

[0086] S4. Based on the positive sample pairs and the negative sample pairs, the autoencoder model is trained using the improved contrastive learning loss function. When the loss value does not decrease after a preset number of consecutive trainings, the training is completed;

[0087] The improved contrastive learning loss function is as follows:

[0088] ;

[0089] In the formula, represents the loss value, Represents semantic encoding, express Enhanced coding, Indicates that the same semantic coding sequence is Other semantic codes besides represents the similarity measure of two encoded features in contrastive learning, represents a class of functions that can be parameterized, represents the distribution of positive sample pairs, represents the data distribution, represents the product of the data itself, that is, the marginal product between any two samples, is a convex function, express The convex conjugate function of , and is monotonically increasing, represents mathematical expectation;

[0090] See also Figure 2 In the training process, not only the word segmentation level enhancement, including word dropping, replacement, masking and shuffling, but also the semantic coding level enhancement is considered for all texts. Using Dropout, enhanced positive sample features can be directly obtained; therefore, the present invention considers two Dropout methods. The first is the traditional Dropout, which uses probability to output features of a certain layer in the neural network. Closed, obeying Bernoulli distribution; the second is Gaussian Dropout, which multiplies the output with random Gaussian noise to obtain an output with a small amount of noise; for negative samples, the present invention ensures that there are no two semantically similar sentences in the same training batch, and regards a sample in a training batch and other samples as negative sample pairs.

[0091] In order to obtain a more general contrastive learning goal, the present invention uses f-divergence to generalize contrastive learning based on KL divergence, and adopts the generalized contrastive loss function to ensure the alignment and uniformity of contrastive learning; wherein, alignment refers to the close alignment of the encoding representation of the current sample with the encoding representation of the positive sample in the encoding space, which is reflected in the first term of the contrastive loss function; uniformity refers to the uniform distribution of the encoding representations between samples in the encoding space, which is reflected in the second term of the contrastive loss. The first term of the optimization objective is regarded as the similarity score between positive sample pairs with similar feature encodings, and the second term is regarded as the similarity score between two random samples (i.e., negative sample pairs); since It is an increasing function. Maximizing f-mutual information is equal to maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs.

[0092] Exemplarily, the preset number of times ranges from 8 to 15 times, for example: 8 times, 9 times, 10 times, 11 times, 12 times, 13 times, 14 times, 15 times.

[0093] S5. Deploy the autoencoder model trained in S4 to the channel in S2 for semantic communication.

[0094] In a second aspect, the present invention provides a semantic communication system based on improved contrastive learning, which is used to execute the semantic communication method based on improved contrastive learning provided in the first aspect, see Figure 3 , comprising: a semantic communication encoder 1, a semantic communication decoder 2 and a channel 3;

[0095] See also Figure 3 , the semantic communication encoder 1 includes a word segmenter 10, an embedding layer 11, a bidirectional long short-term memory network 12, a first linear layer 13, a second linear layer 14 and a re-parameterizer 15;

[0096] Channel 3 is one of an AWGN channel, a uniform fading channel, a Rayleigh fading channel or a Rice fading channel;

[0097] See also Figure 3 , the semantic communication decoder 2 includes: a first Dropout layer 20, a second Dropout layer 21, a third linear layer 22, a fourth linear layer 23, a long short-term memory network 24, and an embedding layer 11 shared with the semantic communication encoder;

[0098] The word segmenter 10 is used to segment each text into a word sequence, standardize the length of the word sequence, and obtain a text sequence of the same length; the embedding layer 11 is used to obtain the embedded representation of the text; the bidirectional long short-term memory network 12 is used to obtain the forward encoding representation and the backward encoding representation of the embedded representation and concatenate them into the encoding representation of the text; the first linear layer 13 is used to extract the mean of the encoding representation, and the second linear layer 14 is used to extract the logarithmic variance of the encoding representation;

[0099] The first Dropout layer 20 and the second Dropout layer 21 are used to close the connections of some neurons during training; the third linear layer 22 is used to map the fuzzy semantic coding sequence to the same dimension as the text sequence embedding representation; the long short-term memory network 24 is used to obtain the decoding representation and hidden state of the word; the fourth linear layer 23 is used to change the dimension of the coding representation.

[0100] As a possible implementation, the word segmenter 10 further includes a word table, which is used to map each word in the word sequence to a unique identifier, and to standardize the length of the word sequence based on the identifier of each word.

[0101] In a third aspect, the present invention provides a terminal, comprising a processor and a communication interface coupled to the processor, wherein the processor is used to run a computer program or instruction to implement the semantic communication method based on improved contrastive learning provided in the first aspect.

[0102] The semantic communication system and semantic communication method based on improved contrastive learning provided by the present invention are described in detail below in conjunction with specific embodiments.

[0103] This embodiment is carried out on a server with the following specific configurations: Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz, two NVIDIA GeForce RTX 4090 24GB GPUs, and an operating system of Ubuntu 20.04.6 LTS. Simulation experiments are carried out on multiple AWGN channels with different SNRs.

[0104] First, two datasets are constructed, namely the Yelp and Yahoo datasets. For each dataset, it is divided into a training set, a validation set, and a test set. The training set is used for autoencoder model training. The validation set is evaluated after each round of training, and the autoencoder model with the smallest loss value on the validation set is saved. After the model training is completed, the test set is used for performance evaluation.

[0105] First, a word segmenter is used to unify the length of the sentence. The method of this embodiment is to assume that the number of texts of different lengths is 16, and each text is divided into several words. The number of words contained in each text is preset, for example, 256. For texts with less than 256 words, special tags are used for padding; for texts with more than 256 words, the text is truncated into 256-word part and the excess part. For the excess part, it is further determined whether it exceeds 256 words. If it is less than, special tags are still used for padding. If it exceeds, truncation is continued to ensure that the number of words contained in the 16 texts is the same; next, the word with the largest length is found. Assuming that the length of this word is , taking the word with the longest length as a reference, add padding marks to other words so that the length of all words becomes , concatenate the words of uniform length according to their positions in the original text, and get 16 text sequences with the same number of words and the same length:

[0106] .

[0107] In addition, the tokenizer uses a vocabulary of 20,000 words for one-hot encoding, which maps each word to a unique identifier in the vocabulary, thus providing a unique vector representation for each word; these vector embeddings are then converted into higher-dimensional embeddings via an embedding layer: , where 128 is the dimension of the embedding layer. The embedding representation contains the semantic information of each word in the text. Generally speaking, although the embedding representation contains the semantic information of each word in the sentence, it cannot fully represent the semantic information of the entire sentence because it only contains the independent information of each word and has no context-related temporal information. The embedding representation is input into BiLSTM to obtain the forward encoding representation of the embedding representation and the backward encoding representation , where 64 represents the dimension of the hidden layer. Concatenate the forward and backward encoding representations to obtain the encoding representation:

[0108] .

[0109] At this point, the encoded representation can represent the semantic information contained in the entire text, because BiLSTM scans the entire text through forward and backward respectively, which is similar to the process of human reading text and articles, thereby obtaining the contextual information of the text. Extract the mean of the encoded representation: And the log variance: ,in represents the semantic encoding dimension in order to achieve better sentence reconstruction. Finally, using the reparameterization technique, the semantic encoding ,in, represents the noise randomly sampled from a standard normal distribution, that is, , Represents the Hadamard product.

[0110] Next, we take the transmission of the autoencoder model on the AWGN channel as an example. The channel model is expressed as

[0111]

[0112] in, represents the channel model, represents the channel factor matrix, is random Gaussian noise, and , and are independent of each other. For the Gaussian additive white noise (AWGN) channel, is the identity matrix , so the above formula can also be expressed as , which indicates that the AWGN channel contains only additive noise.

[0113] Next, we enter the decoding operation. There are two branches at the beginning of decoding. The first branch first defines a special mark , which is the start marker used to initialize the output sequence: . Then, Enter the embedding layer and obtain the text sequence embedding representation: , and then enters the first Dropout layer. Since the embedding layer is shared with the semantic communication encoder and the Dropout layer does not change the embedding dimension, the output Keep in The second branch encodes the semantic information after the channel transmission. After a linear layer, the semantic encoding is mapped to the same dimensional space as the embedding layer , and then expands to To match the dimension of the embedded representation. Add the outputs of the two branches and use them as the input of the LSTM to get two outputs: one is the decoded representation , the second is the internal state of LSTM, including the cell state and hidden state .

[0114] Decoded representation After the second Dropout layer and the fourth linear layer, the probability distribution of the predicted next word is obtained. Through different decoding strategies, such as greedy, sampling, and Top-k strategies, the predicted next word can be obtained. , fill this word into the decoded text set The next iteration is performed at the end of the decoding. At the same time, the hidden state will also be used as the input of the LSTM in the next iteration. Predefine the decoding end marker Until the end tag is decoded <eos>When , stop the iteration and get the complete semantic encoding sequence:

[0115] .

[0116] Abstract the entire processing process as a semantic decoder ,in Represents the set of parameters of the decoder neural network. When receiving the semantic code transmitted through the channel When , the semantic decoder uses these codes and the partially decoded text information to reconstruct the complete representation of the input text, and obtains ,Right now , in Represents the length of the output text.

[0117] Next, we construct contrastive learning positive and negative sample pairs. Based on the positive and negative sample pairs, we use the improved contrastive learning loss function to train the autoencoder model. During the training process, we consider not only the word segmentation level enhancement, including word dropping, replacement, masking and shuffling, but also the semantic coding level enhancement for all texts. Using Dropout, we can directly obtain enhanced positive sample features; therefore, the present invention considers two Dropout methods. The first is the traditional Dropout, which uses probability to output the features of a certain layer in the neural network. Closed, obeying Bernoulli distribution; the second is Gaussian Dropout, which multiplies the output with random Gaussian noise to obtain an output with a small amount of noise; for negative samples, the present invention ensures that there are no two semantically similar sentences in the same training batch, and regards a sample in a training batch and other samples as negative sample pairs.

[0118] Finally, the trained autoencoder model is deployed on the channel for semantic communication. The text to be transmitted is semantically encoded at the sender to obtain a hidden code; then, the hidden code is transmitted through the channel, which will introduce semantic noise; finally, the semantic code reaches the receiver and is semantically decoded to obtain reconstructed semantic information. Figures 4 to 6 , respectively, are the experimental results of the present invention under three different text generation evaluation indicators: BLEU-4, ROUGE-L, and METEOR. The larger the indicator value, the better the experimental result. It can be seen that the results after using the improved contrastive learning are better than those without the improved contrastive learning.

[0119] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the drawings, etc. In the specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the specification. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0120] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the present invention and its equivalents.< / eos>

Claims

1. A semantic communication method based on improved contrastive learning, characterized in that: The semantic communication method is performed based on a trained autoencoder model, wherein the autoencoder model includes: a semantic communication encoder, a semantic communication decoder, and a channel. The semantic communication method includes the following steps: S1. Configuring several texts of different lengths, inputting the several texts of different lengths into the semantic communication encoder in sequence to obtain a semantic coding sequence; comprising the following sub-steps: S10. Each text is divided into word sequences, and the length of the word sequences is standardized to obtain text sequences of the same length; S11. For each text in the text sequence of the same length, one-hot encode the words in the text, and obtain an embedded representation of the text based on the one-hot encoding; S12. Using a bidirectional long short-term memory network to obtain a forward encoding representation and a backward encoding representation of the embedded representation; S13. Calculating the mean and logarithmic variance of the forward encoding representation and the backward encoding representation, and obtaining the semantic encoding of the text based on the mean and logarithmic variance; S2. The semantic coding sequence is transmitted through the channel to become a fuzzy semantic coding sequence, and the fuzzy semantic coding sequence is input into the semantic communication decoder for semantic decoding; S3. Construct positive and negative sample pairs for contrastive learning; S4. Based on the positive sample pairs and the negative sample pairs, the autoencoder model is trained using an improved contrastive learning loss function, and the training is completed when the loss value does not decrease in the preset number of consecutive trainings; The improved contrastive learning loss function is as follows: ； In the formula, represents the loss value, Represents semantic encoding, express Enhanced coding, Indicates that the same semantic coding sequence is Other semantic codes besides represents the similarity measure of two encoded features in contrastive learning, represents a class of functions that solve for divergence, represents the distribution of positive sample pairs, represents the data distribution, represents the product of the data itself, that is, the marginal product between any two samples, is a convex function, express The convex conjugate function of , and is monotonically increasing, represents mathematical expectation; S5. Deploy the autoencoder model trained in S4 to the channel described in S2 for semantic communication.

2. The semantic communication method based on improved contrastive learning according to claim 1, characterized in that: The positive sample pair is encoded by a semantic code in a training batch. And its corresponding enhanced coding The negative sample pair is composed of a semantic code in a training batch Compared with other encodings in the training batch constitute.

3. The semantic communication method based on improved contrastive learning according to claim 1, characterized in that: The S2 specifically includes the following sub-steps: S20. Initialize the embedding representation described in S11 to obtain the text sequence embedding representation; S21. After adding the fuzzy semantic coding sequence to the text sequence embedding representation, a long short-term memory network is used to obtain a decoding representation of the first word and a hidden state of the first word; S22. Based on the hidden state of the first word, a decoded representation and a hidden state of the second word are obtained, and the decoded representation and the hidden state of the third word are obtained based on the hidden state of the second word, until the decoded representations of all words are obtained, and semantic decoding is completed.

4. The semantic communication method based on improved contrastive learning according to claim 1, characterized in that: The preset number of times ranges from 8 to 15 times.

5. A semantic communication system based on improved contrastive learning, used to execute the semantic communication method based on improved contrastive learning according to any one of claims 1 to 4, characterized in that: include: semantic communication encoder, semantic communication decoder and channel; The semantic communication encoder includes a word segmenter, an embedding layer, a bidirectional long short-term memory network, a first linear layer, a second linear layer and a reparameterizer; The word segmenter is used to segment each of the plurality of texts into a word sequence, and to standardize the length of the word sequence to obtain a text sequence of the same length; the embedding layer is used to obtain an embedding representation of the text; the bidirectional long short-term memory network is used to obtain a forward encoding representation and a backward encoding representation of the embedding representation; the first linear layer is used to extract the mean of the encoding representation, and the second linear layer is used to extract the logarithmic variance of the encoding representation; The semantic communication decoder comprises: a first Dropout layer, a second Dropout layer, a third linear layer, a fourth linear layer, a long short-term memory network, and an embedding layer shared with the semantic communication encoder; The first Dropout layer and the second Dropout layer are used to close the connections of some neurons during training; the third linear layer is used to map the fuzzy semantic coding sequence to the same dimension as the text sequence embedding representation; the long short-term memory network is used to obtain the decoding representation and hidden state of the word; the fourth linear layer is used to change the dimension of the coding representation.

6. The semantic communication system based on improved contrastive learning according to claim 5, characterized in that: The channel is one of an AWGN channel, a uniform fading channel, a Rayleigh fading channel or a Rice fading channel.

7. The semantic communication system based on improved contrastive learning according to claim 5, characterized in that: The word segmenter also includes a word table, which is used to map each word in the word sequence to a unique identifier, and to standardize the length of the word sequence based on the identifier of each word.

8. A terminal, characterized in that: The invention comprises a processor and a communication interface coupled to the processor, wherein the processor is used to run a computer program or instruction to implement the semantic communication method based on improved contrastive learning as described in any one of claims 1 to 4.

Images