Denoising semantic communication system optimization method based on semantic compression

Abstract

The invention discloses a denoising semantic communication system optimization method based on semantic compression. According to the method, firstly, the improved RAKE algorithm is combined with a BERT encoder with fine tuning, so that the problem that the BERT cannot process an overlong text can be solved, and the data volume needing to be transmitted can be remarkably reduced under the condition that key information is not lost; the method comprises the following steps of: receiving semantic information, transmitting the semantic information through a channel, introducing a denoising module, namely a diffusion model noise removal algorithm, at a decoder, modeling random noise through a Markov chain in a diffusion model, and learning the distribution characteristics of the noise so as to denoise the semantic information. According to the method, noise and uncertainty in the data can be effectively processed, so that clearer and more accurate semantic information can be extracted. According to the method, the semantic redundancy problem is effectively solved, the influence of noise is reduced, and the accuracy of text semantics transmitted by a semantic communication system and the generalization ability of a model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of semantic communication, and in particular to a method for optimizing a denoising semantic communication system based on semantic compression. Background Art

[0002] Traditional communication systems focus on accurately transmitting communication symbols without considering the meaning of the transmitted content. In contrast, semantic communication focuses more on conveying the meaning of information, rather than just the symbols themselves. Therefore, semantic communication systems are closer to the core of communication, pay more attention to semantic transmission issues, and are a deeper way of communication. In recent years, with the development of intelligent communication technology, semantic communication has attracted much attention as an emerging communication method. This transformation improves the transmission efficiency of communication systems by converting traditional data information into semantic information for transmission, rather than simply transmitting specific information. Semantic communication represents the future development direction of communication systems and will gradually become a research hotspot in the field of communications.

[0003] Existing research is mostly devoted to extracting and learning semantic information using emerging artificial intelligence algorithms and neural networks. Although some progress has been made, there is still the problem of information redundancy; during the transmission process, the noise problem is inevitable. Therefore, in the context of semantic communication, how to achieve more accurate semantic information expression and how to solve the noise problem are still challenges. Summary of the invention

[0004] In order to solve the technical problems raised by the above background technology, the present invention aims to provide a denoising semantic communication system optimization method based on semantic compression, so as to solve the problems in the existing semantic communication schemes that semantic redundancy fails to effectively save communication resources and channel noise affects the accuracy of semantic decoding.

[0005] In order to achieve the above technical objectives, the technical solution of the present invention is:

[0006] A method for optimizing a denoising semantic communication system based on semantic compression, characterized by comprising the following steps:

[0007] (1) obtaining a sentence to be transmitted, and using an improved Rapid Automatic Keyword Extraction (RAKE) algorithm to extract keywords from the sentence to be transmitted, so as to obtain more concise and effective semantic information;

[0008] (2) Using fine-tuned Bidirectional Encoder Representations from Transformers (BERT) as a semantic encoder to extract semantic features, semantically encode the effective semantic information extracted according to step (1) to obtain a semantic encoding vector, which is convenient for capturing the semantic information in the text and improving the generalization ability of the model;

[0009] (3) Through channel coding, the stability of the semantic coding vector transmitted on the channel is ensured; in the channel encoder, the semantic coding vector is mapped into a complex symbol stream x, and then the coding result is sent to the receiving end through the channel transmission; after the receiving end receives the distorted symbol stream y, the complex symbol is mapped into a vector form through the channel decoder for further processing by the denoising module;

[0010] (4) The semantic vector received through the wireless channel is sent to the proposed denoising module. The denoising module injects random noise according to the principle of Markov Chain (MC) in the diffusion model, thereby learning the distribution of noise in the channel data, and then uses the learned knowledge to remove the channel noise.

[0011] (5) According to the semantic vector obtained by removing noise in step (4), semantic decoding is performed on it to obtain the final semantic information.

[0012] Furthermore, in step (1), a document is first divided into several sentences using punctuation marks such as half-width periods, question marks, exclamation marks, commas, etc., and then for each sentence, stop words such as "and", "the", "of", etc. are used as separators to divide the sentence into several phrases. These phrases are candidates for the keywords finally extracted, and it is allowed to identify word co-occurrence without applying a sliding window of any size. Word co-occurrence refers to the situation where two or more words appear at the same time. The position features, word frequency features, context features, and frequency features of the sentences in which the words are located are introduced to construct a word co-occurrence matrix.

[0013] The word position feature W P Calculated by the following formula:

[0014] W P =log 2 (log 2 (2+Median(Sen w ))) (1)

[0015] Among them, Sen w Indicates the position of word w in its clause, Median(Sen w ) means to take Sen wThe median of the set, the double logarithmic transformation further compresses the variation range of the position eigenvalues, making the position eigenvalues ​​smoother; W P The smaller the value, the more important the word is;

[0016] Word frequency feature W F Calculated by the following formula:

[0017]

[0018] Where freq(w) represents the number of times word w appears in the text, MeanF represents the average number of times all words appear in the text, and σ is the standard deviation of the number of times all words appear, which is used to adjust the degree of difference between different word frequencies; W F The larger the value, the more important the word is;

[0019] Contextual Features H Calculated by the following formula:

[0020]

[0021] Where deg(w) represents the degree of word w. The degree of a word is the number of times a word co-occurs with other words. Every time it co-occurs with a word in a phrase, the degree increases by 1. At the same time, considering that the word itself has a degree of at least 1, F W Indicates the information weight of each word in the short sentence; W H The larger the value, the more important the word is;

[0022] The frequency feature W of the sentence where the word is located D Calculated as follows:

[0023]

[0024] Among them, Sf(w) is the number of sentences containing word w, Sentence all is the total number of sentences in the text; W D The larger the value, the more important the word is;

[0025] Each phrase can be further divided into several words by spaces, and the score of each word is calculated, and then the score of each phrase is accumulated; the word score is calculated by the following formula:

[0026]

[0027] The smaller the word score Score, the more important the word w is. After accumulating the scores of the candidate keywords, the first m candidate keywords with the smallest scores are selected as the key semantic information of the document.

[0028] Furthermore, in step (2), BERT with fine-tuning is used as the semantic encoder, and the text S is defined as [s 1 ,...,s n ], where s n Represents the nth word in the sentence; Use the pre-trained BERT model to segment each text in S, convert each word into a vector representation with a fixed length, and input the obtained word vector together with the sentence encoding and position encoding as features into BERT. First, perform multi-head self-attention and residual connection operations, and then perform feed-forward transmission operations on the obtained residual vector. The feature vector is obtained through two layers of linear mapping and activation with the activation function, namely GELU. Among them, GELU is an activation function based on the Gaussian error function, which combines the linear part and the nonlinear part, and introduces nonlinearity through the error function erf; the formula is as follows:

[0029]

[0030] Where erf(x) is the Gaussian error function, which is defined as:

[0031]

[0032] Furthermore, in step (4), the semantic vector transmitted through the wireless channel will be affected by random channel noise. In order to reduce the impact of noise, a denoising module, namely, a diffusion model noise removal algorithm, is introduced to obtain a more realistic semantic vector. The operation process of the diffusion model noise removal algorithm is implemented as follows:

[0033] First, in the semantic vector containing random channel noise, based on the principle of MC, noise is gradually added to the semantic vector at different time steps, so as to simulate the forward diffusion process of data, so that the semantic vector containing some noise gradually becomes a pure noise vector; assuming x 0 represents the initial semantic vector, C n represents a weight matrix to better simulate the randomness of channel noise; according to the diffusion probability mechanism, with probability q(x t |x 0 ) Step by step, the semantic vector x 0 Add noise, q(x t |x 0 ) indicates that given the initial state x 0 Next, at time step t, x t The conditional probability distribution of is as follows:

[0034]

[0035] Among them, x tThe vector represents the semantic vector at time step t, t∈(1, 2, ..., Z); is a time-varying parameter, representing all the noise adding process parameters α from step 1 to step t i ∈(0,1), that is reflects the information retention at each step from the initial state to the current state, α i It decreases as the time step t increases; is the variance under the influence of noise, which controls the intensity of the noise. As the time step increases, the intensity of the noise gradually increases. In this way, the semantic vector x at any time step can be calculated. t ;

[0036]

[0037] Among them, ò~N(0,I) is a random noise that obeys a Gaussian distribution with a mean of o and a covariance matrix of the identity matrix I;

[0038] Then, the injected noise is gradually removed from the completely chaotic noise data, so that the data is gradually ordered and continuously approaches the real semantic vector, thereby obtaining the real data; in the reverse denoising process, the x in the mean 0 is unknown, this is the target sequence to be generated, so we need to use Gaussian distribution p η (x t-1 |x t ) to fit the inverse process q(x t-1 |x t , x 0 ), to achieve the effect of gradual denoising;

[0039] q(x t-1 |x t , x 0 )~N(x t-1 ; μ(x t , x 0 ), β t I) (10)

[0040] Among them, x t-1 represents the semantic vector at time step -1, with mean μ(x t , x 0 ) and the covariance matrix β t Gaussian distribution of I; β t is the scaling factor in the covariance matrix, reflecting the intensity of the noise; I is the identity matrix;

[0041] Through the Gaussian distribution p η (x t-1 |x t) gradually removes the noise and obtains the target semantic vector; the inverse process is complex and nonlinear, so η (x t-1 |x t ) uses a neural network to predict the mean and covariance matrix of the noise. This neural network is based on the Transformer structure. η is the parameter of the neural network, including the weights and bias terms of all layers. During the training process, η will be adjusted according to the data so that the neural network model can better fit the training data. The neural network can learn the input data x through its complex nonlinear transformation ability. t The complex relationship between the time step t and the output data gives an estimate of the noise term ò η (x t , t), and thus the mean μ is obtained by calculation η (x t , t) and the covariance matrix ∑ η (x t , t); Gaussian distribution p η (x t-1 |x t ) are distributed as follows:

[0042] p η (x t-1 |x t )~N(x t-1 ;μ η (x t , t), ∑ η (x t , t)) (11)

[0043] Among them, the mean μ of the Gaussian distribution η (x t , t) is calculated by the following formula:

[0044]

[0045] Among them, μ η (x t , t) indicates that at the current time step t, by t Subtract an estimate of the noise term from x to reduce t The noise component in makes the result closer to the true value; η (x t , t) represents the neural network model according to the input data x t and time step t to estimate the value of this noise term;

[0046] ∑ η (x t, t) is the covariance matrix, which is calculated by a predefined function and does not require separate training; the covariance matrix describes the degree of change of the data processed at time step t between maintaining the original state and being affected by noise, and is calculated by the following formula:

[0047]

[0048] Among them, α t is the parameter that controls the noise addition process; is the product of all control noise adding process parameters from step 1 to step t-1;

[0049] Thus, we get the data x at time step t-1 t-1 As shown below:

[0050]

[0051] When t = 1, it can be predicted that

[0052]

[0053] Finally, through the reverse process in the diffusion model, the noise distribution in the data can be learned to eliminate the impact of noise on the data; the optimized loss function is used to evaluate the performance of the model, and the model is continuously optimized to reduce the impact of noise on the data. The loss function calculation formula is as follows:

[0054]

[0055] in, It represents the expected value of the mean square error between the real noise and the model predicted noise given the input data and noise; It is a mean square error metric, which represents the square of the error between the real noise and the model predicted noise; this mathematical expectation is used to evaluate the average performance of the model under all possible input and noise conditions. If the expected value is smaller, it means that the model can predict the noise more accurately, thereby more effectively removing the noise in the data, indicating that the model has better performance.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] The present invention uses an improved RAKE algorithm to preprocess text and combines it with a BERT encoder with fine-tuning, which can not only solve the problem that BERT cannot process too long text, but also significantly reduce the amount of data that needs to be transmitted without losing key information, thereby obtaining more representative semantic information; the random process of noise is modeled using Markov chain, the statistical characteristics of channel noise are described, and the distribution characteristics of noise are learned to denoise the semantic information, thereby improving the accuracy of text semantics transmitted by the semantic communication system and the transmission efficiency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the model framework of the present invention;

[0059] Figure 2 is a schematic diagram of the denoising method of the present invention;

[0060] Figure 3 A comparison diagram of simulation results of semantic similarity scores of the present invention and other communication systems on a Gaussian white noise channel; DETAILED DESCRIPTION

[0061] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0062] The present invention designs a denoising semantic communication system optimization method based on semantic compression, such as Figure 1 As shown, the specific steps are as follows.

[0063] Step 1: Obtain the sentence to be transmitted, and use the improved RAKE algorithm to extract keywords from the sentence to be transmitted to obtain more concise and effective semantic information.

[0064] First, a document is divided into several sentences, that is, T = [S 1 , S 2 , ..., S n ], for each clause S i , use stop words as delimiters to divide sentences into several phrases. Stop words are common words that do not have semantic information. Stop words are usually deleted in all information systems and are not included in various text analyses, such as "and", "the", "of", etc. These phrases are used as candidates for the final extracted keywords, and allow the identification of word co-occurrence without applying a sliding window of any size. Word co-occurrence refers to the situation where two or more words appear at the same time. The word position features, word frequency features, context features, and the frequency of occurrence features of the sentence in which the word is located are introduced to construct a word co-occurrence matrix;

[0065] The word position feature W P Calculated by the following formula:

[0066] WP =log 2 (log 2 (2+Median(Sen w ))) (1)

[0067] Among them, Sen w Indicates the position of word w in its clause, Median(Sen w ) means to take Sen w The median of the set, the double logarithmic transformation further compresses the variation range of the position eigenvalues, making the position eigenvalues ​​smoother; W P The smaller the value, the more important the word is;

[0068] Word frequency feature W F Calculated by the following formula:

[0069]

[0070] Where freq(w) represents the number of times word w appears in the text, MeanF represents the average number of times all words appear in the text, and σ is the standard deviation of the number of times all words appear, which is used to adjust the degree of difference between different word frequencies; W F The larger the value, the more important the word is;

[0071] Contextual Features H Calculated by the following formula:

[0072]

[0073] Where deg(w) represents the degree of word w. The degree of a word is the number of times a word co-occurs with other words. Every time it co-occurs with a word in a phrase, the degree increases by 1. At the same time, considering that the word itself has a degree of at least 1, F W Indicates the information weight of each word in the short sentence; W H The larger the value, the more important the word is;

[0074] The frequency feature W of the sentence where the word is located D Calculated as follows:

[0075]

[0076] Among them, Sf(w) is the number of sentences containing word w, Sentence all is the total number of sentences in the text; W D The larger the value, the more important the word is;

[0077] Each phrase can be further divided into several words by spaces, and the score of each word is calculated, and then the score of each phrase is accumulated; the word score is calculated by the following formula:

[0078]

[0079] The smaller the word score Score is, the more important the word w is. After accumulating the scores of the candidate keywords, the first m candidate keywords with the smallest scores are selected as the key semantic information S of the document.

[0080] Step 2: Use the fine-tuned BERT as a semantic encoder to extract semantic features and semantically encode the extracted effective semantic information.

[0081] Define the text S = [s 1 ,...,s n ],in sn Represents the nth word in a sentence; Using the pre-trained BERT model, the input text is first segmented into tokens by a tokenizer, and the tokens are mapped to a high-dimensional space to form token embeddings. This is achieved by looking up a pre-trained embedding matrix that provides a fixed-size vector representation for each token. The resulting word vector is added together with the sentence encoding and position encoding as the feature input X eb .

[0082] The transmitted sentences pass through the semantic encoding layer. The encoder layer is composed of multiple encoders in the Transformer stack, and each encoder includes two sub-layers, namely the multi-head self-attention layer and the feedforward neural network layer. Each sub-layer is followed by a residual connection and a normalization layer. It can encode the input text sequence, learn the relationship between tokens in the sequence, and thus extract the context information of the sequence.

[0083] First, it passes through the multi-head attention layer, which is composed of multiple self-attentions. Each self-attention can learn context information independently, so that different context information can be learned, so that the multi-head self-attention layer can capture richer context information and obtain the vector X at When calculating self-attention, the scaled dot product attention formula is used:

[0084]

[0085] Among them, tensors Q, K, and V are the feature input X eb Multiply by the matrix W q , W k , W v Get, dk It is the dimension of Embedding.

[0086] Then, a residual connection is performed to add the input to the output as the new output, so that the model learning goal becomes the difference between the learning input and output, while also reducing the amplitude of the change between the input and output. The result is then layer normalized to obtain the residual vector X′ at .

[0087] Finally, the residual vector X′ at Perform feed-forward transmission operation, obtain the semantic encoding vector b through two layers of linear mapping and activation with activation function, namely GELU. GELU is an activation function based on Gaussian error function, which combines linear part and nonlinear part, and introduces nonlinearity through error function erf. The formula is as follows:

[0088]

[0089] Where erf(x) is the Gaussian error function, which is defined as:

[0090]

[0091] Step 3: After the source coding, the channel coding is performed. The channel coding is composed of a network conversion layer, namely a Reshape layer, and a network dense layer, namely a Dense layer. In the channel coding, the semantic coding vector is mapped to a complex symbol stream x, and then the coding result is sent to the receiving end through the channel transmission. The present invention takes additive white Gaussian noise, namely AWGN, as an example.

[0092] The receiving end receives the signal y∈C N×c After that, the complex symbols are mapped into vector form by the channel decoder for further processing by the denoising module. Where y = hx + n, h represents the channel coefficient, n ~ CN (0, σ 2 ) represents AWGN. The channel decoder consists of a Reshape layer and a Dense layer.

[0093] Step 4: Send the semantic vector transmitted through the channel to the denoising module. The semantic vector transmitted through the wireless channel will be affected by random channel noise. In order to reduce the impact of noise, a denoising module is introduced, namely the diffusion model noise removal algorithm, to obtain a more realistic semantic vector; its structure is as follows Figure 2 shown.

[0094] First, in the semantic vector containing random channel noise, based on the principle of MC, noise is gradually added to the semantic vector at different time steps, so as to simulate the forward diffusion process of data, so that the semantic vector containing some noise gradually becomes a pure noise vector; assuming x 0 represents the initial semantic vector, C n represents a weight matrix to better simulate the randomness of channel noise; according to the diffusion probability mechanism, with probability q(x t |x 0 ) Step by step, the semantic vector x 0 Add noise, q(x t |x 0 ) indicates that given the initial state x 0 Next, at time step t, x t The conditional probability distribution of is as follows:

[0095]

[0096] Among them, x t The vector represents the semantic vector at time step t, t∈(1, 2, ..., Z); is a time-varying parameter, representing all the noise adding process parameters α from step 1 to step t i ∈(0,1), that is reflects the information retention at each step from the initial state to the current state, α i It decreases as the time step t increases; is the variance under the influence of noise, which controls the intensity of the noise. As the time step increases, the intensity of the noise gradually increases. In this way, the semantic vector x at any time step can be calculated. t ;

[0097]

[0098] Among them, ò~N(0,I) is a random noise that obeys a Gaussian distribution with a mean of 0 and a covariance matrix of the identity matrix I;

[0099] Then, the injected noise is gradually removed from the completely chaotic noise data, so that the data is gradually ordered and continuously approaches the real semantic vector, thereby obtaining the real data; in the reverse denoising process, the x in the mean 0 is unknown, this is the target sequence to be generated, so we need to use Gaussian distribution p η (x t-1 |x t ) to fit the inverse process q(x t-1 |x t , x 0 ), to achieve the effect of gradual denoising;

[0100] q(x t-1 |x t , x 0 )~N(x t-1 ; μ(x t , x 0 ), β t I) (11) Where x t-1 represents the semantic vector at time step t-1, with the mean μ(x t , x 0 ) and the covariance matrix β t Gaussian distribution of I; β t is the scaling factor in the covariance matrix, reflecting the intensity of the noise; I is the identity matrix;

[0101] Through the Gaussian distribution p η (x t-1 |x t ) gradually removes the noise and obtains the target semantic vector; the inverse process is complex and nonlinear, so η (x t-1 |x t ) uses a neural network to predict the mean and covariance matrix of the noise. This neural network is based on the Transformer structure. η is the parameter of the neural network, including the weights and bias terms of all layers. During the training process, η will be adjusted according to the data so that the neural network model can better fit the training data. The neural network can learn the input data x through its complex nonlinear transformation ability. t The complex relationship between the time step t and the output data gives an estimate of the noise term ò η (x t , t), and thus the mean μ is obtained by calculation η (x t , t) and the covariance matrix ∑ η (x t , t); Gaussian distribution p η (x t-1 |x t ) are distributed as follows:

[0102] p η (x t-1 |x t )~N(x t-1 ;μ η (x t , t), ∑ η (x t , t)) (12)

[0103] Among them, the mean μ of the Gaussian distribution η (xt , t) is calculated by the following formula:

[0104]

[0105] Among them, μ η (x t , t) indicates that at the current time step t, by t Subtract an estimate of the noise term from x to reduce t The noise component in makes the result closer to the true value; η (x t , t) represents the neural network model according to the input data x t and time steps, estimate the value of this noise term;

[0106] ∑ η (x t , t) is the covariance matrix, which is calculated by a predefined function and does not require separate training; the covariance matrix describes the degree of change of the data processed at time step t between maintaining the original state and being affected by noise, and is calculated by the following formula:

[0107]

[0108] Among them, α t is the parameter that controls the noise addition process; is the product of all control noise adding process parameters from step 1 to step t-1;

[0109] Thus, we get the data x at time step t-1 t-1 As shown below:

[0110]

[0111] When t = 1, it can be predicted that

[0112]

[0113] Finally, through the reverse process in the diffusion model, the noise distribution in the data can be learned to eliminate the impact of noise on the data; the optimized loss function is used to evaluate the performance of the model, and the model is continuously optimized to reduce the impact of noise on the data. The loss function calculation formula is as follows:

[0114]

[0115] in, It represents the expected value of the mean square error between the real noise and the model predicted noise given the input data and noise; It is a mean square error metric, which represents the square of the error between the real noise and the model predicted noise; this mathematical expectation is used to evaluate the average performance of the model under all possible input and noise conditions. If the expected value is smaller, it means that the model can predict the noise more accurately, thereby more effectively removing the noise in the data, indicating that the model has better performance.

[0116] Step 5: According to the semantic vector after noise removal obtained in step 4, semantic decoding is performed on it through the decoding layer, and the decoded sentence text is processed by probabilistic logistic regression and output to obtain the final semantic information S′.

[0117] Depend on Figure 3 It can be seen that when the method of the present invention is compared with the traditional communication system and the source-channel joint coding, the semantic similarity score of the proposed model is higher than that of other communication systems in different signal-to-noise ratio intervals, and the performance is improved.

Claims

1. A denoising semantic communication system optimization method based on semantic compression, characterized in that: The following steps are involved: (1) obtaining a sentence to be transmitted, and using an improved Rapid Automatic Keyword Extraction (RAKE) algorithm to extract keywords from the sentence to be transmitted, so as to obtain more concise and effective semantic information; (2) Using fine-tuned Bidirectional Encoder Representations from Transformers (BERT) as a semantic encoder to extract semantic features, semantically encode the effective semantic information extracted according to step (1) to obtain a semantic encoding vector, which is convenient for capturing the semantic information in the text and improving the generalization ability of the model; (3) Through channel coding, the stability of the semantic coding vector transmitted on the channel is ensured; in the channel encoder, the semantic coding vector is mapped into a complex symbol stream x, and then the coding result is sent to the receiving end through the channel transmission; after the receiving end receives the distorted symbol stream y, the complex symbol is mapped into a vector form through the channel decoder for further processing by the denoising module; (4) The semantic vector received through the wireless channel is sent to the proposed denoising module. The denoising module injects random noise according to the principle of Markov Chain (MC) in the diffusion model, thereby learning the distribution of noise in the channel data, and then uses the learned knowledge to remove the channel noise. (5) According to the semantic vector obtained by removing noise in step (4), semantic decoding is performed on it to obtain the final semantic information.

2. The method for optimizing a denoising semantic communication system based on semantic compression according to claim 1, characterized in that: In step (1), the improved RAKE algorithm first uses punctuation marks such as half-width periods, question marks, exclamation marks, commas, etc. to divide a document into several sentences, and then for each sentence, uses stop words such as "and", "the", "of", etc. as separators to divide the sentence into several phrases, and these phrases are used as candidates for the keywords finally extracted, and allow word co-occurrence to be identified without applying a sliding window of any size. Word co-occurrence refers to the situation where two or more words appear at the same time, and introduces the position features, word frequency features, context features, and frequency features of the sentences in which the words are located to construct a word co-occurrence matrix; The word position feature W P Calculated by the following formula: W P =log2(log2(2+Median(Sen w ))) (1) Among them, Sen w Indicates the position of word w in its clause, Median(Sen w ) means to take Sen w The median of the set, the double logarithmic transformation further compresses the variation range of the position eigenvalues, making the position eigenvalues ​​smoother; W P The smaller the value, the more important the word is; Word frequency feature W F Calculated by the following formula: Where freq(w) represents the number of times word w appears in the text, MeanF represents the average number of times all words appear in the text, and σ is the standard deviation of the number of times all words appear, which is used to adjust the degree of difference between different word frequencies; W F The larger the value, the more important the word is; Contextual Features H Calculated by the following formula: Where deg(w) represents the degree of word w. The degree of a word is the number of times a word co-occurs with other words. Every time it co-occurs with a word in a phrase, the degree increases by 1. At the same time, considering that the word itself has a degree of at least 1, F W Indicates the information weight of each word in the short sentence; W H The larger the value, the more important the word is; The frequency feature W of the sentence where the word is located D Calculated as follows: Among them, Sf(w) is the number of sentences containing word w, Sentence all is the total number of sentences in the text; W D The larger the value, the more important the word is; Each phrase can be further divided into several words by spaces, and the score of each word is calculated, and then the score of each phrase is accumulated; the word score is calculated by the following formula: The smaller the word score Score, the more important the word w is. After accumulating the scores of the candidate keywords, the first m candidate keywords with the smallest scores are selected as the key semantic information of the document.

3. The method for optimizing a denoising semantic communication system based on semantic compression according to claim 1, characterized in that: In step (4), the semantic vector transmitted through the wireless channel will be affected by random channel noise. In order to reduce the impact of noise, a denoising module, namely the diffusion model noise removal algorithm, is introduced to obtain a more realistic semantic vector. The operation process of the diffusion model noise removal algorithm is implemented as follows: First, in the semantic vector containing random channel noise, based on the principle of MC, noise is gradually added to the semantic vector at different time steps, so as to simulate the forward diffusion process of data, so that the semantic vector containing some noise gradually becomes a pure noise vector; assuming that x0 represents the initial semantic vector, C n represents a weight matrix to better simulate the randomness of channel noise; according to the diffusion probability mechanism, with probability q(x t |x0) gradually adds noise to the semantic vector x0, q(x t |x0) means that given the initial state x0, at time step t, the state x t The conditional probability distribution of is as follows: Among them, x t The vector represents the semantic vector at time step t, t∈(1, 2, ..., Z); is a time-varying parameter, representing all the noise adding process parameters α from step 1 to step t i ∈(0,1), that is reflects the information retention at each step from the initial state to the current state, α i It decreases as the time step t increases; is the variance under the influence of noise, which controls the intensity of the noise. As the time step increases, the intensity of the noise gradually increases. In this way, the semantic vector x at any time step can be calculated. t ; Among them, ò~N(0,I) is a random noise that obeys a Gaussian distribution with a mean of 0 and a covariance matrix of the identity matrix I; Then, the injected noise is gradually removed from the completely chaotic noise data, so that the data is gradually ordered and continuously approached to the real semantic vector, thereby obtaining the real data; in the reverse denoising process, the x0 in the mean is unknown, which is the target sequence to be generated, so the Gaussian distribution p is needed. η (x t-1 |x t ) to fit the inverse process q(x t-1 |x t , x0), achieving the effect of gradual denoising; q(x t-1 |x t (x0)~N(x) t-1 ;μ(x t (x0),β t I) (8) Among them, x t-1 represents the semantic vector at time step t-1, with the mean μ(x t , x0) and the covariance matrix β t Gaussian distribution of I; β t is the scaling factor in the covariance matrix, reflecting the intensity of the noise; I is the identity matrix; Through the Gaussian distribution p η (x t-1 |x t ) gradually removes the noise and obtains the target semantic vector; the inverse process is complex and nonlinear, so η (x t-1 |x t ) uses a neural network to predict the mean and covariance matrix of the noise. This neural network is based on the Transformer structure. η is the parameter of the neural network, including the weights and bias terms of all layers. During the training process, η will be adjusted according to the data so that the neural network model can better fit the training data. The neural network can learn the input data x through its complex nonlinear transformation ability. t The complex relationship between the time step t and the output data gives an estimate of the noise term ò η (x t , t), and thus the mean μ is obtained by calculation η (x t , t) and the covariance matrix ∑ η (x t , t); Gaussian distribution p η (x t-1 |x t ) are distributed as follows: p η (x t-1 |x t )~N(x t-1 ;μ η (x t ,t),∑ η (x t ,t)) (9) Among them, the mean μ of the Gaussian distribution η (x t , t) is calculated by the following formula: Among them, μ η (x t , t) indicates that at the current time step t, by t Subtract an estimate of the noise term from x to reduce t The noise component in makes the result closer to the true value; η (x t , t) represents the neural network model according to the input data x t and time step t to estimate the value of this noise term; ∑ η (x t , t) is the covariance matrix, which is calculated by a predefined function and does not require separate training; the covariance matrix describes the degree of change of the data processed at time step t between maintaining the original state and being affected by noise, and is calculated by the following formula: Among them, α t is the parameter that controls the noise addition process; is the product of all control noise adding process parameters from step 1 to step t-1; Thus, we get the data x at time step t-1 t-1 As shown below: When t = 1, it can be predicted that Finally, through the reverse process in the diffusion model, the noise distribution in the data can be learned to eliminate the impact of noise on the data; the optimized loss function is used to evaluate the performance of the model, and the model is continuously optimized to reduce the impact of noise on the data. The loss function calculation formula is as follows: in, It represents the expected value of the mean square error between the real noise and the model predicted noise given the input data and noise; It is a mean square error metric, which represents the square of the error between the real noise and the model predicted noise; this mathematical expectation is used to evaluate the average performance of the model under all possible input and noise conditions. If the expected value is smaller, it means that the model can predict the noise more accurately, thereby more effectively removing the noise in the data, indicating that the model has better performance.

Images