A deep semantic autoencoder based on turbo coding and decoding

Abstract

The application discloses a kind of deep semantic self-encoder based on Turbo encoding and decoding, including semantic encoder, channel encoder, channel decoder, semantic decoder;Wherein, semantic encoder is used to obtain the semantic feature of original input text according to original input text, in combination with based on multi-head self-attention mechanism;Channel encoder further encodes the semantic feature of the original input text;Channel decoder decodes the received signal transmitted to the receiver, and obtains the semantic feature of input text;Semantic decoder further decodes the semantic feature of input text in combination with based on multi-head self-attention mechanism, and obtains the decoded original input text.The application realizes effectiveness, generalization and robustness under poor environment, i.e.low SNR condition, has high adaptability to poor environment.

Description

Technical Field

[0001] This invention relates to the field of autoencoder technology, and more particularly to a deep semantic autoencoder based on Turbo encoding and decoding. Background Technology

[0002] Today, the challenges posed by complex and ever-changing harsh environments and long-distance communication make achieving reliable communication in adverse conditions increasingly difficult. Communication in harsh environments requires overcoming numerous problems, such as signal attenuation, multipath propagation, atmospheric effects, and water surface fluctuations. Reliable communication is particularly critical in applications such as unmanned vessels, underwater equipment, and coastal surveillance, as these applications require timely and accurate information transmission to ensure vessel safety, monitor harsh environments, and perform missions. Therefore, to address these challenges, researchers have been seeking innovative communication technologies to improve the performance and reliability of communication systems in harsh environments.

[0003] Against this backdrop, communication systems based on Turbo codes have attracted widespread attention. Turbo codes are a powerful error-correcting coding technique that has achieved great success in many fields, including satellite communication, mobile communication, and digital broadcasting. Their strong error-correcting capabilities make them highly promising for communication in harsh environments plagued by signal interference and multipath propagation. However, when facing complex communication environments, traditional Turbo encoding and decoding may require many iterations to achieve satisfactory performance. Frequent parameter adjustments or modifications to the coding scheme are needed under different channel conditions, and their scalability and flexibility are low, resulting in poor adaptability to harsh environments. Summary of the Invention

[0004] This invention provides a deep semantic autoencoder based on Turbo encoding and decoding to overcome the above-mentioned technical problems.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A deep semantic autoencoder based on Turbo encoding and decoding includes a semantic encoder, a channel encoder, a channel decoder, and a semantic decoder; The semantic encoder is an encoder built using a Transformer structure, used to obtain the semantic features of the original input text based on the original input text and combined with a multi-head self-attention mechanism. The channel encoder is an encoder with a Turbo encoder structure, used to further encode the semantic features of the original input text, obtain the transmitter's transmitted signal, and obtain the received signal transmitted to the receiver. The channel decoder is a decoder with a Turbo decoder structure, used to decode the received signal transmitted to the receiver and obtain the semantic features of the input text; The semantic decoder is a decoder built using a Transformer structure. It is used to further decode the semantic features of the input text by combining a multi-head self-attention mechanism to obtain the decoded original input text.

[0006] Beneficial Effects: This invention discloses a deep semantic autoencoder based on Turbo encoding and decoding. It designs a semantic encoder and a semantic decoder with a multi-head attention mechanism to obtain the semantic features of the original input text. The semantic encoder, combined with the multi-head attention mechanism, further encodes the semantic features of the original input text to obtain the received signal transmitted to the receiver. The received signal is then decoded by a channel decoder to obtain the semantic features of the input text. Finally, the semantic decoder with the multi-head attention mechanism further decodes the semantic features of the input text to obtain the decoded original input text. This achieves effectiveness, generalization, and robustness under harsh environments, i.e., low SNR conditions, demonstrating high adaptability to adverse environments. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a schematic diagram of the structure of a deep semantic autoencoder based on Turbo encoding and decoding according to an embodiment of the present invention.

[0009] Figure 2 This is a schematic diagram of a semantic encoder based on a Turbo encoding / decoding deep semantic autoencoder according to an embodiment of the present invention.

[0010] Figure 3 This is a schematic diagram of a channel encoder based on a Turbo codec deep semantic autoencoder according to an embodiment of the present invention.

[0011] Figure 4 This is a schematic diagram of a channel decoder for a deep semantic autoencoder based on Turbo encoding and decoding, according to an embodiment of the present invention.

[0012] Figure 5 This is a schematic diagram of a semantic decoder based on a Turbo-encoded deep semantic autoencoder according to an embodiment of the present invention.

[0013] Figure 6 This is a schematic diagram comparing the BLEU performance of a deep semantic autoencoder based on Turbo encoding and decoding under different Rice factors, according to an embodiment of the present invention.

[0014] Figure 7 This diagram illustrates the performance comparison of the SS (Search Engine Response) of a deep semantic autoencoder based on Turbo encoding and decoding under different Rice factors, according to an embodiment of the present invention.

[0015] Figure 8 This is a schematic diagram comparing the BER performance of a deep semantic autoencoder based on Turbo encoding and decoding under different Rice factors according to an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] This embodiment provides a deep semantic autoencoder (Turbo-DSA) based on Turbo encoding and decoding, such as... Figure 1 This includes semantic encoders, channel encoders, channel decoders, and semantic decoders; The semantic encoder is an encoder built using the Transformer structure, used to obtain the semantic features of the original input text based on the original input text; The channel encoder is an encoder with a Turbo encoder structure, used to further encode the semantic features of the original input text, obtain the transmitter's transmitted signal, and obtain the received signal transmitted to the receiver. The channel decoder is a decoder with a Turbo decoder structure, used to decode the received signal transmitted to the receiver and obtain the semantic features of the input text; The semantic decoder is a decoder built using the Transformer structure, used to further decode the semantic features of the input text to obtain the decoded original input text.

[0018] Specifically, to achieve reliable transmission of semantic information in complex and harsh environmental channels, we construct a deep semantic autoencoder based on Turbo encoding and decoding, such as... Figure 1As shown, in communications in harsh environments, we use a drone as the transmitter, which includes a semantic encoder and a channel encoder, responsible for extracting the semantic features of the raw information and encoding them for transmission. The receiver is an unmanned surface vessel (USV), which includes a channel decoder and a semantic decoder. Its task is to decode the transmitted information, understand its meaning, and convert it into corresponding commands.

[0019] Specifically, the semantic encoder employs a Transformer architecture to extract semantic features from the input text. The channel encoder uses a Turbo encoder architecture, further encoding the semantic features output by the semantic encoder using component coding techniques, adding redundant information to obtain the transmitter's transmitted signal. This transmitted signal is transmitted through the channel to become the receiver's received signal. The channel decoder uses a Turbo decoder architecture, reducing the impact of channel noise on the received signal through iterative decoding techniques, and decoding the received signal to obtain the semantic features of the transmitted text. The semantic decoder, also based on a Transformer architecture, can further decode the semantic features, converting them back into the original text.

[0020] Preferred, such as Figure 2 As shown, the semantic encoder includes multiple Transformer encoder layers; the multiple Transformer encoder layers are connected end to end in sequence, wherein the first Transformer encoder layer receives the original input text; and the last Transformer encoder layer outputs the semantic features of the original input text. The Transformer encoder layer includes a multi-head self-attention mechanism module and a position-feedforward neural network; the multi-head self-attention mechanism module is used to compute a representation for each position in the input sequence based on the original input text, and obtain the output of the multi-head attention. To obtain the input of the position feedforward neural network. And based on the input of the position feedforward neural network, obtain the output of the feedforward neural network. ; to obtain the output of the semantic encoder; Preferably, the output of the feedforward neural network is obtained. The method is as follows: Specifically, the semantic encoder is composed of It consists of stacked Transformer encoder layers. Input text After the embedding layer, we obtain the word representation vector. This serves as the input to the Transformer encoder layer. Each Transformer encoder layer consists of two sub-layers: a multi-head self-attention mechanism module and a position-feedforward neural network. The multi-head self-attention mechanism module contains... An attention mechanism (or rather, an attention mechanism) The system uses multiple attention heads to compute in parallel to model the global relations of the input sequence. It can compute a representation for each position in the input sequence while taking into account information from other positions in the input sequence, thus capturing more different types of relations and semantic information.

[0021] S11: Obtain the output of the multi-head self-attention mechanism as follows:

[0022] in, This represents a concatenation function that joins together multiple attention heads; Here, represents the additional weight matrix, and represents the trainable parameters; Indicates the total number of heads of attention; Indicates the first One point of attention;

[0023] in, Indicates the activation function; , , The first Each attention head's query, key, and value, they all come from the same... Derived from a linear transformation. express Dimension size; in,

[0024] in, , , All are learnable parameter matrices; For input text Word representations are obtained after the embedding layer; S12: Output of multi-head attention Summation and normalization are performed to obtain the input of the position feedforward neural network. ; Specifically, summing and normalizing the output of multi-head attention are standard techniques in the field and will not be described in detail here.

[0025] Specifically, a feedforward neural network is a fully connected feedforward neural network with two linear layers and an activation function (such as ReLU). It is applied independently at each position, enabling nonlinear transformations at each position in the sequence, further improving the model's representational power.

[0026] S13: Based on the input of the position-based feedforward neural network The output of the position feedforward neural network is obtained, and it is represented as:

[0027] in, , All of them are learnable weight matrices. , All are learnable bias vectors; This represents the function that takes the maximum value. This represents the output of the position-fedforward neural network; Specifically, after summation and normalization, we obtain the first... The output of the Transformer encoder layer As the ( The encoder is the input of one Transformer encoder layer. Multiple encoder layers are stacked, and the output of each encoder layer is used as the input of the next layer, enabling the transfer and processing of information between layers. After processing by one Transformer encoder layer, the output of the semantic encoder is obtained. , is the semantic representation of each word in the context, and serves as the input to the channel encoder.

[0028] The output of the semantic encoder is the semantic feature of the original input text. The semantic encoder of the deep semantic autoencoder based on Turbo encoding and decoding uses a Transformer encoder to encode the input sequence and extract its semantic information. The signal from the semantic encoder can be represented as:

[0029] in, This represents a semantic encoding function. This indicates the input text. Represents the parameters of the semantic encoder; This represents the semantic features extracted from the input text.

[0030] Preferred, such as Figure 3 As shown, the channel encoder includes a first component encoder, a second component encoder, and a main encoder; The first component encoder is used to obtain a sequence based on the output of the semantic encoder. It includes a first interleaver and a first convolutional layer; the first interleaver is connected to the output of the semantic encoder, and the input of the first convolutional layer is connected to the output of the first interleaver; The second component encoder is used to obtain an output sequence based on the output of the semantic encoder. It includes a second interleaver and a second convolutional layer; the second interleaver is connected to the output of the semantic encoder, and the input of the second convolutional layer is connected to the output of the second interleaver. The master encoder is used to obtain the sequence of the master encoder based on the output of the semantic encoder. The main encoder includes a third convolutional layer; the third convolutional layer is connected to the output of the semantic encoder. The outputs of the first convolutional layer, the second convolutional layer, and the third convolutional layer are all connected to the channel decoder to acquire the input signal of the channel decoder. Specifically, the channel encoder of the deep semantic autoencoder based on Turbo encoding and decoding adopts a Turbo encoder structure. This structure uses a parallel concatenated coding method, introducing redundant information to enhance the fault tolerance of signal transmission. Turbo encoders are widely used in modern communication systems to improve the reliability and fault tolerance of data transmission. Through the use of component coding algorithms, Turbo encoders enable communication systems to better cope with noise interference and channel loss. The Turbo encoder consists of three parts: a main encoder, a first component encoder, and a second component encoder. Turbo encoders typically use convolutional coding, but in this embodiment of the deep semantic autoencoder based on Turbo encoding and decoding, the Turbo encoder is implemented using a CNN. The component encoders rearrange the input sequence, changing the temporal relationship between bits to enhance their ability to resist channel errors.

[0031] The Turbo encoder's structure increases the system's fault tolerance and improves the transmission quality of communication signals in noisy channels by adding a channel encoder.

[0032] Preferably, the method for obtaining the input signal of the channel decoder is as follows: S21: Obtain the sequence from the main encoder. as follows: Specifically, the output signal of the Turbo encoder is a sequence of semantic features of the text, that is, a signal. .Signal Through the sequence of the main encoder Represented as:

[0033] in, It is the main encoder function; This represents the semantic features extracted from the input text; S22: Obtain the sequence generated after encoding by the first component encoder. as follows: Specific signals After passing through the first interleaver, the sequence is generated after being encoded by the first component encoder. It can be represented as:

[0034] in, It is a function of the first component encoder; S23: Obtain the output sequence of the second component encoder as follows: Specifically, the structure of the second component encoder is the same as that of the first component encoder; the only difference lies in the rearrangement order of the input sequence by the interleaver. The output sequence of the second component encoder... Represented as:

[0035] in, It is a function of the second component encoder.

[0036] S24: Generate a sequence based on the encoding performed by the first component encoder. The output sequence of the second component encoder and the sequence of the main encoder Obtain the encoded bit sequence as follows: Specifically, the outputs of the master encoder and the component encoders are concatenated in parallel to form the encoded bit sequence. Represented as:

[0037] S25: Based on the encoded bit sequence The input signal of the channel decoder is obtained as follows: Specifically, when transmitting signals over ocean channels, the signals are affected by fading and additive white Gaussian noise, which impacts both transmission and reception. The signal transmitted through the channel is represented as:

[0038] in, This indicates the signal emitted by the transmitter. This refers to the signal received by the receiver, i.e., the input signal of the channel decoder. Indicates a noise signal; This represents the channel fading coefficient.

[0039] Specifically, in this embodiment, the signal transmitted through the channel encoder can also be represented as:

[0040] in, Represents the channel coding function. These represent the parameters of the channel encoder.

[0041] Preferred, such as Figure 4 As shown, the channel decoder includes a third interleaver, a fourth convolutional layer, a first deinterleaver, a fourth interleaver, a fifth convolutional layer, and a second deinterleaver; The input of the third interleaver is connected to the output of the channel encoder; The input of the fourth convolutional layer is connected to the output of the channel encoder and the output of the third interleaver, respectively; the output of the fourth convolutional layer is connected to the input of the first deinterleaver. The input terminal of the fourth interleaver is connected to the output terminal of the channel encoder and the output terminal of the first deinterleaver, respectively. The input of the fifth convolutional layer is connected to the output of the fourth interleaver and the output of the channel encoder, respectively; the output of the fifth convolutional layer is connected to the second deinterleaver, and the second deinterleaver is connected to the input of the third interleaver and the semantic decoder, respectively.

[0042] Specifically, the channel decoder of the deep semantic autoencoder based on Turbo encoding and decoding employs a Turbo decoder, which improves the decoding accuracy of the received encoded data through multiple iterations. The Turbo decoder consists of two identical iterative decoders, referred to as the forward iterative decoder and the backward iterative decoder. The Turbo decoder uses an iterative decoding algorithm, which is achieved by repeatedly running the forward and backward iterative decoders alternately.

[0043] Specifically, in this embodiment, the Turbo encoder uses three signals (the first component encoder generates a sequence after encoding). The output sequence of the second component encoder and the sequence of the main encoder The signals are sequentially concatenated to form a single signal r, which is then output and fed into the Turbo decoder. At the receiving end, the received signal... After serial-to-parallel conversion, the input signal is divided into three sequences. The Turbo decoder then sequentially separates the input signal into three signals: a system information sequence and a system information sequence. First verification information sequence and the second verification information sequence The above signal conversions all utilize existing technologies and will not be described in detail here. System information sequence First verification information sequence and the second verification information sequence They are respectively represented as

[0044] System information sequence After passing through the third interleaver, a system information sequence after interleaving is generated. Prior information sequence It is generated from external information produced by the reverse iterative decoder, and then de-interleaved. During the first iteration, Prior information sequence After passing through the third interleaver, a sequence of prior information is generated after interleaving. The sequence of prior information after interleaving. The system information sequence after interleaving and the first verification information sequence Together they pass through the forward iterative decoder to generate a sequence. , can be represented as

[0045] In the formula: This represents the sequence of prior information generated after interleaving. This represents the system information sequence after interleaving. This represents the first verification information sequence; It is a function of the forward iterative decoder. Sequence After processing by the first deinterleaver and the fourth interleaver, a new sequence of prior information can be obtained. It and and It is irrelevant, therefore it can be used as prior information for the reverse iterative decoder. System information sequence A new system information sequence is generated after passing through the fourth interleaver. New sequence of prior information New system information sequence Second verification information sequence Together they pass through a reverse iterative decoder to generate a second sequence. , can be represented as

[0046] In the formula: This represents a new sequence of prior information; This represents a new sequence of system information; This represents the second verification information sequence; It is a function of the reverse iterative decoder.

[0047] second sequence After the second deinterleaver, a new sequence of prior information is generated. Repeat the above steps. Through continuous feedback, interleaving, and deinterleaving operations, each iteration corrects the information obtained in the previous iteration and then passes the corrected information to the next iteration. The Turbo decoder can iteratively decode the received signal and gradually correct and recover the semantic features of the text. After many iterations, the Turbo decoder gradually converges. If the set number of iterations is reached, then the sequence... After passing through the second deinterleaver, no further iterations are performed; the signal is directly used as the output signal. Output.

[0048] Specifically, the output signal of the channel decoder can be represented as

[0049] in, Represents the channel decoding function. This represents the parameters of the channel decoder.

[0050] Preferred, such as Figure 5 As shown, the semantic decoder includes multiple DecoderLayer layers; the multiple DecoderLayer layers are connected end to end in sequence, wherein the first DecoderLayer layer receives the output signal of the channel decoder; and the last Transformer layer outputs the decoded original input text.

[0051] The deep semantic autoencoder based on Turbo encoding and decoding uses a Transformer decoder for decoding, which consists of multiple stacked DecoderLayers. Each DecoderLayer consists of two sub-layers: a self-attention mechanism, an encoder-decoder attention mechanism, and a position-feedforward neural network.

[0052] The self-attention mechanism is used to compute a representation based on the decoder's own output sequence, further capturing the relationship between the input and output sequences. The encoder-decoder attention mechanism is used to align the encoder's output sequence with the decoder's current position, achieving efficient utilization of the encoder's representation. The position-feedforward neural network is the same as the position-feedforward neural network in the encoder.

[0053] Similar to an encoder, a decoder is constructed by stacking multiple decoder layers. The decoder's output is a conditional probability distribution for generating the next target word based on the preceding context and the currently generated partial target sequence. The signal from the semantic decoder can be represented as...

[0054] in, This represents a semantic decoding function. The parameters represent the semantic decoder.

[0055] Specifically, the Turbo-based deep semantic autoencoder is an end-to-end model for data transmission. The semantic encoding function... Channel coding function is used to encode the input sequence and extract its semantic information. The channel function is used to encode the output of the semantic encoder into a sequence of symbols. The process of simulating channel transmission, channel decoding function Used to decode the symbol sequence received by the channel decoder, and finally the semantic decoding function. The semantic information is recovered by calculating the representation based on the output sequence of the decoder itself.

[0056] After the design of the deep semantic autoencoder based on Turbo encoding and decoding in this embodiment is completed, the entire deep semantic autoencoder based on Turbo encoding and decoding is trained based on the training set. The parameter set is updated using gradient descent, and the loss function is minimized until the loss function converges. Then, the training is stopped and the corresponding parameter set is saved. Specifically, in the design of the Turbo-encoder-decoder-based deep semantic autoencoder, our goal is to minimize the semantic difference between the prediction and the reference sentence and maximize the semantic understanding rate through end-to-end training. We use the cross-entropy loss function to calculate the difference between the predicted and true values ​​for each word, and sum them across all words to obtain the total loss. Specifically, the loss function can be expressed as the formula...

[0057] in Sentences expressing prediction index, Indicates the number of samples. Indicates the first The length of each sample Indicates the first In the nth sample The true value of each label Indicates the first In the nth sample The predicted value for each label, It is a very small constant, to avoid in calculations Dividing by zero will produce an error.

[0058] The training process of the deep semantic autoencoder based on Turbo encoding and decoding in this embodiment is as follows: Input training dataset Initialize the number of iterations. Then, iterative training using gradient descent is performed until the loss function converges. In each iteration, the following steps are executed. In the transmitter, the signal... Encoded by a semantic encoder Encoded by the channel encoder The transmitter will send a signal It is sent into the channel for transmission. Within the channel, it is processed according to specific channel parameters. and noise Processing transmitted signals Received signal In the decoder, the received signal Decoded by the channel decoder Decoded into a semantic decoder The output signal is obtained. According to the input signal and output signal Calculate the loss function. Adjust the parameters using gradient descent. Repeat this process until the loss function converges. Finally, save the trained network parameters of the deep semantic autoencoder based on Turbo encoding and decoding. , , , And output it.

[0059] After training the Turbo-encoder-decoder-based deep semantic autoencoder, it is tested on a test set, and the optimal parameter set is determined based on the test results. Specifically, during the testing process, the input includes the test dataset. Network parameters of a deep semantic autoencoder based on Turbo encoding and decoding , , , The signal-to-noise ratio (SNR) range is also considered. The output is the recovered sequence at different SNR values. In the test algorithm, for each SNR value, the following steps are performed. In the transmitter, the signal... Encoded by a semantic encoder Encoded by a semantic encoder The transmitter will send a signal The signal is sent into the channel for transmission. The signal processing in the channel is the same as during training, resulting in the received signal. In the decoder, the received signal Decoded by the channel decoder Decoded into a semantic decoder The output signals under different SNR values ​​were obtained. .

[0060] A specific simulation experiment embodiment of the present invention is as follows: To evaluate the semantic extraction performance of the Turbo-encoded deep semantic autoencoder, the simulation experiment focused on the transmission of data segments. The transmission quality of data segments is crucial to user experience and application performance. In a communication network, a data segment consists of a header and a payload. The header includes control information such as source address, destination address, and sequence number. The payload refers to the actual transmitted data content, such as binary data like text, images, audio, or video. The size and type of the payload depend on the nature of the transmitted data and application requirements. Since the payload carries the actual data content, the focus is primarily on the payload portion for data transmission and processing. In this embodiment, the channel module is a wireless channel. Taking a Ricean fading channel as an example, Table 1 shows the training parameters used by the Turbo-encoded deep semantic autoencoder:

[0061] In this experiment, we used the Adam optimizer with an initial learning rate of 0.001. Network performance metrics included BLEU, SS, and BER. The batch size during training was 128, and the training signal-to-noise ratio was set to 3 dB. These parameters play a crucial role in the training process of the Turbo-encoder-based deep semantic autoencoder, contributing to good performance. The performance of the Turbo-encoder-based deep semantic autoencoder was optimized by updating the parameter set.

[0062] In this embodiment, the performance metrics include BLEU (Bilingual Evaluation Understudy) performance, SS (Sentence Similarity) performance, and BER (Bit Error Rate) performance.

[0063] exist Figure 6 The text demonstrates the Turbo-based deep semantic autoencoder in different models (1-gram, 2-gram, 3-gram, and 4-gram) and different Ricean implementations. Factors (1, 5, 9) and BLEU values ​​under different SNR conditions. For different n-gram models, the Turbo-encoded deep semantic autoencoder exhibits high BLEU values ​​under all SNR conditions. Specifically, under the 1-gram model, the Turbo-encoded deep semantic autoencoder (Rice)... Factors of 1, 5, and 9 all exhibited the highest BLEU values ​​at different SNR levels. Especially under low SNR conditions, such as -10dB and -7dB, the Turbo-based deep semantic autoencoder (Rice) showed the highest BLEU values. The BLEU values ​​with factors of 1, 5, and 9 are significantly superior, demonstrating stronger robustness in harsh environments. Even with an SNR greater than 0 dB, the Turbo-encoded deep semantic autoencoder still exhibits superior performance. Furthermore, the Turbo-encoded deep semantic autoencoder is competitive in 2-gram, 3-gram, and 4-gram models, and generally performs better under various SNR conditions.

[0064] Regarding different Rice The Turbo-based deep semantic autoencoder exhibits strong performance robustness. Especially under low SNR conditions, such as -10dB and -7dB, the BLEU value of the Turbo-based deep semantic autoencoder is relatively high. Regardless of Rice... Whether the factor is 1, 5, or 9, the deep semantic autoencoder based on Turbo encoding and decoding maintains a high BLEU value under various SNR conditions, demonstrating its adaptability to different Ricean factors. The adaptability of the factor. Furthermore, as the SNR increases, the BLEU value of the Turbo-encoded deep semantic autoencoder remains consistent under different SNR conditions, while under high SNR conditions (8dB), the Turbo-encoded deep semantic autoencoder typically performs best. The Turbo-encoded deep semantic autoencoder exhibits stable performance in communication across various noise levels, demonstrating particular adaptability in harsh environments.

[0065] exist Figure 7 The image shows the Turbo-based deep semantic autoencoder in different Ricean models. SS values ​​under factors (1, 5, 9). For Rice With a factor of 1, the SS value of the Turbo-based deep semantic autoencoder ranges from 0.9261 to 0.9774. (In Rice) With factors of 5 and 9, the SS values ​​of the Turbo-based deep semantic autoencoder range from 0.9524 to 0.9738 and from 0.9567 to 0.9754, respectively. These data clearly demonstrate the excellent performance of the Turbo-based deep semantic autoencoder under different factor conditions.

[0066] As the SNR increases, the SS value of the Turbo-encoded deep semantic autoencoder shows an increasing trend, indicating its enhanced capabilities in semantic extraction and recovery. The Turbo-encoded deep semantic autoencoder exhibits superior BLEU performance under various SNR conditions, thanks to its superior semantic encoder and decoder. Furthermore, because the Turbo-encoded deep semantic autoencoder uses a traditional Turbo encoder and decoder from a communication system as the channel encoder and decoder, rather than a simple neural network design, its BLEU performance is even more outstanding. Especially under conditions where the SNR is less than 0 dB, the Turbo-encoded deep semantic autoencoder demonstrates superior BLEU performance, showcasing robustness and adaptability to harsh environments in semantic extraction and recovery.

[0067] exist Figure 8 The image shows the Turbo-based deep semantic autoencoder in different Ricean models. BER values ​​under factors (1, 5, 9). The Turbo-based deep semantic autoencoder exhibits significantly lower BER values ​​under all SNR conditions. Particularly under low SNR conditions (-10dB and -7dB), the BER values ​​of the Turbo-based deep semantic autoencoder are significantly low, demonstrating excellent performance in low SNR environments. Under high SNR conditions, the Turbo-based deep semantic autoencoder consistently exhibits the lowest bit error rate. With increasing SNR, the performance advantage of the Turbo-based deep semantic autoencoder further expands, indicating its greater sensitivity to SNR.

[0068] Next, for different Rice Comparison of BER performance under different factors. (In Rice) With a factor of 1, the Turbo-based deep semantic autoencoder exhibits the lowest BER value across all SNR conditions, highlighting its performance in low Rice levels. Stability performance under certain factor conditions. (In Rice) Factors 5 and Rice Even with a factor of 9, the Turbo-based deep semantic autoencoder still maintains a low BER value, demonstrating its effectiveness across different Ricean factors. Excellent performance under various factor conditions. Especially under extremely low SNR conditions (-10dB and -7dB), the Turbo-based deep semantic autoencoder still exhibits the lowest BER value, highlighting its robustness to harsh channels and high-noise environments. This robustness makes the Turbo-based deep semantic autoencoder more reliable in practical applications, especially in harsh environments requiring stable data transmission.

[0069] In summary, the Turbo-encoder-based deep semantic autoencoder exhibits superior BLEU, SS, and BER performance under various conditions. As a semantic communication system, this Turbo-encoder-based deep semantic autoencoder can effectively achieve semantic extraction and recovery of text, especially demonstrating significant advantages under low SNR conditions. Under the given conditions, the deep semantic autoencoder based on Turbo encoding and decoding exhibits robustness and data transmission capability, making it suitable for harsh environments.

[0070] The deep semantic autoencoder based on Turbo encoding and decoding in this embodiment integrates the advantages of nonlinear transformation and deep source-channel joint coding. The deep semantic autoencoder based on Turbo encoding and decoding utilizes a Transformer and a Turbo encoder-decoder structure to extract semantic features from text. The proposed structure of the deep semantic autoencoder based on Turbo encoding and decoding includes a semantic encoder, a channel encoder, a channel decoder, and a semantic decoder. The semantic encoder and semantic decoder utilize a Transformer to extract and reconstruct text features. The channel encoder and channel decoder utilize a Turbo encoder and Turbo decoder structure to extract and reconstruct text features. These encoders and decoders cooperate with each other, enhancing the effectiveness of the model's source-channel joint coding.

[0071] This embodiment addresses the overfitting problem of the Turbo-encoded deep semantic autoencoder by employing a backpropagation algorithm to train the Turbo-encoded deep semantic autoencoder, thereby optimizing parameters more quickly and improving overall performance. The learning results of the Turbo-encoded deep semantic autoencoder enable efficient and fast text semantic feature transmission. Furthermore, a testing algorithm for the Turbo-encoded deep semantic autoencoder under different channel conditions is used to evaluate the generalization ability of the trained Turbo-encoded deep semantic autoencoder to SNR.

[0072] This embodiment demonstrates the performance of a Turbo-encoded deep semantic autoencoder in a Ricean fading channel. Results show that for text data transmission, the proposed Turbo-encoded deep semantic autoencoder achieves superior semantic coding and transmission performance across various evaluation metrics. The Turbo-encoded deep semantic autoencoder improves semantic transmission performance under different channel conditions. Particularly in harsh environments, i.e., under low SNR conditions, simulation results verify the effectiveness, generalization, and robustness of the Turbo-encoded deep semantic autoencoder, exhibiting high adaptability to adverse environments.

[0073] This embodiment of the deep learning-based autoencoder communication system uses an encoder to compress and represent the input data, transforming it into a latent space code. This code is an abstract representation of the data, typically with low dimensionality, which helps reduce redundancy in transmitted data and improve channel utilization. The encoded data is then transmitted to the receiver via the communication channel. The decoder restores the received encoded data to the original data. This decoding process involves channel noise repair and information reconstruction, where the deep learning neural network can improve its performance through learning and training. The deep learning neural network can also improve the performance of Turbo codes by learning the characteristics of the communication channel. This adaptability allows the deep learning-based autoencoder communication system to perform well under different channel conditions without frequent parameter adjustments or modifications to the encoding scheme. Furthermore, the deep learning-based autoencoder communication system exhibits good scalability and flexibility, and is well-adapted to harsh environments. Researchers can design and train autoencoder networks according to different communication needs and channel environments to achieve customized communication solutions.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this utility model, and are not intended to limit it. Although the utility model has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this utility model.

Claims

1. A deep semantic autoencoder based on Turbo encoding and decoding, characterized in that, This includes semantic encoders, channel encoders, channel decoders, and semantic decoders. The semantic encoder is an encoder built using a Transformer structure, used to obtain the semantic features of the original input text based on the original input text and combined with a multi-head self-attention mechanism. The channel encoder is an encoder with a Turbo encoder structure, used to further encode the semantic features of the original input text, obtain the transmitter's transmitted signal, and obtain the received signal transmitted to the receiver. The channel encoder includes a first component encoder, a second component encoder, and a main encoder; The first component encoder is used to obtain a sequence based on the output of the semantic encoder. ; The first component encoder includes a first interleaver and a first convolutional layer; the first interleaver is connected to the output of the semantic encoder, and the input of the first convolutional layer is connected to the output of the first interleaver. The second component encoder is used to obtain an output sequence based on the output of the semantic encoder. The second component encoder includes a second interleaver and a second convolutional layer; the second interleaver is connected to the output of the semantic encoder, and the input of the second convolutional layer is connected to the output of the second interleaver. The master encoder is used to obtain the sequence of the master encoder based on the output of the semantic encoder. The main encoder includes a third convolutional layer; the third convolutional layer is connected to the output of the semantic encoder. The outputs of the first convolutional layer, the second convolutional layer, and the third convolutional layer are all connected to the channel decoder; Used to acquire the input signal of the channel decoder; The channel decoder is a decoder with a Turbo decoder structure, used to decode the received signal transmitted to the receiver and obtain the semantic features of the input text; The channel decoder includes a third interleaver, a fourth convolutional layer, a first deinterleaver, a fourth interleaver, a fifth convolutional layer, and a second deinterleaver. The input of the third interleaver is connected to the output of the channel encoder; The input of the fourth convolutional layer is connected to the output of the channel encoder and the output of the third interleaver, respectively; the output of the fourth convolutional layer is connected to the input of the first deinterleaver. The input terminal of the fourth interleaver is connected to the output terminal of the channel encoder and the output terminal of the first deinterleaver, respectively. The input of the fifth convolutional layer is connected to the output of the fourth interleaver and the output of the channel encoder, respectively; the output of the fifth convolutional layer is connected to the second deinterleaver, and the second deinterleaver is connected to the input of the third interleaver and the semantic decoder, respectively. The semantic decoder is a decoder built using a Transformer structure. It is used to further decode the semantic features of the input text by combining a multi-head self-attention mechanism to obtain the decoded original input text.

2. The deep semantic autoencoder based on Turbo encoding and decoding according to claim 1, characterized in that, The semantic encoder includes multiple Transformer encoder layers; the multiple Transformer encoder layers are connected end to end in sequence, wherein the first Transformer encoder layer receives the original input text; and the last Transformer encoder layer outputs the semantic features of the original input text. The Transformer encoder layer includes a multi-head self-attention mechanism module and a position-feedforward neural network; the multi-head self-attention mechanism module is used to compute a representation for each position in the input sequence based on the original input text, and obtain the output of the multi-head attention. To obtain the input signal of the position feedforward neural network. ; The position feedforward neural network is used to calculate the input signal of the position feedforward neural network. Obtain the output of the feedforward neural network To obtain the output of the semantic encoder.

3. A deep semantic autoencoder based on Turbo encoding and decoding according to claim 2, characterized in that, Obtain the output of the feedforward neural network The method is as follows: S11: Obtain the output of the multi-head self-attention mechanism as follows: in, This represents a concatenation function that joins together multiple attention heads; For the additional weight matrix; Indicates the total number of heads of attention; Indicates the first One point of attention; in, Indicates the activation function; , , The first The query, key, and value of an attention head. express Dimension size; in, in, , , All are learnable parameter matrices; For input text Word representations are obtained after the embedding layer; S12: Output of multi-head attention Summation and normalization are performed to obtain the input of the position feedforward neural network. ; S13: Based on the input of the position-based feedforward neural network The output of the position feedforward neural network is obtained, and it is represented as: in, , All of them are learnable weight matrices. , All are learnable bias vectors; This represents the function that takes the maximum value. This represents the output of the position feedforward neural network.

4. A deep semantic autoencoder based on Turbo encoding and decoding according to claim 2, characterized in that, The method for obtaining the input signal of the channel decoder is as follows: S21: Obtain the sequence from the main encoder. as follows: in, It is the main encoder function; This represents the semantic features extracted from the input text; S22: Obtain the sequence generated after encoding by the first component encoder. as follows: in, It is a function of the first component encoder; S23: Obtain the output sequence of the second component encoder as follows: in, It is a function of component encoder 2; S24: Generate a sequence based on the encoding performed by the first component encoder. The output sequence of the second component encoder and the sequence of the main encoder Obtain the encoded bit sequence as follows: S25: Based on the encoded bit sequence The input signal of the channel decoder is obtained as follows: in, This indicates the signal emitted by the transmitter. This refers to the signal received by the receiver, i.e., the input signal of the channel decoder. Indicates a noise signal; This represents the channel fading coefficient.

5. A deep semantic autoencoder based on Turbo encoding and decoding according to claim 1, characterized in that, The semantic decoder includes multiple DecoderLayer layers; the multiple DecoderLayer layers are connected end to end in sequence, wherein the first DecoderLayer layer receives the output signal of the channel decoder; and the last DecoderLayer layer outputs the decoded original input text.

Images