Semantic information transmission method and device, equipment and storage medium

Abstract

The invention relates to the technical field of wireless semantic communication, and discloses a semantic information transmission method, device and equipment and a storage medium, a sending end extracts features of to-be-transmitted data, and matches the features with vectors in a preset discrete vector codebook to obtain discrete feature vectors; and determining an orthogonal frequency division multiplexing time-frequency resource distribution grid according to the feature importance weights of the discrete feature vector and the initial feature vector, generating a sending signal, and transmitting the sending signal to a receiving end. And the receiving end inputs the semantic feature vector obtained after processing the received signal into the vector quantization variational self-decoder, and the semantic feature vector is matched with the vector in the discrete vector codebook again to correct the transmission error, thereby obtaining the reconstructed data to be transmitted. Discretization processing of the features is achieved through a discrete vector codebook, important semantic feature bits are quantized according to the importance weight of each feature and then distributed to orthogonal frequency division multiplexing for transmission, so that key semantic information is protected, and anti-interference transmission of the semantic information is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wireless semantic communication, and particularly relates to a semantic information transmission method and device, equipment and a storage medium. BACKGROUND

[0002] Traditional semantic communication based on quantized variational autoencoder can be transmitted by matching with a codebook to obtain a feature index, but in a complex wireless environment, the transmitted feature index is prone to errors, and the received error index will cause the mapping to an error feature vector from the codebook, resulting in serious distortion of semantics, and the anti-interference transmission capability is weak. SUMMARY

[0003] The main purpose of the present application is to provide a semantic information transmission method, device, equipment and storage medium, which aims to solve the technical problems that the traditional quantized variational autoencoder semantic communication transmission feature index is prone to errors, resulting in serious distortion of semantics.

[0004] To achieve the above purpose, the present application provides a semantic information transmission method, which is applied to a sending end and includes the following steps.

[0005] The to-be-transmitted data is input into a vector quantization variational autoencoder, and the features of the to-be-transmitted data are extracted through a first neural network to obtain an initial feature vector;

[0006] The initial feature vector is matched with the vectors in a preset discrete vector codebook to obtain a discrete feature vector, and the matching is to replace each vector in the initial feature vector with a vector having the smallest Euclidean distance in the discrete vector codebook;

[0007] A first orthogonal frequency division multiplexing time-frequency resource distribution grid is determined according to the feature importance weight of the discrete feature vector and the initial feature vector, and a sending signal is generated based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid;

[0008] The sending signal is transmitted to a receiving end through a time-frequency fading channel.

[0009] Optionally, the step of determining a first orthogonal frequency division multiplexing time-frequency resource distribution grid according to the feature importance weight of the discrete feature vector and the initial feature vector, and generating a sending signal based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid includes the following steps.

[0010] The discrete feature vector is uniformly bit-quantized to obtain a discrete bit stream;

[0011] The gradient change degree of a loss function in the first neural network is determined according to the initial feature vector to generate a feature importance weight;

[0012] distribute the discrete bit stream into an initial orthogonal frequency division multiplexing grid based on the feature importance weight, to obtain a first filled orthogonal frequency division multiplexing time-frequency resource distribution grid;

[0013] processing the first orthogonal frequency division multiplexing time-frequency resource distribution grid through inverse Fourier transform, and adding a cyclic prefix to the processing result to generate a transmission signal.

[0014] Optionally, the step of distributing the discrete bit stream into an initial orthogonal frequency division multiplexing grid based on the feature importance weight, to obtain a first filled orthogonal frequency division multiplexing time-frequency resource distribution grid, comprises:

[0015] initializing a two-dimensional time-frequency resource grid of a coherent processing interval, and setting a preset reference signal in a rectangular manner through a discrete uniform distribution in the two-dimensional time-frequency resource grid, to obtain an initial orthogonal frequency division multiplexing grid;

[0016] determining a target resource element quantity of important semantics according to a resource element quantity of the reference signal and a quantity of the feature importance weight;

[0017] based on the feature importance weight and the target resource element quantity, dividing the discrete bit stream into important semantic features and non-important semantic features in the initial orthogonal frequency division multiplexing grid, modulating the important semantic features around the reference signal, and modulating and distributing the non-important semantic features to blank resource elements, to obtain a first orthogonal frequency division multiplexing time-frequency resource distribution grid.

[0018] Optionally, before the step of inputting the to-be-transmitted data into the vector quantization variational autoencoder, extracting features of the to-be-transmitted data through the first neural network, and obtaining an initial feature vector, the method further comprises:

[0019] training the vector quantization variational autoencoder based on a preset data set, to obtain a first neural network, a second neural network, and a discrete vector codebook, the first neural network being arranged in the vector quantization variational autoencoder, the second neural network being arranged in the vector quantization variational autoencoder, and the discrete vector codebook being shared by the sending end and the receiving end.

[0020] In addition, to achieve the above-mentioned purposes, the present application further provides a semantic information transmission method, which is applied to a receiving end and comprises:

[0021] receiving a reception signal transmitted by a sending end through a time-frequency fading channel;

[0022] performing channel equalization on the reception signal to obtain a target semantic feature;

[0023] bit dequantize the target semantic feature to obtain a semantic feature vector;

[0024] input the semantic feature vector into a vector quantization variational self-decoder, and perform re-matching with vectors in a discrete vector codebook through a nearest neighbor algorithm to correct transmission errors and obtain a corrected feature vector;

[0025] input the corrected feature vector into a preset second neural network to obtain reconstructed data to be transmitted.

[0026] Optionally, the step of channel equalizing the received signal to obtain a target semantic feature comprises:

[0027] remove the cyclic prefix from the received signal, and perform Fourier transform on the removal result to obtain a second orthogonal frequency division multiplexing time-frequency resource distribution grid;

[0028] estimate a first channel of a reference signal in the second orthogonal frequency division multiplexing time-frequency resource distribution grid, and estimate a second channel of semantic data in the second orthogonal frequency division multiplexing time-frequency resource distribution grid according to the first channel;

[0029] determine a semantic signal at the semantic data according to the first channel and the second channel;

[0030] demodulate the semantic signal and restore the order of the semantic feature to generate a target semantic feature.

[0031] In addition, to achieve the above-mentioned purpose, the present application also provides a semantic information transmission device, which is applied to a sending end and comprises:

[0032] a feature extraction module configured to input data to be transmitted into a vector quantization variational self-encoder, extract features of the data to be transmitted through a first neural network, and obtain an initial feature vector;

[0033] a vector matching module configured to match the initial feature vector with vectors in a preset discrete vector codebook to obtain a discrete feature vector, wherein the matching is to replace each vector in the initial feature vector with a vector having the smallest Euclidean distance in the discrete vector codebook;

[0034] a signal generation module configured to determine a padded first orthogonal frequency division multiplexing time-frequency resource distribution grid according to the discrete feature vector and a feature importance weight of the initial feature vector, and generate a sending signal based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid;

[0035] a signal generation module configured to transmit the sending signal to a receiving end through a time-frequency fading channel.

[0036] In addition, to achieve the above object, the present application also provides a semantic information transmission device, which is applied to a receiving end and comprises:

[0037] a signal receiving module, configured to receive a receiving signal transmitted by a sending end through a time-frequency fading channel;

[0038] a channel equalization module, configured to perform channel equalization on the receiving signal to obtain a target semantic feature;

[0039] a bit inverse quantization module, configured to perform bit inverse quantization on the target semantic feature to obtain a semantic feature vector;

[0040] a vector correction module, configured to input the semantic feature vector into a vector quantization variational autoencoder, and perform re-matching with vectors in a discrete vector codebook through a nearest neighbor algorithm to correct transmission errors and obtain a corrected feature vector;

[0041] a data reconstruction module, configured to input the corrected feature vector into a preset second neural network to obtain reconstructed to-be-transmitted data.

[0042] In addition, to achieve the above object, the present application also provides a semantic information transmission device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the semantic information transmission method.

[0043] In addition, to achieve the above object, the present application also provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program is executed by a processor to implement the steps of the semantic information transmission method.

[0044] In the present application, the sending end inputs the to-be-transmitted data into a vector quantization variational autoencoder, extracts the features of the to-be-transmitted data through a first neural network to obtain an initial feature vector; matches the initial feature vector with vectors in a preset discrete vector codebook to obtain a discrete feature vector; determines a first orthogonal frequency division multiplexing time-frequency resource distribution grid according to the feature importance weight of the discrete feature vector and the initial feature vector, and generates a sending signal based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid; and transmits the sending signal to the receiving end through a time-frequency fading channel. The receiving end receives the receiving signal transmitted by the sending end through the time-frequency fading channel; performs channel equalization on the receiving signal to obtain a target semantic feature; performs bit dequantization on the target semantic feature to obtain a semantic feature vector; inputs the semantic feature vector into a vector quantization variational auto-decoder, and performs re-matching with the vectors in the discrete vector codebook through a nearest neighbor algorithm to correct transmission errors and obtain a corrected feature vector; and inputs the corrected feature vector into a preset second neural network to obtain reconstructed to-be-transmitted data. The semantic features of the to-be-transmitted data are extracted through the vector quantization variational autoencoder, the discrete vector codebook is combined to realize the discretization processing of the features, the resource allocation of the orthogonal frequency division multiplexing time-frequency resource distribution grid is optimized based on the feature importance weight, the important semantic feature bits are quantized and then allocated to the vicinity of the orthogonal frequency division multiplexing reference signal for transmission, and the received feature vector is re-matched with the codebook to correct the errors in transmission, thereby further reducing the distortion of semantic transmission, retaining the key semantic features, and improving the anti-interference capability of transmission. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced here. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0047] Figure 1 Flowchart of the first embodiment of the semantic information transmission method of the present application;

[0048] Figure 2 System model diagram of the semantic information transmission method of the present application;

[0049] Figure 3 Flowchart of the second embodiment of the semantic information transmission method of the present application;

[0050] Figure 4 OFDM transmission flowchart of the semantic information transmission method of the present application;

[0051] Figure 5 The flowchart of the third embodiment of the semantic information transmission method of the present application is shown in the figure.

[0052] Figure 6 The module structure diagram of the semantic information transmission device of the sending end of the embodiment of the present application is shown in the figure.

[0053] Figure 7 The module structure diagram of the semantic information transmission device of the receiving end of the embodiment of the present application is shown in the figure.

[0054] Figure 8 The device structure diagram of the hardware running environment involved in the semantic information transmission method in the embodiment of the present application is shown in the figure.

[0055] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0056] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0057] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings of the specification and the specific embodiments.

[0058] In traditional semantic communication, the semantic features extracted by deep neural network are continuous vectors, usually using analog transmission, which is not compatible with existing digital communication system. While the traditional VQ-VAE (Variational Quantized Variational Autoencoder) based semantic communication can get feature index by matching with codebook for transmission, which greatly reduces the amount of data transmission on the one hand, and is compatible with digital communication system on the other hand. However, in complex wireless environment, the transmitted feature index is prone to error, and the received error index will lead to the mapping of error feature vector from the codebook, resulting in serious distortion of the semantic, and the ability of anti-interference transmission is weak.

[0059] Therefore, the application provides a semantic information anti-interference transmission technology for wireless digital communication, establishes a digital semantic communication system based on a vector quantization variational autoencoder and semantic importance, matches the extracted feature vector with the codebook at the sending end to obtain a discrete semantic feature vector, allocates important feature signals to the vicinity of an OFDM (Orthogonal Frequency Division Multiplexing) reference signal according to the importance weight of each feature to protect key semantic information, and matches the received feature vector with the codebook again to correct errors in transmission, reduce semantic transmission distortion, and realize anti-interference transmission of semantic information.

[0060] Based on this, the embodiment of the application provides a semantic information transmission method, referring to Figure 1 , Figure 1 FIG. 1 is a flowchart of a first embodiment of the semantic information transmission method of the application.

[0061] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a computer, or an electronic device capable of realizing the above functions. The following takes the sending end as an example to describe the embodiment and the following embodiments.

[0062] In the embodiment, the semantic information transmission method comprises:

[0063] In step S10, the to-be-transmitted data is input into a vector quantization variational autoencoder, and a feature of the to-be-transmitted data is extracted through a first neural network to obtain an initial feature vector.

[0064] It should be noted that the to-be-transmitted data refers to original information that needs to be sent through a communication channel, and can include various types of data information carrying semantics such as text, image and audio. The vector quantization variational autoencoder (VQ-VAE) is a deep learning model that combines vector quantization mechanism and variational autoencoder, and has feature learning and discrete expression capabilities. The first neural network is an encoder network in the VQ-VAE, which is used to extract feature information of the to-be-transmitted data. Different networks can be set according to the type of the to-be-transmitted data, for example, when the to-be-transmitted data is a picture, the first neural network can be a convolutional neural network. The initial feature vector is a high-dimensional vector output by the first neural network after encoding the original data, which contains the core semantic information of the data.

[0065] Further, in order to ensure the consistency of feature extraction and reconstruction at both the sending and receiving ends, a reliable model basis is provided for the entire semantic transmission system. Before step S10, the method further comprises:

[0066] The vector quantization variational autoencoder is trained based on a preset data set to obtain a first neural network, a second neural network and a discrete vector codebook, the first neural network is arranged in the vector quantization variational autoencoder, the second neural network is arranged in the vector quantization variational auto-decoder, and the discrete vector codebook is shared by the sending end and the receiving end.

[0067] It should be noted that the preset data set refers to a known sample set used for model training, which contains to-be-transmitted data matched with an actual application scenario, such as specific types of images, texts and the like. The vector quantization variational autoencoder includes an encoder and a decoder. The first neural network and the second neural network are core components of the encoder and the decoder, respectively, and are responsible for encoding input data into a feature vector or reconstructing a quantized feature vector into original data. The discrete vector codebook is a set of pre-defined discrete vectors, which is used to quantize continuous feature vectors and is a benchmark for feature matching between the sending end and the receiving end, and serves as a semantic knowledge base shared by the sending end and the receiving end.

[0068] It can be understood that when the VQ-VAE encoder-decoder is trained end-to-end through the preset data set, the first neural network encodes the input data into an initial feature vector, and then matches the initial feature vector with the discrete vector codebook to obtain a quantized feature vector; the quantized feature vector is input into the vector quantization variational auto-decoder, and the second neural network reconstructs data based on the vector. By minimizing the objective function of the difference between the original data and the reconstructed data and the difference between the initial feature vector and the codebook vector, the parameters of the first neural network, the second neural network and the vector values of the discrete vector codebook are iteratively optimized until the model converges, and finally the trained network parameters are obtained, and the discrete vector codebook is stored and used by the sending end and the receiving end. The sharing of the codebook by the sending end and the receiving end ensures the consistency of the feature quantization and matching standard, and avoids transmission errors caused by codebook differences.

[0069] In an example, the transmission and reception process of an image data is described in detail, the VQ-VAE encoder-decoder is trained using an image data set to obtain the neural network parameters θ1 of the CNN encoder and the neural network parameters θ2 of the CNN decoder , and a discrete vector codebook E = [e1, … e j , … e J ] shared by the sending end and the receiving end, e j ∈R D , where j is the number of vectors in the discrete vector codebook.

[0070] The original image z ∈ R l is input into the VQ-VAE encoder, and a feature vector z e ∈R D×H×W can be extracted by the CNN encoder, which can be represented as where D is the feature dimension, H and W are the height and width of the feature map in the first neural network.

[0071] In step S20, the initial feature vector is matched with vectors in a preset discrete vector codebook to obtain a discrete feature vector, and the matching is replacing each vector in the initial feature vector with a vector having the smallest Euclidean distance in the discrete vector codebook.

[0072] It should be understood that the discrete feature vector is a discretization result obtained after the initial feature vector is matched, and the Euclidean distance is an index for measuring the difference in spatial position of two vectors, and the calculation formula is the square root of the sum of squares of the differences in each dimension of the vectors, which is used to quantify the similarity between vectors.

[0073] Specifically, for each sub-vector in the initial feature vector, the Euclidean distances of the sub-vector from all vectors in the discrete vector codebook are calculated one by one, the codebook vector having the smallest distance is screened out, and the corresponding sub-vector in the initial feature vector is replaced with the codebook vector. Finally, all the replaced sub-vectors are combined to form a discrete feature vector, realizing the conversion of continuous features to discrete features. The discretization realized by the Euclidean distance matching can preserve the core semantics of the initial features while correcting transmission errors by matching with the codebook again at the receiving end.

[0074] In an example, the feature vector z e The discrete feature vector K = [k1,…k i ,…,k D ] is obtained by matching with the vectors in the shared codebook E through the nearest neighbor algorithm, K ∈ R D×H×W , i.e., each vector in the semantic feature z e is replaced with a vector having the smallest Euclidean distance in the codebook E, which can be represented as

[0075]

[0076] where ||·||2 represents the 2-norm, and the feature vector K is composed of discrete vectors in the codebook. Unlike the traditional semantic communication based on VQ-VAE, the feature vector itself is directly transmitted here, rather than the index corresponding to the feature vector in the codebook. By selecting the codebook vector having the smallest distance for replacement, the semantic loss in the quantization process can be minimized, ensuring the semantic consistency of the discrete feature and the original feature.

[0077] In step S30, a first orthogonal frequency division multiplexing time-frequency resource distribution grid is determined according to the feature importance weight of the discrete feature vector and the initial feature vector, and a transmission signal is generated based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid.

[0078] It should be noted that by analyzing the gradient change degree of the loss function in the VQ-VAE neural network, the influence of each feature on image reconstruction can be obtained. The first orthogonal frequency division multiplexing (OFDM) time-frequency resource distribution grid is a two-dimensional time-frequency resource structure suitable for transmitting semantic features, which is composed of symbols in the time dimension and subcarriers in the frequency dimension, and is used to carry discrete feature vectors. The transmission signal is a time-domain electrical signal that can be transmitted through a channel and is generated based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid.

[0079] Specifically, first, the discrete feature vectors are prioritized based on the feature importance weights, and the important features with high weights are preferentially allocated to resource elements with better channel conditions in the OFDM time-frequency resource distribution grid, and the features with low weights are allocated to the remaining resources to complete grid filling; then, inverse Fourier transform is performed on the filled OFDM time-frequency resource distribution grid to map the frequency domain resources to the time domain, and a cyclic prefix is added to resist inter-symbol interference, and finally a transmission signal that can be transmitted through a time-frequency fading channel is generated.

[0080] Step S40, transmitting the transmission signal to the receiving end through the time-frequency fading channel.

[0081] It can be understood that the time-frequency fading channel refers to a channel in wireless communication that causes amplitude and phase fluctuations of signals in the time and frequency dimensions due to multipath propagation, Doppler effect, etc., which is manifested as signal strength attenuation or distortion with time and frequency changes.

[0082] In an example, referring to Figure 2 , Figure 2 is a system model diagram of the semantic information transmission method of the present application. The VQ-VAE encoder and decoder of the receiving end and the transmitting end both use a convolutional neural network (CNN) to extract semantic features and a nearest neighbor algorithm to realize the discretization of the features, wherein the CNN is composed of a series of convolutional layers, ReLU activation functions and residual blocks. The original image is input into the VQ-VAE encoder, and the continuous feature vector z e is obtained by matching the nearest neighbor algorithm with the semantic knowledge base (shared codebook) shared by the transmitting and receiving ends; the discrete feature is converted into a semantic feature B through uniform bit flow after being quantized, and is combined with the feature importance weight ω to map to the OFDM transmitting end time-frequency grid to generate a signal x, which is transmitted to the receiving end through a wireless channel; the receiving end obtains the semantic feature B' through OFDM demodulation, and restores it to a discrete feature K' through bit stream dequantization, and inputs the feature vector into the VQ-VAE decoder, and matches the codebook again through the nearest neighbor algorithm to correct the feature vector transmission error, wherein z q represents the feature vector after correction, and then the feature vector z qThe reconstructed image z' is input into the CNN decoder to realize a closed loop of semantics from encoding, quantization transmission to decoding reconstruction.

[0083] In the embodiment, the data to be transmitted is input into a vector quantization variational autoencoder, a first neural network is used to extract features of the data to be transmitted to obtain an initial feature vector; the initial feature vector is matched with vectors in a preset discrete vector codebook to obtain a discrete feature vector, the matching is to replace each vector in the initial feature vector with a vector having the smallest Euclidean distance in the discrete vector codebook; a first orthogonal frequency division multiplexing time-frequency resource distribution grid is determined according to a feature importance weight of the discrete feature vector and the initial feature vector, and a sending signal is generated based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid; the sending signal is transmitted to a receiving end through a time-frequency fading channel, so that the receiving end is matched with the codebook again to correct transmission errors. The semantic features of the data to be transmitted are extracted by the vector quantization variational autoencoder, the discrete vector codebook is used to realize the discretization of the features, and the resource allocation of the orthogonal frequency division multiplexing time-frequency resource distribution grid is optimized based on the feature importance weight, so that the important semantic feature bits are quantized and allocated to the vicinity of the orthogonal frequency division multiplexing reference signal for transmission, thereby retaining the key semantic features and improving the anti-interference capability of transmission.

[0084] Reference Figure 3 , Figure 3 FIG. 2 is a flowchart of a second embodiment of the semantic information transmission method of the present application, which is based on the first embodiment.

[0085] In the second embodiment, the step S30 comprises:

[0086] In step S301, the discrete feature vectors are uniformly bit-quantized to obtain a discrete bit stream.

[0087] It should be noted that the uniform bit quantization is a process of converting each discrete feature vector into a fixed-length binary number, and the discrete bit stream is a continuous binary sequence obtained after quantization, which is composed of 0 and 1 and is a signal form that can be directly processed by a digital communication system.

[0088] It should be understood that the uniform bit quantization of the discrete feature vectors to obtain the discrete bit stream can be to determine the number of quantization bits corresponding to each discrete feature vector, to allocate a unique binary code to each vector based on the uniqueness of the vectors in the discrete vector codebook, to replace each element in the discrete feature vector with the corresponding binary code, and finally to concatenate all the binary codes in order to form a continuous discrete bit stream.

[0089] In an example, the feature vector ki The discrete bit stream can be obtained by using the uniform bit quantization method

[0090] b i =Q c (k i )

[0091] wherein Q c (·) represents a bit quantizer, and c represents the number of quantized bits. The quantized semantic feature can be represented as B=[b1,…b i ,…,b D ], and b i ∈{0,1} L .

[0092] In step S302, the gradient change degree of the loss function in the first neural network is determined according to the initial feature vector, and a feature importance weight is generated.

[0093] It should be noted that the gradient change degree is the absolute value of the partial derivative of the loss function with respect to each element of the initial feature vector, which reflects the influence degree of the slight change of the feature on the model performance; the feature importance weight is a numerical value generated based on the gradient change degree, which is used to quantify the contribution degree of each feature to the semantic expression.

[0094] It should be understood that by analyzing the gradient change degree of the loss function in the VQ-VAE neural network, the influence of each feature on the image reconstruction can be obtained. The loss function is

[0095]

[0096] Specifically, the gradient of the i-th feature is first calculated to obtain a gradient matrix

[0097]

[0098] Then, the importance weight ω i of the i-th feature can be obtained by calculating the feature matrix by using the global average pooling method.

[0099]

[0100] Therefore, the feature importance weight vector can be represented as ω=[ω1,…ω i ,…,ω D ].

[0101] In step S303, the discrete bit stream is allocated to the initial orthogonal frequency division multiplexing grid based on the feature importance weight, and a filled first orthogonal frequency division multiplexing time-frequency resource distribution grid is obtained.

[0102] It should be noted that the initial orthogonal frequency division multiplexing grid is a two-dimensional time-frequency resource framework containing preset reference signals, which is composed of resource elements of symbols in the time dimension and subcarriers in the frequency dimension.

[0103] It can be understood that, based on the feature importance weight, the discrete bit stream is allocated to the initial orthogonal frequency division multiplexing grid, which can be to sort the semantic features corresponding to the discrete bit stream according to the feature importance weight from high to low, and mark the resource elements around the reference signals in the initial OFDM grid as high-quality resources (more accurate channel estimation, less interference), and the rest as ordinary resources; then the discrete bit stream corresponding to the high-weight features at the top of the order is modulated (such as using a higher order modulation method) and mapped to the high-quality resources, and the discrete bit stream corresponding to the low-weight features at the bottom of the order is modulated and allocated to the ordinary resources, and if the high-quality resources are not filled, the next high-weight features are continued to be filled, until all the discrete bit streams are allocated, forming the OFDM grid.

[0104] It should be understood that, based on the feature importance weight, the discrete bit stream is allocated to the initial orthogonal frequency division multiplexing grid, which can be to divide the discrete bit stream into several groups according to the feature importance weight, and the sum of the weights of the important groups exceeds the preset threshold, and the rest is the ordinary group; in the initial OFDM grid, resource blocks of different sizes are divided, and large resource blocks (or continuous resource blocks to reduce interference) are allocated to the important group, and small resource blocks are allocated to the ordinary group; at the same time, the important group adopts a modulation and coding scheme with stronger anti-interference ability, and the ordinary group adopts a scheme with higher efficiency, and finally the discrete bit streams of each group are filled into the corresponding resource blocks to form the filled OFDM time-frequency resource distribution grid.

[0105] Further, in order to realize the differentiated and efficient use of time-frequency resources, and significantly improve the anti-noise and anti-fading ability of semantic information in a complex channel environment. The step S303 can include:

[0106] Initialize a two-dimensional time-frequency resource grid of a coherent processing interval, and set a preset reference signal in a rectangular manner through discrete uniform distribution in the two-dimensional time-frequency resource grid to obtain an initial orthogonal frequency division multiplexing grid; determine the target resource element quantity of important semantics according to the resource element quantity of the reference signal and the quantity of the feature importance weight; based on the feature importance weight and the target resource element quantity, in the initial orthogonal frequency division multiplexing grid, the discrete bit stream is divided into important semantic features and non-important semantics, and the important semantics are modulated around the reference signals, and the non-important semantics are modulated and allocated to the blank resource elements, to obtain a first orthogonal frequency division multiplexing time-frequency resource distribution grid.

[0107] It should be noted that the coherent processing interval (CPI) is a time-frequency interval in wireless communication assuming that the channel characteristics (such as fading, Doppler effect) are approximately stationary, and the two-dimensional time-frequency resource grid defined by the interval divides the resource elements (RE) with time (symbol) and frequency (subcarrier) as axes; the reference signal is a known sequence embedded in the grid with rectangular discrete uniform distribution, used for channel estimation and synchronization at the receiving end; the number of target resource elements of important semantics refers to the RE quota that can be allocated to key semantics according to the association rule of the number of REs of the reference signal and the characteristic importance weight, such as proportional mapping.

[0108] Specifically, first, according to the length of the coherent processing interval and the number of subcarriers, a two-dimensional time-frequency resource grid is initialized, the reference signal is inserted in a rectangular area with discrete uniform spacing (such as fixed symbol / subcarrier spacing), and an initial OFDM grid is constructed; second, the target number of REs of high importance semantics is calculated through the total number of REs of the reference signal and the dimension of the characteristic importance weight; finally, the semantic features of the target number of REs are sorted according to the characteristic importance, and the remaining semantic features are modulated and filled into the blank REs after being modulated and mapped to the REs around the reference signal, and the OFDM grid construction is completed.

[0109] In an example, the reference Figure 4 , Figure 4 is the OFDM transmission flowchart of the semantic information transmission method of the present application. According to the importance weight parameters of different characteristics, important characteristics are allocated near the reference signal, the reference signal is modulated based on the Gold sequence for channel estimation and data demodulation, and is uniformly distributed in the time slots of the frame structure. The transmitter combines the feature importance weight ω and the discrete bit stream B, and divides the semantic features into important and unimportant categories, and allocates them to the OFDM time-frequency grid in combination with the reference signal, and generates a transmission signal X after inverse discrete Fourier transform (IDFT) and cyclic prefix (CP) addition, and sends it into the wireless channel.

[0110] The steps of constructing the OFDM time-frequency grid include: first, initializing a two-dimensional time-frequency resource grid of a coherent processing interval (CPI) and discretely and uniformly distributing the reference signal in a rectangular manner, a CPI is composed of N t OFDM symbols and N c subcarriers, wherein the smallest unit in the OFDM time-frequency resource grid is represented as a resource element (RE): the blue RE represents the reference signal, the green RE represents the important semantic symbol, and the blank RE represents the unimportant semantic symbol. Then the total number of REs of the time-frequency resource in a CPI is N CPI = N t N c .

[0111] Then, 4 green REs are allocated around each reference signal, and the remaining REs are allocated with non-important semantic features. The number of REs of a reference signal within one CPI is N ref The number of REs allocated to important semantics is 4N ref The number of REs of non-important semantics is N CPI -5N ref That is, the number of important semantic features is

[0112]

[0113] Finally, the first D imp semantic features are modulated and allocated to green REs according to the feature importance weight ω, and the remaining semantic features are modulated and allocated to blank REs.

[0114] After the receiving end receives the signal Y in the wireless channel, it performs CP removal and DFT (Fourier transform) processing in sequence, completes channel estimation through the reference signal, demodulates semantic data, and outputs the reconstructed discrete features K' after de-mapping and inverse quantization.

[0115] In step S304, the first orthogonal frequency division multiplexing time-frequency resource distribution grid is processed by inverse Fourier transform, and the processing result is added with a cyclic prefix to generate a transmission signal.

[0116] It can be understood that after the filled OFDM time-frequency resource is processed by inverse Fourier transform (IDFT) and added with a cyclic prefix (CP), a transmission signal x∈R Nc×Nt is generated, which is transmitted through a time-frequency fading channel, and the received signal y∈R Nc×Nt can be represented as

[0117] y=h*x+n

[0118] Where * represents convolution operation, h represents the impulse response of a multipath channel, and n is a circular symmetric complex Gaussian noise with zero mean and variance , where I is the unit matrix.

[0119] In the embodiment, the discrete feature vector is uniformly bit-quantized to obtain a discrete bit stream; a gradient variation degree of a loss function in the first neural network is determined according to the initial feature vector to generate a feature importance weight; the discrete bit stream is distributed into an initial orthogonal frequency division multiplexing grid based on the feature importance weight to obtain a filled first orthogonal frequency division multiplexing time-frequency resource distribution grid; and the first orthogonal frequency division multiplexing time-frequency resource distribution grid is processed through inverse Fourier transform, and a processing result is added with a cyclic prefix to generate a sending signal. The feature importance weight generated based on the gradient of the loss function can accurately reflect the contribution of the feature to semantic expression, and in combination with the inverse Fourier transform of the orthogonal frequency division multiplexing time-frequency resource distribution grid and the cyclic prefix addition, the inter-symbol interference is effectively reduced, and the transmission performance of the sending signal is further optimized.

[0120] Reference Figure 5 , Figure 5 FIG. 3 is a flowchart of a semantic information transmission method according to a third embodiment of the present application. Based on the above embodiments, the third embodiment of the semantic information transmission method of the present application is proposed.

[0121] In the third embodiment, the semantic information transmission method is applied to a receiving end, and includes:

[0122] In step S501, a receiving signal transmitted by a sending end through a time-frequency fading channel is received.

[0123] It should be noted that the receiving signal is an electrical signal obtained by the receiving end from the time-frequency fading channel, and carries the semantic encoding information transmitted by the sending end. Due to the multipath propagation and additive noise of the channel, the signal presents time-frequency domain fading distortion in amplitude and phase.

[0124] In step S502, the receiving signal is channel equalized to obtain a target semantic feature.

[0125] It should be understood that channel equalization is a signal processing process for compensating for channel distortion, and the purpose is to eliminate or weaken the distortion influence of the channel on the signal; the target semantic feature is a feature representation recovered after channel equalization and carrying core semantic information.

[0126] Further, in order to realize accurate demodulation and sequential recovery of the semantic signal, a high-quality target feature is provided for subsequent inverse quantization and reconstruction of the semantic feature. The step S502 can include:

[0127] The received signal is removed from the cyclic prefix, and the removal result is Fourier transformed to obtain a second orthogonal frequency division multiplexing time-frequency resource distribution grid; a first channel of a reference signal in the second orthogonal frequency division multiplexing time-frequency resource distribution grid is estimated, and a second channel of semantic data in the second orthogonal frequency division multiplexing time-frequency resource distribution grid is estimated according to the first channel; a semantic signal at the semantic data is determined according to the first channel and the second channel; the semantic signal is demodulated, and the order of semantic features is recovered to generate a target semantic feature.

[0128] It can be understood that the second orthogonal frequency division multiplexing time-frequency resource distribution grid is a frequency domain resource structure obtained by Fourier transformation at the receiving end, corresponding to the OFDM grid of the sending end, and containing frequency domain symbols of the reference signal and the semantic data. The semantic data is a frequency domain symbol carrying semantic information, and demodulation is the conversion of the frequency domain symbol into a discrete bit stream. The target semantic feature is a discrete representation corresponding to the semantic feature of the sending end after demodulation and recovery of the order.

[0129] Specifically, the receiving end performs synchronization processing on the received signal, locates the start and end positions of the cyclic prefix, removes the cyclic prefix from the time domain signal, and retains the valid OFDM symbol; the discrete Fourier transform is performed on the time domain signal after removing the cyclic prefix to convert it into a frequency domain signal to obtain a second OFDM grid corresponding to the sending end, which contains frequency domain symbols of the reference signal and the semantic data. Secondly, using the known reference signal in the second OFDM grid, the channel response of the resource element where the reference signal is located is estimated by using the least square or interpolation algorithm, and the channel response of the resource element where the semantic data is located is calculated by using the bilinear interpolation method, to complete the channel estimation; based on the estimated channel response, the frequency domain symbol of the semantic data is equalized to obtain the compensated semantic signal. Finally, the semantic signal is demodulated according to the modulation mode used by the sending end to convert the frequency domain symbol into a discrete bit stream; according to the feature allocation rule of the sending end (such as the position mapping according to the importance order), the order of the semantic features corresponding to the discrete bit stream is recovered, and finally the target semantic feature is generated.

[0130] In an example, at the receiving end, the received signal y is subjected to channel equalization after removal of the CP and Fourier transform (DFT). First, the least square method is used to estimate the channel h LS of the reference signal, and then the bilinear interpolation method is used to estimate the channel of the semantic data. Finally, the signal at the semantic data is determined according to the estimated channel . Next, the equalized semantic signal is demodulated, and the order of the semantic features is recovered to obtain B'. The order of recovering the semantic features is the same as the operation order of the encoder segment.

[0131] Step S503, bit dequantization is performed on the target semantic feature to obtain a semantic feature vector.

[0132] It can be understood that bit dequantization is the inverse process of quantization, which converts the discrete bit sequence into a continuous or discrete feature vector by restoring the quantization rule of the sending end. Bit dequantization is performed on the target semantic feature B' that restores the order of the semantic feature to obtain a semantic feature vector K', which is represented as

[0133]

[0134] wherein, denotes a bit dequantizer.

[0135] Step S504, the semantic feature vector is input into a vector quantization variational autoencoder, and a nearest neighbor algorithm is used to match the vectors in the discrete vector codebook again to correct the transmission error to obtain a corrected feature vector.

[0136] It should be understood that the vector quantization variational autoencoder is a model component for semantic reconstruction, corresponding to the VQ-VAE encoder of the sending end. The transmission error refers to the feature distortion of the signal caused by interference during transmission, and the corrected feature vector is a vector that is closer to the original feature of the sending end after being matched and corrected.

[0137] Specifically, after the semantic feature vector is input into the vector quantization variational autoencoder, the Euclidean distance between each sub-vector in the vector and all vectors in the discrete vector codebook is calculated one by one. The nearest neighbor algorithm is used to select the codebook vector with the smallest distance, and the codebook vector is used to replace the distorted sub-vector in the semantic feature vector. All replaced sub-vectors are recombined to obtain a corrected feature vector, and the compensation of the transmission error is completed. The feature vector is input into the VQ-VAE decoder, and the shared codebook is matched again by the nearest neighbor algorithm to correct the transmission error of the feature vector, and the data is reconstructed through the neural network opposite to the encoder.

[0138] In an example, the feature vector K' = [k'1, … k'N], K' ∈ RNis input into the VQ-VAE decoder, and the distorted vector in K' is matched again with the vector in the shared codebook E through the nearest neighbor algorithm to correct the transmission error, which can be represented as i D D×H×W

[0139]

[0140] wherein, q D×H×W denotes the corrected feature vector.

[0141] ​​​​Step S505, inputting the corrected feature vector into a preset second neural network to obtain reconstructed to-be-transmitted data.

[0142] It should be noted that the preset second neural network is a core network component in the vector quantization variational autoencoder, and constitutes an inverse process with the first neural network (encoder) of the sending end, and the parameters thereof are determined through training of a preset data set.

[0143] It should be understood that reconstructing the corrected feature vector according to the preset second neural network can be inputting the corrected feature vector into the preset second neural network, and the network decodes the feature vector layer by layer through an operation opposite to the first neural network of the sending end; through the network parameters optimized through training, the high-dimensional feature vector is gradually mapped back to the dimension space of the original data, and finally an output of the reconstructed result consistent with the data structure of the input data of the sending end is obtained, that is, the to-be-transmitted data is generated.

[0144] In this embodiment, the receiving signal transmitted by the sending end through the time-frequency fading channel is disclosed; the channel equalization is performed on the receiving signal to obtain a target semantic feature; the bit dequantization is performed on the target semantic feature to obtain a semantic feature vector; the semantic feature vector is input into the vector quantization variational autoencoder, and the nearest neighbor algorithm is used to match the vector in the discrete vector codebook again to correct the transmission error and obtain a corrected feature vector; and the corrected feature vector is reconstructed according to the preset second neural network to generate to-be-transmitted data. Through the channel equalization to compensate the signal distortion caused by the time-frequency fading, the combination of the bit dequantization to recover the semantic feature vector, and the use of the nearest neighbor algorithm to match the vector in the discrete vector codebook again to correct the transmission error, and finally the reconstruction of the data through the second neural network, the channel noise and the transmission loss can be effectively counteracted, the recovery precision of the semantic information is significantly improved, and the core semantic of the sending end can be accurately obtained by the receiving end.

[0145] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the semantic information transmission method of the present application, and more forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0146] The present application also provides a semantic information transmission device, please refer to Figure 6 , the semantic information transmission device is applied to a sending end and includes:

[0147] The feature extraction module 10 is configured to input to-be-transmitted data into a vector quantization variational autoencoder, extract features of the to-be-transmitted data through a first neural network, and obtain an initial feature vector.

[0148] The vector matching module 20 is configured to match the initial feature vector with vectors in a preset discrete vector codebook to obtain a discrete feature vector, and the matching is replacing each vector in the initial feature vector with a vector having the minimum Euclidean distance in the discrete vector codebook.

[0149] The signal generation module 30 is configured to determine a filled first orthogonal frequency division multiplexing time-frequency resource distribution grid according to the discrete feature vector and a feature importance weight of the initial feature vector, and generate a transmission signal based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid.

[0150] The signal generation module 40 is configured to transmit the transmission signal to a receiving end through a time-frequency fading channel.

[0151] The semantic information transmission device provided in the application adopts the semantic information transmission method in the above embodiments, and can solve the technical problem that the feature index of the traditional quantized variational autoencoder semantic communication transmission is prone to errors, resulting in serious distortion of semantics. Compared with the prior art, the semantic information transmission device provided in the application has the same beneficial effects as the semantic information transmission method provided in the above embodiments, and other technical features in the semantic information transmission device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0152] The application also provides a semantic information transmission device, please refer to Figure 7 , the semantic information transmission device is applied to a receiving end, and comprises:

[0153] The signal receiving module 50 is configured to receive a receiving signal transmitted by a transmission end through a time-frequency fading channel.

[0154] The channel equalization module 60 is configured to perform channel equalization on the receiving signal to obtain a target semantic feature.

[0155] The bit inverse quantization module 70 is configured to perform bit inverse quantization on the target semantic feature to obtain a semantic feature vector.

[0156] The vector correction module 80 is configured to input the semantic feature vector into a vector quantized variational autoencoder, and perform re-matching with vectors in a discrete vector codebook through a nearest neighbor algorithm to correct transmission errors and obtain a corrected feature vector.

[0157] The data reconstruction module 90 is configured to input the corrected feature vector into a preset second neural network to obtain reconstructed data to be transmitted.

[0158] The semantic information transmission device provided in the present application adopts the semantic information transmission method in the above embodiment, and can solve the technical problem that the traditional quantization variational autoencoder semantic communication transmission feature index is prone to error, resulting in serious distortion of semantics. Compared with the prior art, the semantic information transmission device provided in the present application has the same beneficial effects as the semantic information transmission method provided in the above embodiment, and other technical features in the semantic information transmission device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0159] The present application provides a semantic information transmission device, which comprises at least one processor and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the semantic information transmission method in the above embodiment one.

[0160] Reference will be made to the following description of the embodiments of the present application, taken in conjunction with the accompanying drawings, in which Figure 8 which shows a structural schematic diagram of a semantic information transmission device suitable for implementing the embodiments of the present application. The semantic information transmission device in the embodiments of the present application can include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 8 The semantic information transmission device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0161] As Figure 8As shown, the semantic information transmission device can include a processing apparatus 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory 1002 or loaded from a storage apparatus 1003 into a random access memory 1004. Various programs and data required for the operation of the semantic information transmission device are also stored in the random access memory 1004. The processing apparatus 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other by a bus 1005. An input / output interface 1006 is also connected to the bus. Generally, the following systems can be connected to the input / output interface 1006: input apparatuses 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output apparatuses 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage apparatus 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication apparatus 1009. The communication apparatus 1009 can allow the semantic information transmission device to perform wireless or wired communication with other devices to exchange data. Although the semantic information transmission device having various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0162] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication apparatus, or installed from the storage apparatus 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing apparatus 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0163] The semantic information transmission device provided by the present application adopts the semantic information transmission method in the above-mentioned embodiments, which can solve the technical problem that the feature index of the traditional quantized variational autoencoder semantic communication transmission is prone to error, resulting in serious distortion of semantics. Compared with the prior art, the semantic information transmission device provided by the present application has the same beneficial effects as the semantic information transmission method provided by the above-mentioned embodiments, and the other technical features in the semantic information transmission device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0164] It should be understood that portions of the application disclosed can be implemented in hardware, software, firmware, or combinations thereof. In the description of the embodiments above, specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0165] The above description is merely illustrative of the application and is not intended to limit the scope of the application. Any changes and modifications that can be made to the application in accordance with the principles of the application would be readily apparent to those skilled in the art and the present application is therefore not limited to the description and examples contained herein but is only limited by the claims.

[0166] The application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e., a computer program) for performing the semantic information transmission method in the above-described embodiments.

[0167] The computer readable storage medium provided by the application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, system or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0168] The above computer readable storage medium can be contained in the semantic information transmission device; or can exist separately and not be assembled into the semantic information transmission device.

[0169] The above computer readable storage medium carries one or more programs, which, when executed by the semantic information transmission device, cause the semantic information transmission device to perform the semantic information transmission method described above.

[0170] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0171] The flow diagrams and the block diagrams in the drawings are meant as methodological and functional description of implementations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0172] The modules involved in the embodiments of the present application can be implemented in software or hardware. In some cases, the names of the modules do not limit the modules themselves.

[0173] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer programs) for executing the above-mentioned semantic information transmission method, and can solve the technical problem that the traditional quantization variational autoencoder semantic communication transmission feature index is prone to error, resulting in serious distortion of semantics. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the semantic information transmission method provided by the above-mentioned embodiments, and will not be described here.

[0174] The above merely provides part of embodiments of the present application, and does not limit the scope of the present application, and any equivalent structure transformation made according to the technical concept of the present application, or direct / indirect application in other related technical fields, all included in the protection scope of the present application.

Claims

1. A method of semantic information transmission, characterized by, The semantic information transmission method is applied to a sending end and comprises the following steps: inputting to-be-transmitted data into a vector quantization variational autoencoder, extracting features of the to-be-transmitted data through a first neural network, and obtaining an initial feature vector; matching the initial feature vector with vectors in a preset discrete vector codebook to obtain a discrete feature vector, wherein the matching is replacing each vector in the initial feature vector with a vector having the smallest Euclidean distance in the discrete vector codebook; determining a first orthogonal frequency division multiplexing time-frequency resource distribution grid according to the discrete feature vector and a feature importance weight of the initial feature vector, and generating a sending signal based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid; transmitting the sending signal to a receiving end through a time-frequency fading channel.

2. The semantic information transmission method of claim 1, wherein, The step of determining the first orthogonal frequency division multiplexing time-frequency resource distribution grid according to the discrete feature vector and the feature importance weight of the initial feature vector, and generating the sending signal based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid, comprises the following steps: uniformly bit-quantizing the discrete feature vector to obtain a discrete bit stream; determining a gradient variation degree of a loss function in the first neural network according to the initial feature vector to generate a feature importance weight; allocating the discrete bit stream into an initial orthogonal frequency division multiplexing grid based on the feature importance weight to obtain a filled first orthogonal frequency division multiplexing time-frequency resource distribution grid; processing the first orthogonal frequency division multiplexing time-frequency resource distribution grid through inverse Fourier transform, and adding a cyclic prefix to a processing result to generate a sending signal.

3. The semantic information transmission method of claim 2, wherein, The step of allocating the discrete bit stream into the initial orthogonal frequency division multiplexing grid based on the feature importance weight to obtain the filled first orthogonal frequency division multiplexing time-frequency resource distribution grid, comprises the following steps: initializing a two-dimensional time-frequency resource grid of a coherent processing interval, and setting a preset reference signal in a rectangular manner through discrete uniform distribution in the two-dimensional time-frequency resource grid to obtain an initial orthogonal frequency division multiplexing grid; determining a target resource element quantity of important semantics according to a resource element quantity of the reference signal and a quantity of the feature importance weights; based on the feature importance weight and the target resource element quantity, dividing the discrete bit stream into important semantic features and non-important semantic features in the initial orthogonal frequency division multiplexing grid, modulating the important semantics around the reference signal, modulating and allocating the non-important semantics to blank resource elements to obtain a first orthogonal frequency division multiplexing time-frequency resource distribution grid.

4. The semantic information transmission method of claim 1, wherein, Before the step of inputting to-be-transmitted data into a vector quantization variational autoencoder, extracting features of the to-be-transmitted data through a first neural network, and obtaining an initial feature vector, the following step is further included: training a vector quantization variational autoencoder based on a preset data set to obtain a first neural network, a second neural network and a discrete vector codebook, wherein the first neural network is arranged in the vector quantization variational autoencoder, the second neural network is arranged in a vector quantization variational decoder, and the discrete vector codebook is shared by a sending end and a receiving end.

5. A method of semantic information transmission, characterized by, The semantic information transmission method is applied to a receiving end and comprises the following steps: receiving a receiving signal transmitted by a sending end through a time-frequency fading channel; performing channel equalization on the receiving signal to obtain a target semantic feature; performing bit dequantization on the target semantic feature to obtain a semantic feature vector; inputting the semantic feature vector into a vector quantization variational autoencoder, and performing re-matching with vectors in a discrete vector codebook through a nearest neighbor algorithm to correct transmission errors and obtain a corrected feature vector; inputting the corrected feature vector into a preset second neural network to obtain reconstructed data to be transmitted.

6. The semantic information transmission method of claim 5, wherein, The step of performing channel equalization on the receiving signal to obtain a target semantic feature comprises the following steps: removing the cyclic prefix of the receiving signal and performing Fourier transform on the removed result to obtain a second orthogonal frequency division multiplexing time-frequency resource distribution grid; estimating a first channel of a reference signal in the second orthogonal frequency division multiplexing time-frequency resource distribution grid, and estimating a second channel of semantic data in the second orthogonal frequency division multiplexing time-frequency resource distribution grid according to the first channel; determining a semantic signal at the semantic data according to the first channel and the second channel; demodulating the semantic signal and restoring the order of semantic features to generate a target semantic feature.

7. A semantic information transmission apparatus characterized by comprising: The device applied to a sending end comprises: a feature extraction module configured to input data to be transmitted into a vector quantization variational autoencoder, extract features of the data to be transmitted through a first neural network, and obtain an initial feature vector; a vector matching module configured to match the initial feature vector with vectors in a preset discrete vector codebook to obtain a discrete feature vector, wherein the matching is to replace each vector in the initial feature vector with a vector having the smallest Euclidean distance in the discrete vector codebook; a signal generation module configured to determine a first orthogonal frequency division multiplexing time-frequency resource distribution grid according to the discrete feature vector and the feature importance weight of the initial feature vector, and generate a sending signal based on the first orthogonal frequency division multiplexing time-frequency resource distribution grid; a signal generation module configured to transmit the sending signal to a receiving end through a time-frequency fading channel.

8. A semantic information transmission apparatus characterized by comprising: The device applied to a receiving end comprises: a signal receiving module configured to receive a receiving signal transmitted by a sending end through a time-frequency fading channel; a channel equalization module configured to perform channel equalization on the receiving signal to obtain a target semantic feature; a bit dequantization module configured to perform bit dequantization on the target semantic feature to obtain a semantic feature vector; a vector correction module configured to input the semantic feature vector into a vector quantization variational autoencoder, and perform re-matching with vectors in a discrete vector codebook through a nearest neighbor algorithm to correct transmission errors and obtain a corrected feature vector; a data reconstruction module configured to input the corrected feature vector into a preset second neural network to obtain reconstructed data to be transmitted.

9. A semantic information transmission apparatus characterized by comprising: The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the semantic information transmission method according to any one of claims 1 to 4 or 5 to 6.

10. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program. The computer program is executed by the processor to implement the steps of the semantic information transmission method in any one of claims 1 to 4 or 5 to 6.

Images