Robust semantic communication method, device and equipment based on sparse vector coding

Abstract

The invention relates to the technical field of wireless communication, and discloses a robust semantic communication method, device and equipment based on sparse vector coding, and the method comprises the steps: extracting continuous semantic features of original data through a semantic encoder model; discretizing the continuous semantic features into index bits through a vector quantization method by using a learnable feature dictionary; the index bit is mapped into a sparse vector by adopting sparse vector coding, and the sparse vector is sent after being expanded by a codebook; and a receiving end identifies the index through a multipath matching pursuit algorithm and reconstructs the original data. According to the method, a semantic feature extraction model, a vector quantization technology and sparse vector coding are fused, and gradient approximation, joint loss function optimization and a transmission parameter dynamic adjustment mechanism are designed, so that the problems of poor digital transmission compatibility and weak channel adaptability of traditional semantic communication are effectively solved; and the semantic transmission reliability and the system resource utilization efficiency under the dynamic channel condition are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of wireless communication, in particular to a robust semantic communication method, device and equipment based on sparse vector coding. BACKGROUND

[0002] In view of the massive communication demand in the dynamic channel environment, semantic communication is a key technology to improve the efficiency of intelligent task execution and the efficiency of system resource allocation. In typical dynamic scenarios such as industrial Internet of Things and Internet of Vehicles, the channel state often shows significant time-varying and non-stationary characteristics. Therefore, how to design a robust semantic communication scheme compatible with existing digital communication systems and adaptive to channel changes is an important link to realize high-reliability and high-efficiency information transmission.

[0003] The existing mainstream semantic communication scheme mainly adopts Deep Joint Source-Channel Coding (DJSCC) based on deep learning, which directly learns the mapping relationship between source information and channel symbols through deep neural network, to realize efficient transmission of semantics. However, such method has obvious defects: first, it relies on analog transmission mode, hindering its integration application in actual digital communication system; second, it has poor channel adaptability, as the model parameters are fixed after training and cannot respond to dynamic changes of channel state in real time. SUMMARY

[0004] Therefore, the present application provides a robust semantic communication method, device and equipment based on sparse vector coding to solve the problems of digital communication incompatibility and poor adaptability to channel changes in semantic communication.

[0005] In the first aspect, the present application provides a robust semantic communication method based on sparse vector coding, which is applied to a sending end, and the method comprises the following steps: Obtaining original data and encoding the original data by using a semantic encoder to extract continuous semantic features; Based on a learnable feature dictionary, the continuous semantic features are vector quantized to obtain discrete code word indexes corresponding to the continuous semantic features; According to the discrete code word indexes, a sparse vector is constructed, and the sparse vector is randomly spread by using a non-orthogonal codebook to obtain a spread spectrum signal; The spread spectrum signal is loaded on multiple subcarriers and sent to a receiving end through a target channel.

[0006] The application provides a robust semantic communication method based on sparse vector coding, which introduces vector quantization technology to discretize continuous semantic features into bit streams, so that the semantic information can be compatible with the modulation and demodulation and coding and decoding processes of existing digital communication systems, solves the technical challenge that the traditional semantic communication scheme is difficult to deploy integration in actual digital networks, and combines the sparse vector transmission technology, so that the key parameters can be dynamically adjusted according to the real-time channel state without relying on complex neural network retraining or fine tuning, solves the technical challenge that the traditional semantic communication scheme cannot adapt to the dynamic change of channel conditions in real time; the index-based information bearing form and the decoding method based on compressed sensing are adopted, so that the channel fading and noise interference are effectively resisted, the reliability of semantic transmission is enhanced, and the technical challenge that the performance of the traditional digital semantic communication scheme sharply decreases in a harsh channel environment is solved.

[0007] In an optional implementation, the continuous semantic features are vector quantized based on a learnable feature dictionary to obtain discrete code word indexes corresponding to the continuous semantic features, including: The learnable feature dictionary and the semantic encoder are jointly trained and optimized to obtain a discretized feature dictionary representation; Based on the discretized feature dictionary representation, the continuous semantic features are discretized into discrete code word indexes by using a nearest neighbor method.

[0008] The application provides a robust semantic communication method based on sparse vector coding, which jointly trains and optimizes the learnable feature dictionary and the semantic encoder to obtain an efficient low-dimensional discretized semantic feature representation, effectively solves the problem of poor compatibility of the traditional semantic communication digital transmission, and improves the reliability of semantic transmission.

[0009] In an optional implementation, the discrete code word indexes include index bits and modulation bits, and the sparse vector is constructed according to the discrete code word indexes, including: According to the index bits, the position indexes of non-zero elements in the sparse vector are determined by using a sparse transformation mapping table, and according to the modulation bits, the modulation constellation values and constellation rotation angles of the non-zero elements are determined; The sparse vector is constructed according to the position indexes, the modulation constellation values and the constellation rotation angles of the non-zero elements.

[0010] In an optional implementation, the method further includes: The real-time channel state of the target channel is obtained, and at least one parameter of the sparse vector is adjusted according to the real-time channel state; Based on the adjusted parameter value, the sparse vector is constructed in combination with the position indexes, the modulation constellation values and the constellation rotation angles of the non-zero elements.

[0011] The application provides a robust semantic communication method based on sparse vector coding.

[0012] In a second aspect, the application provides a robust semantic communication method based on sparse vector coding. Receiving the spread spectrum signal transmitted through the target channel, and performing sparse recovery on the spread spectrum signal by using a multipath matching algorithm to obtain a discrete code word index; Based on the discrete code word index and a learnable feature dictionary, determining the continuous semantic features corresponding to the discrete code word index; Reconstructing the continuous semantic features by using a semantic decoder to obtain reconstructed data, wherein the semantic decoder is obtained by joint training and optimization of the semantic encoder and the learnable feature dictionary.

[0013] In an optional implementation, the sparse recovery on the spread spectrum signal by using the multipath matching algorithm to obtain the discrete code word index comprises: Obtaining a transmission configuration parameter, and determining a search path order table according to the transmission configuration parameter; Iteratively calculating each search path based on the search path order table to determine the non-zero element index and the corresponding non-zero element value under each search path; According to the non-zero element index and the corresponding non-zero element value under each search path, calculating the residual of each search path and comparing; Selecting the search path with the smallest residual as the optimal solution, and restoring the index bits and the modulation bits according to the optimal solution.

[0014] The robust semantic communication method based on sparse vector coding provided by the application combines the sparse vector transmission technology, and the key parameters can be dynamically adjusted according to the real-time channel state without relying on complex neural network retraining or fine-tuning, thereby solving the technical challenge that the traditional semantic communication scheme cannot adapt to the dynamically changing channel conditions in real time. Through the design of vector quantization and gradient stopping, the problem that the quantization link is not derivable in neural network training is solved. Through the dictionary learning and basis vector representation method, only the index needs to be transmitted without transmitting the complete feature vector, so that the data amount of transmission can be reduced, and the semantic loss caused by quantization can be reduced.

[0015] In a third aspect, the application provides a robust semantic communication device based on sparse vector coding. The data encoding module is configured to obtain original data, and encode the original data by using a semantic encoder to extract continuous semantic features. a vector quantization module, configured to perform vector quantization on the continuous semantic features based on a learnable feature dictionary to obtain discrete code word indexes corresponding to the continuous semantic features; a sparse vector construction module, configured to construct a sparse vector according to the discrete code word indexes and perform random spread spectrum on the sparse vector by using a non-orthogonal codebook to obtain a spread spectrum signal; a data sending module, configured to load the spread spectrum signal onto a plurality of subcarriers and send the spread spectrum signal to a receiving end through a target channel.

[0016] In a fourth aspect, the present application provides a robust semantic communication device based on sparse vector coding, which is located at a receiving end and comprises: a data receiving module, configured to receive the spread spectrum signal transmitted through the target channel and perform sparse recovery on the spread spectrum signal by using a multipath matching algorithm to obtain the discrete code word indexes; a code word index module, configured to determine the continuous semantic features corresponding to the discrete code word indexes based on the discrete code word indexes and the learnable feature dictionary; a data decoding module, configured to reconstruct the continuous semantic features by using a semantic decoder to obtain reconstructed data.

[0017] In a fifth aspect, the present application provides a computer device, which comprises a memory and a processor, the memory and the processor are communicatively connected with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the method of the first aspect or any of the corresponding embodiments thereof or the method of the second aspect or any of the corresponding embodiments thereof.

[0018] In a sixth aspect, the present application provides a computer readable storage medium, which stores computer instructions, and the computer instructions are used to make a computer execute the method of the first aspect or any of the corresponding embodiments thereof or the method of the second aspect or any of the corresponding embodiments thereof. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0020] Figure 1 is a flowchart of a robust semantic communication method based on sparse vector coding according to an embodiment of the present application applied to a sending end; Figure 2 is a data transmission diagram of a robust semantic communication method based on sparse vector coding according to an embodiment of the present application processing data. Figure 3 FIG. 7 is a schematic diagram of a codebook vector quantization model in a sparse vector coding-based robust semantic communication method according to an embodiment of the present application; Figure 4 FIG. 8 is a schematic diagram of the performance relationship between the minimum number of transmission subcarriers and the channel signal-to-noise ratio in a sparse vector coding-based robust semantic communication method according to an embodiment of the present application; Figure 5 FIG. 9 is a flowchart of the application of a sparse vector coding-based robust semantic communication method to a receiving end according to an embodiment of the present application; Figure 6 FIG. 10 is a schematic diagram of the peak signal-to-noise ratio performance comparison of different transmission methods in a specific embodiment of a sparse vector coding-based robust semantic communication method according to an embodiment of the present application; Figure 7 FIG. 11 is a structural block diagram of a sparse vector coding-based robust semantic communication device located at a sending end according to an embodiment of the present application; Figure 8 FIG. 12 is a structural block diagram of a sparse vector coding-based robust semantic communication device located at a receiving end according to an embodiment of the present application; Figure 9 FIG. 13 is a hardware structure schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings in the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0022] For semantic communication scenarios in complex dynamic channel environments, such as dynamic unmanned clusters in industrial Internet of Things and Internet of Vehicles, achieving high-reliability and high-efficiency transmission of semantic information is a core requirement to support such communication scenarios. In these scenarios, on the one hand, key semantic information contained in the transmission content, such as device status, control instructions, and environmental understanding, often has highly generalized characteristics, and semantic coding can greatly reduce the transmission data volume. On the other hand, wireless channel states, such as signal-to-noise ratio and multipath fading characteristics, have significant time variability. Therefore, directly deploying an end-to-end semantic communication scheme trained under specific channel conditions and based on continuous analog value transmission to actual dynamic channels is likely to cause a significant performance decline, and cannot simultaneously meet the dual requirements of high efficiency and high reliability in dynamic environments.

[0023] The related art and the technical problems existing therein include: a. Directly map the original image to channel transmission symbols using a convolutional neural network, which can significantly improve performance compared to traditional schemes in poor channel conditions. However, this method uses analog transmission, which lacks compatibility with existing mature digital communication systems, limiting the practical deployment of semantic communication; and the semantic transmission model needs to be trained under fixed channel conditions, and the model performance decreases significantly when the actual deployment environment is inconsistent with the training environment.

[0024] b. Add a feature attention module to the network to dynamically adjust the feature weight according to the channel signal-to-noise ratio and transmission feature mean value, realizing the self-adaptation of a single model to different channel states. This method uses an attention mechanism to realize channel adaptation, which has a large amount of calculation and many parameters, which is not conducive to deployment on resource-constrained devices.

[0025] c. Use a prediction network to combine features, channel conditions and compression ratio to predict image transmission quality, and then realize optimal rate control under a given target. This method uses analog transmission, which lacks compatibility with existing mature digital communication systems, limiting the practical deployment of semantic communication.

[0026] Based on the problems of the above related technologies, the embodiment of the present application provides a robust semantic communication method based on sparse vector coding, which discretizes continuous semantic features into bit streams through vector quantization technology, and dynamically adjusts key parameters by combining sparse vector transmission technology, to achieve the effects of digital compatibility of semantic transmission and self-adaptation to dynamic channels.

[0027] According to the embodiment of the present application, a robust semantic communication method based on sparse vector coding is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] In this embodiment, a robust semantic communication method based on sparse vector coding is provided, which can be used in the above computer system, Figure 1 is a flowchart of a robust semantic communication method based on sparse vector coding according to the embodiment of the present application. The method is applied to the sending end of semantic communication, as shown in Figure 1 The flowchart includes the following steps: Step S101, obtaining original data and encoding the original data using a semantic encoder to extract continuous semantic features.

[0029] Specifically, as shown in Figure 2As shown, a data transmission schematic diagram for processing data by the robust semantic communication method based on sparse vector coding. The sending end determines the original data to be sent, including but not limited to: image data, sound data, text data, etc. The original data is encoded by using a semantic encoder to extract semantic features. The semantic encoder includes a semantic feature extraction module composed of a convolutional neural network, which is used to map the original data to low-dimensional continuous semantic features. The semantic feature extraction module in the semantic encoder and the semantic feature reconstruction module in the semantic decoder both use a convolutional neural network model as a semantic transmission model to realize the compression mapping of the original information to low-dimensional continuous semantic features and the semantic recovery of the quantized features to the reconstructed information.

[0030] In step S102, the continuous semantic features are vector quantized based on the learnable feature dictionary to obtain the discrete code word index corresponding to the continuous semantic features.

[0031] Specifically, in order to make the constructed semantic transmission model compatible with the digital communication system, a learnable feature dictionary with a dimension of is constructed, wherein, is the number of codebooks, i.e. the number of basis vectors , and the range of the output index corresponds to the number of is the feature dimension, which corresponds to the output channel number of the semantic feature extraction module.

[0032] The process of semantic feature extraction is to find a discrete code word for representing the corresponding feature in the learnable feature dictionary in a nearest neighbor manner, and output the discrete code word index corresponding to the discrete code word for transmission, realizing the transformation from continuous semantic features to discrete indexes The specific process of vector quantization is shown in Figure 3 , only the index represented by low-dimensional bits needs to be transmitted without transmitting the complete feature vector , and the code words in the feature dictionary are shared at both the sending and receiving ends as a priori, significantly reducing the amount of data transmitted.

[0033] In step S103, a sparse vector is constructed according to the discrete code word index, and a non-orthogonal codebook is used to randomly spread the sparse vector to obtain a spread spectrum signal.

[0034] Specifically, in order to ensure high reliable transmission and dynamic adaptability of semantic features, in the actual deployment transmission stage, sparse vector coding with significant advantages in short packet communication is introduced. The transmission performance in bad channels is improved by carrying information in the form of index, and the key transmission parameters of sparse vector coding are not determined by model training, so the transmission parameters can be flexibly adjusted according to the channel conditions to realize dynamic environment adaptation.

[0035] ​The sending end first splits the original data bits to be transmitted into two parts, one part as index part bits, used to determine the position index of the non-zero elements in the sparse vector (the mapping relationship between the index and the bits is pre-set through a mapping table, and needs to meet a specific logarithmic relationship constraint); the other part as a modulation part bit group, used to determine the modulation symbol and load it onto the non-zero element value, to complete the construction of the sparse vector.

[0036] After the sparse vector is constructed, the Bernoulli distribution codebook matrix (non-orthogonal matrix) realizes random spread spectrum of the sparse vector and loads it onto subcarriers for transmission, and the transmitted spread spectrum signal can be expressed as: (1) wherein, is the spread spectrum code word corresponding to the non-zero index in the codebook, is the corresponding modulation constellation value, is the corresponding constellation rotation angle.

[0037] Step S104 loads the spread spectrum signal onto multiple subcarriers and transmits it to the receiving end through the target channel.

[0038] Specifically, the signal obtained by non-orthogonal codebook spread spectrum is distributed to multiple subcarriers, and based on the set channel transmission parameters, it is transmitted to the receiving end through the target channel.

[0039] The sparse vector coding-based robust semantic communication method provided in the embodiment solves the technical challenge that the traditional semantic communication scheme is difficult to deploy and integrate in actual digital networks by introducing vector quantization technology to discretize continuous semantic features into bit streams, so that semantic information can be compatible with the modulation and demodulation and coding and decoding processes of existing digital communication systems. Combined with the sparse vector transmission technology, the key parameters do not need to rely on complex neural network retraining or fine-tuning, but can be dynamically adjusted according to the real-time channel state, solving the technical challenge that the traditional semantic communication scheme cannot adapt to dynamically changing channel conditions in real time. The index-based information carrying form and the decoding method based on compressed sensing effectively resist channel fading and noise interference, enhance the reliability of semantic transmission, and solve the technical challenge that the performance of the traditional digital semantic communication scheme sharply decreases in a harsh channel environment.

[0040] In some optional embodiments, the above step S102 includes: Step S1021, jointly train and optimize the learnable feature dictionary and the semantic encoder to obtain a discretized feature dictionary representation.

[0041] Specifically, the semantic feature extraction module, semantic feature reconstruction module and vector quantization feature dictionary are jointly trained and optimized to obtain efficient low-dimensional discretized semantic feature representation. The sparse vector coding transmission module does not participate in model training to ensure the model's differentiability.

[0042] The loss function of the semantic transfer model is designed to jointly optimize the semantic encoder, semantic decoder, and learnable feature dictionary while maintaining different learning rates, significantly reducing the semantic quantization loss.

[0043] The loss function setting during training considers the following two aspects: First, the gap between the original image and the reconstructed image is made as small as possible, and the mean square error is used for evaluation, that is, , which is used to optimize the feature extraction and reconstruction module; secondly, to minimize the loss caused by quantization, the output of the feature extraction module and the basis vectors in the feature dictionary To be as close as possible, Considering the inconsistent learning rates of the feature extraction module and the feature dictionary, this item is divided into Approach Obtain a better feature dictionary and Approach Make the output of the feature extraction module not deviate too much from the two parts of the feature dictionary, corresponding to and . That is, the total loss function of the training process is set as: (2) in, represents the loss value, Indicates the transmission signal, represents the feature vector to be encoded, Represents the basis vector in the feature dictionary, sg is the gradient stop operator, when forward transmission, , while in reverse gradient descent, .

[0044] After the training process is completed, the semantic transfer model can be used to obtain better semantic encoding and decoding and discrete representation effects.

[0045] Step S1022 : Based on the discretized feature dictionary representation, the continuous semantic features are discretized into discrete codeword indexes using a nearest neighbor approach.

[0046] Specifically, the quantization stage uses the stopping gradient operator sg to solve the non-differentiable problem, that is, the forward propagation output , and back propagation transfers the gradient value to , the quantization process is considered to have no gradient, so the input of the semantic decoder can be expressed as: (3) wherein, denotes the input of the semantic decoder, and in the forward transmission, and in the backward gradient descent, The gradient function is defined as to meet the required function, that is, the quantization link itself is not derivable, so the gradient is directly set to 0, the gradient update of the quantization link is skipped, and the derivability of the overall model is ensured.

[0047] Assuming that the transmission signal is an input image, and a low-dimensional feature is output by a semantic feature extraction module, The quantization process refers to finding the basis vector with the smallest distance in the feature dictionary and outputting its index The conversion formula for the discrete semantic feature representation is: wherein, is the feature vector to be encoded, is the code word corresponding to in the learnable feature dictionary, is the index to be transmitted (a discrete value from 1 to ), is the number of code words in the feature dictionary. Discretizing the continuous semantic feature into a discrete code word index realizes the discrete representation of the semantic feature, and the transmission process only needs to transmit the index , and the basis vector does not need to participate in the transmission as prior knowledge of the transmitting and receiving parties.

[0048] The robust semantic communication method based on sparse vector coding provided in the embodiment solves the problem of poor compatibility of traditional semantic communication digital transmission by jointly training and optimizing the learnable feature dictionary and the semantic encoder to obtain efficient low-dimensional discrete semantic feature representation, and improves the reliability of semantic transmission by ensuring the derivability of the model without participating in the model training of the sparse vector coding transmission.

[0049] In some optional embodiments, the discrete code word index includes an index bit and a modulation bit, and the step S103 of constructing the sparse vector according to the discrete code word index includes: Step S1031, determining the position index of the non-zero element in the sparse vector according to the index bit by using a sparse transformation mapping table, and determining the modulation constellation value and constellation rotation angle of the non-zero element according to the modulation bit.

[0050] Step S1032, constructing the sparse vector according to the position index, the modulation constellation value and the constellation rotation angle of the non-zero element.

[0051] The transmitting end transmits the index The information bits are divided into index bits and modulation bits , wherein the index bits determine the position index of the non-zero elements in the sparse vector through a sparse transformation mapping table, and satisfy the relationship , wherein is the length of the sparse vector, is the number of non-zero elements; the modulation bits are modulated as quadrature amplitude modulation (M-order QAM modulation) constellation values as the non-zero element values , and satisfy .

[0052] In some optional embodiments, the method further comprises: obtaining a real-time channel state of the target channel, and adjusting at least one parameter of the sparse vector according to the real-time channel state.

[0053] Based on the adjusted parameter value, the position index of the non-zero element, the modulation constellation value and the constellation rotation angle are combined to construct the sparse vector.

[0054] Specifically, according to the real-time channel state such as signal-to-noise ratio, the parameters such as the length of the sparse vector , the number of non-zero elements , the number of transmission bits , the number of transmission subcarriers are dynamically optimized, and the present embodiment is described by taking the signal-to-noise ratio as an example. As shown in Figure 4 , when the fixed peak signal-to-noise ratio threshold is 20 dB, the relationship diagram of the minimum number of transmission subcarriers required by the system transmission and the signal-to-noise ratio is shown, and it can be seen from Figure 4 that as the signal-to-noise ratio increases, the number of required subcarriers decreases significantly, and only 16% of the number of transmission subcarriers is required when the signal-to-noise ratio is 10 dB compared with the signal-to-noise ratio of -5 dB to achieve the same performance. Therefore, the transmission parameters such as the number of transmission subcarriers can be dynamically optimized according to the real-time channel state (including but not limited to the signal-to-noise ratio): the coding rate is reduced to improve the transmission reliability at low signal-to-noise ratio, and the coding rate is increased to reduce the transmission resource occupation at high signal-to-noise ratio. Flexible adjustment of transmission resources, dynamic adaptation to changing channel environment, improve the adaptability of the system, and optimize the resource allocation efficiency of the system.

[0055] The robust semantic communication method based on sparse vector coding provided in the present embodiment adopts sparse vector coding to transmit discrete index values, carries information in the form of sparse index, and decodes at the receiving end by using the compression sensing method. The key transmission parameters can be flexibly adjusted according to the channel conditions, which can improve the adaptability of the model to dynamically changing channel environment.

[0056] A robust semantic communication method based on sparse vector coding is provided in the embodiment, which can be applied to the computer system described above, Figure 5 is a flowchart of the robust semantic communication method based on sparse vector coding according to the embodiment of the application, applied to the receiving end of semantic communication, as shown in Figure 5 , the flowchart includes the following steps: Step S201, receiving the spread spectrum signal transmitted through the target channel, and performing sparse recovery on the spread spectrum signal by using a multipath matching algorithm to obtain discrete code word indexes.

[0057] Specifically, the spread spectrum signal is recovered by using a multipath matching (MMP) algorithm, and the received signal after passing through a channel with channel characteristics can be expressed as a linear combination of some columns in the observation matrix with non-zero element values , and the expression is as follows: (5) wherein, denotes the channel gain of the kth subcarrier in the Orthogonal Frequency Division Multiplexing (OFDM) channel, denotes the additive white Gaussian noise with variance . The decoding process can be modeled as a sparse recovery problem in compressed sensing, and the expression is as follows:

[0058] (6) wherein, is the estimated sparse vector index, is the received signal, is the candidate index set, is the observation matrix. By constructing multiple parallel search paths, and calculating the correlation between the observation matrix

[0059] and the residual at each iteration, the estimation of the non-zero position index in the sparse vector is realized, and the least square method is used to iteratively update the residual to obtain all the non-zero position indexes and element values.

[0060] Step S202, determining the continuous semantic features corresponding to the discrete code word indexes based on the discrete code word indexes and the learnable feature dictionary.

[0061] Specifically, as shown in Figure 2 , the continuous semantic features corresponding to the discrete code word indexes are determined based on the discrete code word indexes and the learnable feature dictionary.​As shown, the receiving end uses the learnable feature dictionary in the semantic database to perform inverse quantization on the discrete code word index, and obtains the corresponding code word from the learnable feature dictionary according to the transmitted discrete code word index Instead of the original feature vector , as the input of the semantic feature reconstruction module in the semantic decoder, the inverse quantization process is completed.

[0062] The quantization stage and the inverse quantization stage use the stop gradient operator sg to solve the non-differentiable problem, that is, the forward propagation output , and the gradient value is transmitted to The quantization process is considered to have no gradient, so the input of the semantic decoder is as shown in formula (3), which will not be described here.

[0063] In step S203, the continuous semantic features obtained are reconstructed by using the semantic decoder to obtain reconstructed data, and the semantic decoder is jointly trained and optimized with the semantic encoder and the learnable feature dictionary.

[0064] Specifically, in the end-to-end model training stage, a semantic feature extraction model and a semantic feature reconstruction module are constructed by using a convolutional neural network, so as to realize the transformation of the sending end from the original data to low-dimensional continuous semantic features , and the receiving end from low-dimensional continuous semantic features to reconstructed data .

[0065] The robust semantic communication method based on sparse vector coding provided in the embodiment combines the sparse vector transmission technology, and the key parameters do not need to rely on complex neural network retraining or fine tuning, but can be dynamically adjusted according to the real-time channel state, solving the technical challenge that the traditional semantic communication scheme cannot adapt to the dynamically changing channel conditions in real time. Through the design of vector quantization and gradient stop, the non-differentiable problem of the quantization link in the neural network training is solved. Through the method of dictionary learning and basis vector representation, only the index needs to be transmitted without transmitting the complete feature vector, which can reduce the amount of data transmitted and reduce the semantic loss generated by quantization.

[0066] In an optional implementation, the above step S201 includes: In step S2011, the transmission configuration parameters are obtained, and the search path order table is determined according to the transmission configuration parameters.

[0067] Specifically, before iteration, according to the number of non-zero elements , the number of sub-paths at each level and the maximum number of search paths , the search path order table is calculated, and a plurality of foreground optimal search paths are determined as the candidate search path order table: (7) wherein, is the order of the search path, is the sub-path number selected at each iteration level, indicating that each level will arrange the inner product in descending order and take the order as the corresponding index.

[0068] Step S2012, based on the search path order table, iteratively calculates each search path to determine the non-zero element index and the corresponding non-zero element value under each search path.

[0069] Specifically, based on the search path order table, iteratively calculates each search path, specifically including: calculating the inner product of each column in the observation matrix and the residual , wherein the initial value of the residual is .

[0070] Select the index of the column corresponding to the th in the descending order of the inner product, as the estimation of the non-zero element index in the sparse vector, and this process can be represented as: (8) wherein, is the estimated non-zero element index of this layer, is the observation matrix, is the residual.

[0071] Update the obtained non-zero element value using the least square method: (9) wherein, is the estimated value of the first non-zero element, is the combination of the columns in corresponding to the first non-zero element index, denotes the pseudo-inverse matrix of the matrix , that is .

[0072] Update the residual .

[0073] Repeat the above process until all non-zero element indexes and corresponding non-zero element values under the corresponding search path are found.

[0074] Step S2013, according to the non-zero element index and the corresponding non-zero element value under each search path, calculate the residual of each search path and compare.

[0075] Specifically, the residual of each search path is calculated respectively, and the final sparse vector is obtained by selecting the index set with the minimum residual, that is: (10) wherein, denotes the sparse vector set obtained by searching each path.

[0076] In step S2014, the search path with the minimum residual is selected as the optimal solution, and the index bits and modulation bits are restored according to the optimal solution.

[0077] Specifically, in the multi-path matching pursuit sparse recovery algorithm, a multi-path search space is constructed, the inner product of the observation matrix and the residual is calculated respectively, the non-zero index is estimated iteratively, and the non-zero value estimation and the residual are updated by using the least square method, and finally the path with the minimum residual is selected as the optimal solution.

[0078] After obtaining the sparse vector estimation by the MMP algorithm, the index bits part is obtained by inverse mapping of the non-zero element index with the aid of the sparse transformation table, and the modulation bits part is obtained by demodulation of the non-zero element value, that is, the process of transmitting the sparse vector is completed.

[0079] In one specific embodiment, the performance of the semantic communication system is simulated and verified by using the Modified National Institute of Standards and Technology (MNIST) database in a Rayleigh fading channel. The system compression ratio is set to 1 / 16, the codebook quantity is 16, the codebook dimension is 128, the number of bits represented by a single sparse vector is 16, the number of non-zero elements is 2, and the modulation mode is QPSK modulation. The relationship between the transmitted image peak signal-to-noise ratio (PSNR) and the channel signal-to-noise ratio is shown in Figure 6 , and the number of transmission subcarriers is set to 32. As can be seen from Figure 6 , the performance of the semantic transmission method proposed in the application is better than that of the traditional digital semantic communication method without using sparse vector coding and the traditional method without using semantic communication, and significant advantages are shown at different signal-to-noise ratios. For example, when the signal-to-noise ratio is 10 dB, the method proposed in the application is improved by 26% compared with the traditional digital semantic communication method, and by 80% compared with the traditional non-semantic method, which fully shows the advantages of the method proposed in the application in digital communication compatibility and improving the reliability of semantic transmission.

[0080] A robust semantic communication apparatus based on sparse vector coding is also provided in the embodiments, which is configured to implement the above-described embodiments and preferred embodiments, and the description of which has been made above and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementation in hardware or a combination of software and hardware is also possible and contemplated.

[0081] The embodiments provide a robust semantic communication apparatus based on sparse vector coding, which is located at a sending end, as shown in Figure 7 The apparatus comprises: A data encoding module 701 is configured to acquire original data, encode the original data by using a semantic encoder, and extract continuous semantic features.

[0082] A vector quantization module 702 is configured to perform vector quantization on the continuous semantic features based on a learnable feature dictionary, to obtain discrete code word indexes corresponding to the continuous semantic features.

[0083] A sparse vector construction module 703 is configured to construct a sparse vector according to the discrete code word indexes, and perform random spread spectrum on the sparse vector by using a non-orthogonal codebook, to obtain a spread spectrum signal.

[0084] A data sending module 704 is configured to load the spread spectrum signal onto a plurality of subcarriers, and send the spread spectrum signal to a receiving end through a target channel.

[0085] The embodiments provide a robust semantic communication apparatus based on sparse vector coding, which is located at a receiving end, as shown in Figure 8 The apparatus comprises: A data receiving module 801 is configured to receive a spread spectrum signal transmitted through a target channel, and perform sparse recovery on the spread spectrum signal by using a multipath matching algorithm, to obtain discrete code word indexes.

[0086] A code word index module 802 is configured to determine continuous semantic features corresponding to the discrete code word indexes based on the discrete code word indexes and a learnable feature dictionary.

[0087] A data decoding module 803 is configured to reconstruct the continuous semantic features by using a semantic decoder, to obtain reconstructed data.

[0088] Further function descriptions of the above-described various modules and units are the same as those of the above-described corresponding embodiments, and will not be repeated here.

[0089] The robust semantic communication device based on sparse vector coding in the embodiment is presented in the form of functional units, where the units refer to ASIC (Application Specific Integrated Circuit) circuits, processors and memories executing one or more software or fixed programs, and / or other devices that can provide the above functions.

[0090] The embodiment of the present application also provides a computer device having the robust semantic communication device based on sparse vector coding shown in the above Figure 7 or Figure 8 .

[0091] Please refer to Figure 9 , Figure 9 is a structural schematic diagram of a computer device provided by an optional embodiment of the present application, as shown in the figure, the computer device comprises one or more processors 10, a memory 20, and an interface for connecting components, including a high-speed interface and a low-speed interface. Various components are communicatively connected to each other by different buses, and can be installed on a common motherboard or in other manners as required. The processor can process instructions executed in the computer device, including instructions stored in the memory or the memory to display graphical information on a GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memories, if necessary. Similarly, multiple computer devices can be connected, each providing part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 9 Figure 9 The processor 10 in the above

[0092] The processor 10 can be a central processor, a network processor, or a combination thereof. The processor 10 can further include a hardware chip. The hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device can be a complex programmable logic device, a field programmable logic gate array, a generic array logic, or any combination thereof.

[0093] The memory 20 stores instructions executable by the at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0094] ​The memory 20 can include a program storage area and a data storage area. The program storage area can store an operating system, application programs required for at least one function, and the like. The data storage area can store data created according to the use of the computer device, and the like. In addition, the memory 20 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one of a magnetic disk storage device, a flash memory device, or other non-transitory solid state memory device. In some alternative embodiments, the memory 20 can optionally include a memory disposed remotely from the processor 10, which can be connected to the computer device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0095] The memory 20 can include a volatile memory such as a random access memory, and can further include a non-volatile memory such as a flash memory, a hard disk, or a solid state disk, and a combination thereof.

[0096] The computer device further includes a communication interface 30 for communication of the computer device with other devices or communication networks.

[0097] The embodiments of the present application also provide a computer readable storage medium. The method according to the embodiments of the present application can be implemented in hardware, firmware, or as software code to be recorded in a storage medium, or originally stored in a remote storage medium or a non-transitory machine readable storage medium to be downloaded through a network and stored in a local storage medium, so that the method described herein can be processed by such software using a general purpose computer, a special purpose processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid state disk, and the like. Further, the storage medium can include a combination of the above-mentioned storage media. It can be understood that the computer, the processor, the microprocessor controller, or the programmable hardware includes a storage component that can store or receive software or computer code, which, when accessed and executed by the computer, the processor, or the hardware, implements the method shown in the above embodiments.

[0098] Although the embodiments of the present application have been described with reference to the accompanying drawings, various modifications and changes can be suggested to one skilled in the art, and it is intended that the present application encompass such modifications and changes as fall within the scope of the appended claims.

Claims

1. A robust semantic communication method based on sparse vector coding, characterized in that: The method is applied to a transmitting end, and the method includes: Obtaining original data, and encoding the original data using a semantic encoder to extract continuous semantic features; Based on a learnable feature dictionary, vector quantization is performed on the continuous semantic feature to obtain a discrete codeword index corresponding to the continuous semantic feature; Constructing a sparse vector according to the discrete codeword index, and randomly spreading the sparse vector using a non-orthogonal codebook to obtain a spread spectrum signal; The spread spectrum signal is loaded onto multiple subcarriers and sent to a receiving end through a target channel.

2. The method according to claim 1, characterized in that Based on a learnable feature dictionary, vector quantization is performed on the continuous semantic feature to obtain a discrete codeword index corresponding to the continuous semantic feature, including: Jointly training and optimizing the learnable feature dictionary and the semantic encoder to obtain a discretized feature dictionary representation; Based on the discretized feature dictionary representation, the continuous semantic features are discretized into discrete codeword indexes using a nearest neighbor approach.

3. The method according to claim 1, characterized in that The discrete codeword index includes: an index bit and a modulation bit, and constructing a sparse vector according to the discrete codeword index includes: Determine, according to the index bit, a position index of a non-zero element in a sparse vector using a sparse transformation mapping table, and determine, according to the modulation bit, a modulation constellation value and a constellation rotation angle of the non-zero element; A sparse vector is constructed according to the position index of the non-zero element, the modulation constellation value and the constellation rotation angle.

4. The method according to claim 3, characterized in that The method further comprises: Acquiring a real-time channel state of a target channel, and adjusting at least one parameter of the sparse vector according to the real-time channel state; Based on the adjusted parameter values, a sparse vector is constructed in combination with the position index of the non-zero element, the modulation constellation value and the constellation rotation angle.

5. A robust semantic communication method based on sparse vector coding, characterized in that The method is applied to a receiving end, and the method includes: receiving a spread spectrum signal transmitted via a target channel, and performing sparse recovery on the spread spectrum signal using a multipath matching algorithm to obtain a discrete codeword index; Determining a continuous semantic feature corresponding to the discrete codeword index based on the discrete codeword index and a learnable feature dictionary; The continuous semantic features are reconstructed using a semantic decoder to obtain reconstructed data, wherein the semantic decoder is jointly trained and optimized with a semantic encoder and a learnable feature dictionary.

6. The method according to claim 5, characterized in that Performing sparse recovery on the spread spectrum signal using a multipath matching algorithm to obtain a discrete codeword index, including: Obtaining transmission configuration parameters, and determining a search path sequence table according to the transmission configuration parameters; Performing iterative calculations on each search path based on the search path sequence table to determine the non-zero element index and the corresponding non-zero element value under each search path; According to the non-zero element index and the corresponding non-zero element value under each search path, the residual of each search path is calculated and compared; The search path with the smallest residual is selected as the optimal solution, and the index bits and modulation bits are restored according to the optimal solution.

7. A robust semantic communication device based on sparse vector coding, characterized in that: The device is located at the transmitting end and includes: A data encoding module is used to obtain original data, encode the original data using a semantic encoder, and extract continuous semantic features; A vector quantization module, configured to perform vector quantization on the continuous semantic features based on a learnable feature dictionary to obtain discrete codeword indexes corresponding to the continuous semantic features; a sparse vector construction module, configured to construct a sparse vector according to the discrete codeword index, and perform random spectrum spreading on the sparse vector using a non-orthogonal codebook to obtain a spread spectrum signal; The data sending module is used to load the spread spectrum signal onto multiple subcarriers and send it to the receiving end through the target channel.

8. A robust semantic communication device based on sparse vector coding, characterized in that: The device is located at the receiving end and includes: A data receiving module is used to receive the spread spectrum signal transmitted via the target channel and perform sparse recovery on the spread spectrum signal using a multipath matching algorithm to obtain a discrete codeword index; A codeword indexing module, configured to determine a continuous semantic feature corresponding to the discrete codeword index based on the discrete codeword index and a learnable feature dictionary; The data decoding module is used to reconstruct the continuous semantic features using a semantic decoder to obtain reconstructed data.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the robust semantic communication method based on sparse vector coding according to any one of claims 1 to 4 or 5 to 6 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to enable a computer to execute the robust semantic communication method based on sparse vector coding according to any one of claims 1 to 4 or 5 to 6.

Images