Transform encoder-based signal source number estimation method and system

Abstract

The invention discloses an information source number estimation method and system based on a transformer encoder, and relates to the data processing and artificial intelligence technology, and the method comprises the steps: obtaining input IQ data; performing complex value convolution residual preprocessing on the obtained IQ data to obtain spatial correlation characteristics of the high-order aliasing modulation signal; using a transformer encoder to extract the time sequence characteristics of the pre-processed signals, and using the transformer encoder to extract the time sequence characteristics of the pre-processed signals; and separating the classification vector to perform high-order aliasing source signal number estimation. According to the embodiment of the invention, the estimation accuracy of the high-order aliasing modulation signal can be effectively improved by using the complex value convolution reconstruction and the transformer encoder module, high accuracy can still be achieved under a low signal-to-noise ratio, and compared with an existing classical deep learning algorithm, the method does not consume excessive computing resources, and the model estimation accuracy is improved.

Description

Technical Field

[0001] The present application relates to the fields of data processing and artificial intelligence, and in particular to a method and system for estimating the number of signal sources based on a transformer encoder. Background Art

[0002] Deep learning networks have a strong learning ability for high-dimensional input features. Compared with traditional algorithms, they are easier to extract the correlation features of time series electromagnetic spectrum sample signals. The source number estimation method based on deep learning is generally divided into two steps. The first step is to adaptively extract the features contained in the signal, and the second step is to use the classifier for estimation. This method has higher robustness against complex external environments such as sampling number, noise environment and channel transmission. It performs better than traditional algorithms under low signal-to-noise ratio and low sampling number, and has higher stability in actual communication signal transmission. The design of the aliasing signal source number estimation model based on deep learning mainly includes two parts: network input feature selection and source number network structure design.

[0003] (1) Network input feature selection

[0004] In the selection of network input features, the aliased modulation signal features passed into the deep learning model mainly include IQ sequence, instantaneous phase and frequency, and signal-related image data. In order to retain the original features of the observed signal to the maximum extent and utilize the powerful ability of deep learning models to extract high-dimensional features, IQ signals are often used as input features of the model. In many fields of radio communications, researchers have designed deep learning networks to prove their feasibility on IQ signal data sets and proposed many interpretable theories for complex domain feature extraction problems. In addition, due to the strong robustness of instantaneous phase features, the instantaneous phase features of the signal extracted by first decomposing the signal through the set empirical mode and then extracting it according to the Hilbert-Huang transform are also often used as inputs to deep learning networks. With the breakthrough progress of deep learning networks in computer vision, researchers use time-frequency diagrams and signal constellation diagrams generated by signal processing algorithms such as Fourier transform, wavelet transform, and short-time Fourier transform to pass into the network model in order to utilize their powerful image feature extraction capabilities.

[0005] (2) Design of network structure for estimating the number of aliasing signal sources

[0006] The deep learning network structures used for source number estimation problems are mainly the following:

[0007] The first is the Convolutional Neural Network (CNN). CNN has the ability to learn and analyze the features of data samples of different categories. It uses the three characteristics of local perception, weight sharing and downsampling in the process of feature extraction and mapping, which distinguishes it from ordinary neural networks. Shrinivas Mahajan et al. used convolutional neural networks to accurately estimate the number of aliasing sources, proving that using deep learning networks to estimate the number of aliasing sources is a feasible and effective method under low signal-to-noise ratio conditions. John Rogers et al. improved the network by stacking more layers and conducted simulation analysis under different numbers of aliasing sources. The estimation results are far better than traditional algorithms. In terms of improving the structure of convolutional neural networks, Zhu Yanmin et al. proposed a one-dimensional convolutional residual network structure. When the number of aliasing sources is less than or equal to 7, the estimation accuracy is 8% higher than that of CNN.

[0008] The second is the Recurrent Neural Network (RNN). RNN can extract relevant features between sequences and is mainly used to process variable-length input sequences. It is widely used in sequence modeling fields such as speech recognition and natural language processing. Soumitro Chakrabarty et al. proposed a bidirectional long short-term memory network architecture (Long Short-Term Memory, LSTM) to estimate the number of aliasing sources. When the maximum number of aliasing is 10, the average absolute error of this method is less than 0.4. In order to reduce the complexity of LSTM and improve the convergence speed of the network, many scholars have combined CNN and LSTM, namely Convolutional, Long Short-term Memory, Fully Connected Deep Neural Network (CLDNN), using the powerful feature preprocessing capabilities of CNN to reduce the dimension of the feature input of the LSTM network, shorten the training time, and extract the time series features of the data through the LSTM network.

[0009] Traditional source number estimation algorithms estimate the number of sources by receiving signal matrices. Due to the difference between single-channel received signals and array signals, traditional algorithms cannot be directly used to estimate the number of sources for single-channel observation signals. In addition, in actual wireless communications, the use of high-order modulated signals can increase the signal transmission rate and make full use of the channel capacity. However, due to the reduction of redundant bits and the decrease of Euclidean distance, the anti-interference ability is poor. Most of the existing aliasing source number estimation algorithms are designed for low-order aliasing modulation signals, and their performance is poor in estimating the number of high-order aliasing signal sources. When there are multiple high-order modulation signals in the aliasing signal, it is easy to cause estimation confusion between adjacent source numbers. Summary of the invention

[0010] The embodiment of the present application provides a method and system for estimating the number of signal sources based on a transformer encoder, which utilizes complex-valued convolution reconstruction and a transformer encoder module to effectively improve the estimation accuracy of high-order aliasing modulated signals and improve the model estimation accuracy.

[0011] The embodiment of the present application provides a method for estimating the number of information sources based on a transformer encoder, comprising:

[0012] Get input IQ data;

[0013] The acquired IQ data is preprocessed by complex-valued convolution residual to obtain spatial correlation characteristics of high-order aliased modulation signals;

[0014] The preprocessed signal is encoded using a transformer to extract the temporal features of the preprocessed signal;

[0015] The classification vector is separated to perform the estimation of the number of high-order aliasing source signals.

[0016] Optionally, preprocessing the acquired IQ data by complex-valued convolution residual includes:

[0017] Decomposing the acquired IQ data into real and imaginary parts;

[0018] Using two sets of convolution kernels with different parameters and optimized gradients, spatial features are extracted from the real and imaginary parts of the IQ signal respectively;

[0019] The extracted spatial features are stitched together.

[0020] Optionally, decomposing the acquired IQ data into a real part and an imaginary part includes:

[0021] The acquired IQ data is defined as X n satisfy:

[0022] X n =I n +jQ n

[0023] In the above formula, I n Represents the input data X n The in-phase component of n Represents the orthogonal components of the input data; the filter factor is defined as W(n):

[0024] W n =A n +jB n

[0025] Perform complex domain convolution on IQ data:

[0026]

[0027] Decompose the convolution result into two parts: real part and imaginary part:

[0028]

[0029] Optionally, using two sets of convolution kernels with different parameters and optimizing gradients, spatial feature extraction is performed on the real part and imaginary part of the IQ signal respectively, including:

[0030] Use neurons to analyze IQ data X n Perform feature mapping:

[0031]

[0032] In the above formula, f(g) is the activation function, y is the output of the neuron,

[0033] Its loss value satisfies:

[0034] Loss = (y-y') 2

[0035] =(Re(y)-Re(y')) 2 +(Im(y)-Im(y')) 2

[0036] The parameter W is optimized using stochastic gradient descent. n Update the weights:

[0037]

[0038] Two sets of convolution kernels with different parameters are used to represent the convolution operation of the real part and imaginary part of the parameters with the IQ signal respectively, so as to extract spatial features respectively.

[0039] Optionally, it also includes introducing a residual structure between the data input and the last layer output of the network, and the residual structure includes an identity mapping layer y=x.

[0040] Optionally, the preprocessed signal is subjected to a transformer encoder to extract temporal features of the preprocessed signal including:

[0041] Slice the input data to convert it into a patch sequence;

[0042] The patch sequence is reduced in dimension through a linear flattening layer to generate a feature vector token;

[0043] Embed the position information of the feature vector token and add the classification vector class_token as a new classification basis;

[0044] Splice the classification vector class_token to the input of the transformer encoder.

[0045] Optionally, embedding position information of feature vector tokens includes:

[0046] Trigonometric functions are used to represent the relative position relationship of different feature vectors to perform position encoding, where the position embedding vector is represented as:

[0047]

[0048] Among them, pos represents the feature position, i represents the feature dimension, and d model represents the embedding encoding dimension, PE represents the position embedding feature vector;

[0049] Position information embedding is performed based on the result of position encoding.

[0050] An embodiment of the present application also proposes a source number estimation system based on a transformer encoder, including a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the aforementioned method are implemented.

[0051] The embodiment of the present application utilizes complex-valued convolution reconstruction and transformer encoder modules to effectively improve the estimation accuracy of high-order aliasing modulated signals, and can still achieve a high accuracy under low signal-to-noise ratio. Compared with existing classic deep learning algorithms, the method of the present application does not consume too many computing resources and improves the model estimation accuracy.

[0052] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0054] Figure 1 The overall process diagram of the method for estimating the number of signal sources based on the transformer encoder according to an embodiment of the present application is shown;

[0055] Figure 2 It is a schematic diagram of a complex convolution residual reconstruction model of a source number estimation method based on a transformer encoder according to an embodiment of the present application;

[0056] Figure 3 Schematic diagram of a two-layer residual structure of a method for estimating the number of signal sources based on a transformer encoder according to an embodiment of the present application;

[0057] Figure 4 This is a schematic diagram of the complete multi-head attention process of the source number estimation method based on the transformer encoder in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0059] The embodiment of the present application provides a method for estimating the number of signal sources based on a transformer encoder. The method of the present application first fully exploits the spatial correlation characteristics of high-order aliasing modulation signals through a complex-valued convolution residual preprocessing structure; then uses a transformer encoding module to extract the temporal characteristics of the preprocessed signal; finally, separates a classification vector as the basis for estimating the number of high-order aliasing source signals. Specifically, Figure 1 As shown, the method of the present application comprises the following steps:

[0060] In step S101, input IQ data is acquired.

[0061] In step S102, the acquired IQ data is preprocessed by complex-valued convolution residual to obtain spatial correlation features of high-order aliasing modulation signals.

[0062] In step S103, the preprocessed signal is subjected to a transformer encoder to extract the temporal features of the preprocessed signal.

[0063] In step S104 , the classification vector is separated to perform high-order aliasing source signal number estimation.

[0064] Digital modulated signals are in complex form, consisting of two mutually orthogonal components, the real part and the imaginary part. It is very important to effectively utilize the correlation characteristics between different components in the convolution operation. Most of the calculation processes, encapsulation functions and basic principles in deep learning are based on real number analysis. Some researchers have redefined the convolution layer for complex convolution, but this method lacks theoretical support for key technologies of neural networks and is difficult to implement. Some literature concatenates complex signals into two-dimensional tensors and then extracts feature information through a two-dimensional convolution kernel. This method does not fully utilize local features during the convolution process, and the convolution result loses complex information. The method of the embodiment of the present application designs a convolution residual model based on complex signals. In some embodiments, Figure 2 The complex-valued convolution residual reconstruction model of the embodiment of the present application is shown, and the acquired IQ data is preprocessed by the complex-valued convolution residual, including:

[0065] Decomposing the acquired IQ data into a real part and an imaginary part. In some embodiments, decomposing the acquired IQ data into a real part and an imaginary part comprises:

[0066] The acquired IQ data is defined as X n satisfy:

[0067] X n =I n +jQ n (1)

[0068] In the above formula, I n Represents the input data X n The in-phase component of n represents the orthogonal components of the input data;

[0069] Define the filter factor as W(n):

[0070] W n =A n +jB n (2)

[0071] Perform complex domain convolution on IQ data:

[0072]

[0073] Decomposing the convolution result into two parts, the real part and the imaginary part, the above formula can be expressed in matrix form:

[0074]

[0075] Using two sets of convolution kernels with different parameters and optimized gradients, spatial features are extracted from the real and imaginary parts of the IQ signal respectively;

[0076] The extracted spatial features are stitched together.

[0077] In some embodiments, using two sets of convolution kernels with different parameters and optimizing gradients to extract spatial features from the real part and the imaginary part of the IQ signal respectively includes:

[0078] Use neurons to analyze IQ data X n Perform feature mapping:

[0079]

[0080] In the above formula, f(g) is the activation function, y is the output of the neuron,

[0081] Taking the mean absolute error (MSE) loss function as an example, its loss value satisfies:

[0082]

[0083] The parameter W is optimized using stochastic gradient descent. n Update the weights:

[0084]

[0085] Two sets of convolution kernels with different parameters are used to represent the convolution operation of the real part and imaginary part of the parameter with the IQ signal respectively, so as to extract spatial features respectively. Specifically, two sets of convolution kernels with different parameters can be used to represent the convolution operation of the real part and imaginary part of the parameter with the IQ signal respectively. Therefore, in the process of complex convolution back propagation, the neural network can use The gradient of the loss function consisting of two sets of different parameters is optimized.

[0086] With the introduction and successful application of deep network models such as VGG and GoogleNet, complex network models have been proven to be able to extract richer and more abstract data features, and dimensionality expansion has become the mainstream direction for building convolutional neural network models. However, after a large number of experiments, it was found that on the training data set, the accuracy of deep network models decreases with the increase in the number of layers, and the network model cannot improve network performance by simply stacking layers. In this application, in order to solve the problems of gradient vanishing and performance degradation caused by too many layers, residual jump connections are introduced to make the model more stable. In some embodiments, a residual structure is also introduced between the data input and the output of the last layer of the network, and the residual structure includes an identity mapping layer y=x. Specifically, Figure 3In the example, F(x) is the output of the first layer of the network, F(x)+x is the final mapping output of the network, the input feature x and the output feature F(x) after convolution have the same dimension, and the residual structure directly maps the input to the output. Compared with the traditional directly connected convolutional neural network, the residual structure introduces the identity mapping layer y=x, which solves the performance degradation problem of deep learning networks due to the stacking of layers and has a more powerful feature learning ability.

[0087] In some embodiments, Figure 4 As shown, the preprocessed signal is subjected to a transformer encoder to extract the temporal features of the preprocessed signal, including:

[0088] Slice the input data to convert it into a patch sequence;

[0089] The patch sequence is reduced in dimension through a linear flattening layer to generate a feature vector token;

[0090] Embed the position information of the feature vector token and add the classification vector class_token as a new classification basis;

[0091] The classification vector class_token is concatenated to the input of the transformer encoder, and the model separates the class_token in the output sequence as the basis for classification.

[0092] In the RNN unit, the calculation order is single, there are long-term dependency problems, and the model's parallel ability is limited. Although the unit structures such as LSTM and GRU (Gate Recurrent Unit, GRU) introduce gated units based on RNN, which alleviate the impact of the long-term dependency problem to a certain extent, it has not been completely eliminated. To solve the above problems, the transformer structure is introduced into the model. There are two main improvements. The first is to use the attention mechanism to reduce the position information to a constant; the second is to use a non-sequential structure to enable the model to have parallel computing capabilities. The transformer model is a new breakthrough beyond RNN and CNN. The model structure does not use CNN and RNN basic units, and completely realizes the mapping from input to output based on the self-attention mechanism. The position encoding is used to give the model the ability to capture the sequence order association characteristics. According to the angle formula sin(α+β)=sinαcosβ-cosαsinβ and cos(α+β)=cosαcosβ-sinαsinβ and the embedding vector of position m+n can be linearly transformed by the position embedding vector of position m. In some embodiments of the present application, embedding the position information of the feature vector token includes:

[0093] Trigonometric functions are used to represent the relative position relationship of different feature vectors to perform position encoding, where the position embedding vector is represented as:

[0094]

[0095] Among them, pos represents the feature position, i represents the feature dimension, and d model represents the embedding encoding dimension, and PE represents the position embedding feature vector.

[0096] The position-encoded model fully utilizes the parallel computing capabilities of the GPU, ensures the correlation between input features, and solves the long-term dependency problem in sequence processing. In addition, the multi-head attention mechanism in the transformer model is used to integrate operations to extract and fuse features of different dimensions in the input features, ensuring the diversity of weights. The dot product calculation process between each weight and the input vector does not affect each other, making the model have stronger parallel computing and feature extraction capabilities.

[0097] Position information embedding is performed based on the result of position encoding.

[0098] This application designs a single-channel aliasing source number estimation method based on complex convolution reconstruction and transformer encoding blocks. First, in data preprocessing, in order to solve the problem that it is difficult to extract the joint feature information of IQ channels when deep learning networks are directly used for complex data, a convolution residual preprocessing module for complex signals is proposed to reconstruct data and fully explore the spatial characteristics of electromagnetic signals. Secondly, the transformer encoder is used to extract the timing correlation characteristics of the signal, enhance the parallel computing capability, and solve the long-term dependency problem. Simulation results show that complex-valued convolution reconstruction and transformer encoder modules can effectively improve the estimation accuracy of high-order aliasing modulated signals, and can still achieve a high accuracy under low signal-to-noise ratio. Compared with the existing classic deep learning algorithms, it improves the model estimation accuracy without consuming too many computing resources.

[0099] An embodiment of the present application also proposes a source number estimation system based on a transformer encoder, including a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the aforementioned method are implemented.

[0100] It should be noted that in the various embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0101] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0102] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0103] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for estimating the number of information sources based on a transformer encoder, characterized in that: include: Get input IQ data; The acquired IQ data is preprocessed by complex-valued convolution residual to obtain spatial correlation characteristics of high-order aliased modulation signals; The preprocessed signal is encoded using a transformer to extract the temporal features of the preprocessed signal; The classification vector is separated to perform the estimation of the number of high-order aliasing source signals.

2. The method for estimating the number of information sources based on a transformer encoder according to claim 1, characterized in that: The acquired IQ data is preprocessed by complex-valued convolution residuals including: Decomposing the acquired IQ data into real and imaginary parts; Using two sets of convolution kernels with different parameters and optimized gradients, spatial features are extracted from the real and imaginary parts of the IQ signal respectively; The extracted spatial features are stitched together.

3. The method for estimating the number of information sources based on a transformer encoder according to claim 2, characterized in that: Decomposing the acquired IQ data into real and imaginary parts includes: The acquired IQ data is defined as X n satisfy: X n =I n +jQ n In the above formula, I n Represents the input data X n The in-phase component of n represents the orthogonal components of the input data; Define the filter factor as W(n): W n =A n +jB n Perform complex domain convolution on IQ data: H=W n *X(n) =(A n +jB n )*(I n +jQ n ) =[A n *I n -B n *Q n ]+j[A n *Q n +B n *I n ] Decompose the convolution result into two parts: real part and imaginary part:

4. The method for estimating the number of information sources based on a transformer encoder according to claim 3, characterized in that: Using two sets of convolution kernels with different parameters and optimized gradients, spatial feature extraction is performed on the real and imaginary parts of the IQ signal, including: Using neurons to analyze IQ data X n Perform feature mapping: In the above formula, f(g) is the activation function, y is the output of the neuron, Its loss value satisfies: Loss=(y-y') 2 =(Re(y)-Re(y')) 2 +(Im(y)-Im(y')) 2 The parameter W is optimized using stochastic gradient descent. n Update the weights: Two sets of convolution kernels with different parameters are used to represent the convolution operation of the real part and imaginary part of the parameters with the IQ signal respectively, so as to extract spatial features respectively.

5. The method for estimating the number of information sources based on a transformer encoder according to claim 4, characterized in that: It also includes introducing a residual structure between the data input and the output of the last layer of the network, wherein the residual structure includes an identity mapping layer y=x.

6. The method for estimating the number of information sources based on a transformer encoder as described in claim 5, characterized in that: The preprocessed signal is then transformed using a transformer encoder to extract the temporal features of the preprocessed signal, including: Slice the input data to convert it into a patch sequence; The patch sequence is reduced in dimension through a linear flattening layer to generate a feature vector token; Embed the position information of the feature vector token and add the classification vector class_token as a new classification basis; Splice the classification vector class_token to the input of the transformer encoder.

7. The method for estimating the number of information sources based on a transformer encoder as described in claim 6, characterized in that: Embedding the position information of the feature vector token includes: Trigonometric functions are used to represent the relative position relationship of different feature vectors to perform position encoding, where the position embedding vector is represented as: Among them, pos represents the feature position, i represents the feature dimension, and d model represents the embedding encoding dimension, PE represents the position embedding feature vector; Position information embedding is performed based on the result of position encoding.

8. A source number estimation system based on a transformer encoder, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images