A method for time series signal recognition based on complex-valued interferometric neural network

Abstract

The present invention proposes a method for time series signal recognition based on a complex-valued interference neural network, comprising: preprocessing the time series signal and calculating the complex-valued time-frequency diagram of the signal; constructing a complex-valued interference neural network including an interference convolution layer and a frequency domain fully connected layer; calculating a classification loss function, and training the complex-valued interference neural network based on a gradient descent method; and deploying and testing a complex-valued interference neural network model.

Description

Technical Field

[0001] The present invention relates to a time series signal recognition method based on a complex-valued interference neural network, mainly targeting the signal recognition problem of complex-valued time-frequency graph data, and belongs to the technical field of machine learning and time series signal processing. Background Art

[0002] Time series data often appears in many fields, such as speech, communication, finance, and biomedicine. Traditional time series problems usually require human feature engineering before the preprocessed data can be input into the machine learning algorithm, and this feature engineering usually requires some expertise in a specific field, which further increases the preprocessing cost. However, with the emergence of deep learning, convolutional neural networks can already handle computer vision tasks more perfectly. The network will automatically combine the most basic features into more advanced and abstract features layer by layer to complete computer vision tasks. However, convolutional neural network algorithms based on one-dimensional time series signals often cannot give full play to the powerful self-learning ability of convolutional neural networks. Studies have shown that convolutional neural networks can efficiently extract the features of spatial data, so the convolutional neural network time series signal recognition method based on the two-dimensional time-frequency diagram of time series signals has become a mainstream algorithm.

[0003] However, signal time-frequency diagrams are very different from natural images. The horizontal and vertical axes of the time-frequency diagram represent time and frequency respectively, while the horizontal and vertical axes of natural images both represent space. The network structure layer designed for natural images, such as the convolution layer, does not take into account the difference in physical meaning between the horizontal and vertical axes. Therefore, although it has achieved some results in the task of time series signal recognition, it still lacks the necessary robustness and interpretability. In response to the above problems, the present invention proposes a time series signal recognition method based on a complex-valued interference neural network. The core idea is to fully exploit the autocorrelation characteristics of the time series signal by constructing a complex-valued interference network including an interference convolution layer and a frequency domain fully connected layer, thereby extracting the distinguishing features of various types of time series signals, and finally realizing the classification or recognition of time series signals. Summary of the invention

[0004] The technical problem solved by the present invention is: time series signal recognition for complex-valued time-frequency graph data. The problem can be described as constructing a complex neural network structure containing optimizable parameters, and optimizing the parameters by gradient descent method, so that the structure can automatically discover the distinguishing features of different categories of time series signals from the complex-valued time-frequency graph, and finally use its features to achieve certain given goals. In such a process, the structure of the neural network directly affects the quality of the extracted features. The present invention provides a neural network structure with good interpretability and noise robustness.

[0005] The present invention discloses a method for identifying time series signals based on a complex-valued interference neural network, comprising: preprocessing the time series signal and calculating the complex-valued time-frequency diagram of the signal; constructing a complex-valued interference neural network including an interference convolution layer and a frequency domain fully connected layer; calculating a classification loss function, and training the complex-valued interference neural network based on a gradient descent method; and deploying and testing a complex-valued interference neural network model.

[0006] The flowchart of using complex-valued interferometric neural network for RF fingerprint recognition is as follows: Figure 1 As shown, it mainly includes the following steps:

[0007] Step S1, preprocessing the time series signal and calculating the complex valued time-frequency diagram of the time series signal;

[0008] Step S2, constructing a complex-valued interference neural network, the complex-valued interference neural network is mainly characterized by alternating interference convolution layers and frequency domain fully connected layers in sequence, wherein the interference convolution layer performs a complex-valued convolution operation in the time dimension on the complex-valued time-frequency graph to simulate the interference process of the wave, and the frequency domain fully connected layer performs a complex-valued fully connected operation in the frequency dimension on the complex-valued time-frequency graph to simulate the nonlinear process of the wave;

[0009] Step S3, calculating the classification loss function, and performing complex-valued interference neural network training based on the gradient descent method to obtain a complex-valued interference neural network model;

[0010] Step S4, according to step S1, obtain the complex-valued time-frequency diagram data to be tested from the time series signal to be tested, use the complex-valued interference neural network model and the complex-valued time-frequency diagram data to be tested to obtain the probability that the time series signal to be tested belongs to each category, and finally take the category corresponding to the maximum probability as the final prediction result of the complex-valued interference neural network, thereby realizing the recognition of the time series signal.

[0011] In step S2, the frequency domain fully connected layer includes: performing a dimension transformation on the input tensor so that the last dimension of the transformed tensor is the frequency dimension; defining two fully connected layers, which are equivalent to a complex-valued fully connected layer; calculating the real tensor and the imaginary tensor of the equivalent complex-valued fully connected layer output tensor based on the dimensionally transformed tensor and the two fully connected layers; performing dimension splicing on the real tensor and the imaginary tensor, and transforming the spliced ​​tensor into a five-dimensional tensor with the same meaning as the input tensor as the final output result of the frequency domain fully connected layer.

[0012] In step S2, the interference convolution layer includes: taking the real and imaginary parts of the input data, and then transforming the dimensions of the real tensor and the imaginary tensor so that the last dimension of the transformed tensor is the time dimension; defining two convolution layers, which are equivalent to a complex-valued convolution layer, wherein the weight parameters of the two convolution layers are initialized using the Kaiming normal initialization method; calculating the real tensor and the imaginary tensor of the equivalent complex-valued convolution layer output tensor according to the transformed tensor and the two convolution layers; dimensionally splicing the real tensor and the imaginary tensor, and transforming the spliced ​​tensor into a five-dimensional tensor with the same meaning as the input tensor as the final output result of the interference convolution layer.

[0013] The constructed complex-valued interference neural network includes repeatedly stacking frequency domain fully connected layers and interference convolution layers for several times to increase the network depth; connecting to a statistical pooling layer to convert complex-valued data into real-valued data and obtain a feature vector for each sample; and finally connecting to two layers of ordinary fully connected layers for feature transformation, and outputting the probability that the sample belongs to each category.

[0014] In the complex-valued interference neural network, when frequency-domain fully connected layers and interference convolutional layers are repeatedly stacked, the skip-layer connection or dense connection method in the ResNet and DenseNet network architectures is adopted to further increase the network depth.

[0015] Step S1: preprocess the time series signal and calculate the complex valued time-frequency diagram of the signal.

[0016] This step includes the following two sub-steps.

[0017] Step S101: preprocess the time series signal, such as normalization, filtering, etc.

[0018] Step S102: Calculate the complex-valued time-frequency diagram of the signal after preprocessing.

[0019] The embodiment of the present invention adopts the classic short-time Fourier transform method. The process of short-time Fourier transform is: multiply a time-limited window function w(n) before Fourier transforming the signal, and assume that the non-stationary signal is stationary within the short time interval of the analysis window. By moving the window function on the time axis, the signal is analyzed section by section to obtain a set of local "spectra" of the signal. The calculation formula is as follows:

[0020]

[0021] Where x(n) is the input time series signal, which can be either a complex-valued signal or a real-valued signal. w(n) is the window function, which can be a Hamming window. R is the number of signal sampling points of the adjacent signal window movement. m(ω) is the complex-valued time-frequency graph data, where m represents time (indicating the mth frame of short-time signal) and ω represents frequency. The complex-valued time-frequency graph data after frequency discretization is recorded as X, where X is a three-dimensional tensor, and its dimensions represent the real and imaginary parts (2D), the number of frequency dimensions, and the number of time dimensions, respectively, as follows:

[0022] X:=[2, Freq, Time].

[0023] Finally, the three-dimensional tensor X is transformed into a five-dimensional tensor X_ReSize, that is:

[0024] X_ReSize:=[1, 2, Freq, 1, Time].

[0025] X_ReSize is the input of the complex-valued interference neural network.

[0026] Step S2: Construct a complex-valued interferometric neural network including an interferometric convolutional layer and a frequency-domain fully connected layer.

[0027] The present invention mainly proposes a new neural network structure, namely the complex-valued interference network, which is a key step of the present invention. The present invention is improved from the traditional convolutional neural network and is similar to the traditional convolutional neural network in principle. The network replaces the convolution layer of the traditional convolutional network with a frequency domain fully connected layer and an interference convolution layer to achieve the purpose of considering the time domain and frequency domain respectively, and has stronger interpretability and robustness. Therefore, this step includes the following sub-steps.

[0028] Step S201: construct a frequency domain fully connected layer.

[0029] The input data of the frequency domain fully connected layer is recorded as FcIn, which is a five-dimensional tensor expressed as:

[0030] FcIn:=[Batch, 2, Freq_FcIn, FeatureMaps, Time].

[0031] Among them, Batch represents the number of samples in each batch, 2 represents the real and imaginary parts of the complex value, Freq_FcIn represents the frequency dimension of the input data of the frequency domain fully connected layer, FeatureMaps represents the number of feature maps (the dimension of the initial time-frequency map is 1), and Time represents the time dimension, that is, the number of frames of the time-frequency map.

[0032] (1) Transform the dimension of the input FcIn so that the last dimension of the transformed data FcIn_ReSize is the frequency dimension, that is:

[0033] FcIn_ReSize:=[Batch*FeatureMaps*Time,2,Freq_FcIn].

[0034] Among them, Batch*FeatureMaps*Time represents the product of the number of samples in each batch, the number of feature maps, and the number of time dimensions.

[0035] The following steps (2)-(4) are the process of FcIn_ReSize passing through the equivalent complex-valued fully connected layer.

[0036] (2) Define the fully connected layers fc_re and fc_im, which represent the real and imaginary parts of the frequency domain fully connected layer, respectively. This step can be directly implemented using the torch.nn.Linear function.

[0037] (3) Calculate the equivalent complex-valued fully connected layer output:

[0038] FcOut_re=fc_re(FcIn_ReSize[:,0,:])-fc_im(FcIn_ReSize[:,1,:]),

[0039] FcOut_im=fc_re(FcIn_ReSize[:,1,:])+fc_im(FcIn_ReSize[:,0,:]).

[0040] For ease of description, the above formula uses the notation for array slicing in Python, which will be used below.

[0041] (4) Concatenate the dimensions of FcOut_re and FcOut_im, and transform the concatenated tensor into a five-dimensional tensor, keeping the meaning of each dimension of the transformed tensor FcOut the same as that of the input tensor FcIn:

[0042] FcOut:=[Batch,2,Freq_FcOut,FeatureMaps,Time].

[0043] Among them, Batch represents the number of samples in each batch, 2 represents the real and imaginary dimensions of the complex-valued output, Freq_FcOut represents the frequency dimension of the output data of the frequency domain fully connected layer, FeatureMaps represents the number of feature maps, and Time represents the time dimension, that is, the number of frames of the frequency map.

[0044] Step S202: construct an interference convolution layer.

[0045] The input data of the interference convolution layer is recorded as ConvIn. ConvIn is a five-dimensional tensor expressed as:

[0046] ConvIn:=[Batch,2,Freq,FeatureMaps_ConvIn,Time].

[0047] Among them, Batch represents the number of samples in each batch, 2 represents the real and imaginary parts of the complex value, Freq represents the frequency dimension, FeatureMaps_ConvIn represents the number of input feature maps of the interference convolution layer (the dimension of the initial time-frequency map is 1), and Time represents the time dimension, that is, the number of frames of the time-frequency map.

[0048] (1) Take the real and imaginary parts of the input data as ConvIn_re and ConvIn_im, that is:

[0049] ConvIn_re=ConvIn[:,0],

[0050] ConvIn_im=ConvIn[:,1].

[0051] Perform dimension transformation on ConvIn_re and ConvIn_im respectively so that the transformed tensors are three-dimensional tensors, namely:

[0052] ConvIn_re_ReSize:=[Batch,Freq*FeatureMaps_ConvIn,Time],

[0053] ConvIn_im_ReSize:=[Batch,Freq*FeatureMaps_ConvIn,Time].

[0054] Among them, Freq*FeatureMaps_ConvIn represents the frequency dimension Freq multiplied by the number of input feature maps FeatureMaps_ConvIn of the interference convolution layer.

[0055] The following steps (2)-(4) are the process of ConvIn_re_ReSize and ConvIn_im_ReSize passing through the equivalent complex-valued interference convolution layer.

[0056] (2) Define the optimizable parameters weight_re and weight_im. The three-dimensional tensor can be expressed as weight_re: = [Freq*FeatureMaps_ConvOut, FeatureMaps_ConvIn, KernelSize], weight_im: = [Freq*FeatureMaps_ConvOut, FeatureMaps_ConvIn, KernelSize].

[0057] Among them, KernelSize, FeatureMaps_ConvIn, and FeatureMaps_ConvOut represent the kernel size of the interference convolution layer, the number of input feature maps, and the number of output feature maps. Freq*FeatureMaps_ConvOut represents the frequency dimension Freq multiplied by the number of output feature maps FeatureMaps_ConvIOut of the interference convolution layer. This step can be implemented using torch.nn.Parameter. The above parameters can all be initialized using the Kaiming normal initialization method.

[0058] (3) Calculate the equivalent complex-valued convolutional layer output:

[0059] ConvOut_re=F.conv1d(ConvIn_re_ReSize, weight=self.weight_re, bias=None, groups

[0060] =Freq)-F.conv1d(ConvIn_im_ReSize, weight=self.weight_im, bias

[0061] =None, groups=Freq)

[0062] ConvOut_im=F.conv1d(ConvIn_re_ReSize, weight=self.weight_im, bias=None, groups

[0063] =Freq)+F.conv1d(ConvIn_im_ReSize,weight=self.weight_re,bias

[0064] =None, groups=Freq)

[0065] In the formula, the variables ConvIn_re_ReSize and ConvIn_im_ReSize represent the results of the dimension transformation of the real and imaginary parts of the input of the interference convolution layer, respectively. The variables ConvOut_re and ConvOut_im represent the real and imaginary parts of the output of the interference convolution layer, respectively. F.conv1d is a one-dimensional convolution operation of torch.nn.functional. Weight, bias, and groups are three parameters of the function F.conv1d, representing the weight, bias, and number of groups of the convolution layer, respectively. Self.weight_re (or self.weight_im), None, and Freq are the values ​​of the three parameters. Among them, self.weight_re and self.weight_im are weight_re and weight_im after Kaiming normal initialization, respectively; bias = None means that the F.conv1d operation does not add bias, and groups = Freq means that the number of groups is equal to the number of frequency dimensions. The number of groups equal to the number of frequency dimensions is the most critical feature that distinguishes the interference convolution layer from the ordinary convolution layer.

[0066] (4) Concatenate the dimensions of ConvOut_re and ConvOut_im, and transform the concatenated tensor into a five-dimensional tensor, keeping the meaning of each dimension of the transformed tensor ConvOut the same as that of the input tensor ConvIn, that is:

[0067] ConvOut:=[Batch, 2, Freq, FeatureMaps_ConvOut, Time].

[0068] Among them, Batch represents the number of samples in each batch, 2 represents the real and imaginary dimensions of the complex-valued output, Freq represents the number of frequency dimensions, FeatureMaps_ConvOut represents the number of feature maps of the output data of the interference convolution layer, and Time represents the number of time dimensions, that is, the number of frames of the frequency map.

[0069] Step S203: construct a complete complex-valued interference neural network.

[0070] The construction method is roughly as follows:

[0071] (1) Repeat the stacking of frequency domain fully connected layers and interference convolutional layers several times to increase the network depth.

[0072] (2) Access the statistical pooling layer. The input PoolingIn of this layer is the following five-dimensional tensor:

[0073] PoolingIn:=[Batch, 2, Freq, FeatureMaps, Time].

[0074] Among them, Batch represents the number of samples in each batch, 2 represents the two dimensions of real and imaginary parts, Freq represents the number of frequency dimensions, FeatureMaps represents the number of feature maps, and Time represents the number of time dimensions, that is, the number of frames of the frequency map.

[0075] The output of the statistical pooling layer can be used as the time series signal feature extracted by the network, which is a two-dimensional tensor, namely:

[0076] Feature:=[Batch, FeatureMaps].

[0077] It can be seen that the statistical pooling layer can convert complex-valued data into real-valued data, and transform the features of each sample into a vector of length FeatureMaps. The specific calculation formula of this layer is:

[0078]

[0079] The above formula uses the notation for array element indexing in Python, that is, Feature[i,n] represents the value of the two-dimensional tensor Feature when the first dimension is i and the second dimension is n, In[i,0,k,n,m] and In[i,1,k,n,m] represent the values ​​of the input tensor In when the dimensions are i,0,k,n,m and i,1,k,n,m respectively.

[0080] (3) The final two layers of ordinary fully connected layers are used for feature transformation and output the probability that the sample belongs to each category, that is:

[0081] Out:=[Batch,Num_Classes].

[0082] Among them, Out represents the final output of the complex-valued interference network, Batch represents the number of samples in each batch, and Num_Classes represents the number of communication terminals to be classified.

[0083] Step S3: Calculate the classification loss function and perform complex-valued interference neural network training based on the gradient descent method.

[0084] First, the network input complex-valued time-frequency graph data is passed through the complex-valued interference neural network layer by layer to obtain its output. Based on its output, the loss function of the network output relative to the true label of the sample can be calculated. For example, the cross entropy loss function:

[0085]

[0086] In the above formula, L represents the value of the loss function, N is the number of samples in each batch, and x i represents the input of the last fully connected layer, y irepresents the true label of the sample, W, and b represents the weight parameter and bias parameter to be optimized in the last fully connected layer.

[0087] Then, the automatic differentiation mechanism in torch is used to calculate the gradient of the loss function relative to each parameter to be optimized in the complex-valued interference network, and the gradient descent method is used to update each parameter.

[0088]

[0089] Among them, θ is any parameter to be optimized in the complex-valued interference network, θ′ is the value of the parameter to be optimized θ after gradient descent update, It is the gradient of the loss function with respect to each parameter to be optimized in the complex-valued interference network. η is the learning rate, which is a hyperparameter and can be adjusted, such as 5e-4.

[0090] Finally, the above loss function calculation and gradient descent process are repeated several times until convergence.

[0091] Step S4: Complex-valued interference neural network model deployment and testing.

[0092] The implementation process of this step is relatively simple. First, according to the method described in step S1, the complex-valued time-frequency diagram data to be tested is obtained from the time series signal to be tested. The complex-valued time-frequency diagram data to be tested uses the complex-valued interference neural network obtained in step S3 to obtain the probability that the time series signal to be tested belongs to each category. Finally, the category corresponding to the maximum probability is taken as the final prediction result of the complex-valued interference neural network.

[0093] The advantages of the present invention compared with the prior art are:

[0094] (1) The present invention focuses on the difference between the time axis and frequency axis of the time-frequency graph and the natural image in physical meaning, and designs a complex-valued interference network including an interference convolution layer and a frequency-domain fully connected layer. The interference convolution layer and the frequency-domain fully connected layer in the network respectively simulate the interference process and nonlinear process of the wave, so that the output of each layer of the network has a clear physical meaning. This shows that the time series signal recognition algorithm based on the complex-valued interference network has good interpretability. On the other hand, the time series signal recognition algorithm based on the complex-valued interference network can not only improve the distinguishability of the learned features (such as Figure 2 ), obtain a higher recognition accuracy, and also have strong robustness (such as Figure 3 ).

[0095] (2) Time series signal recognition (including voice signal recognition, communication signal recognition, stock signal recognition, etc.) has broad application prospects in real life. Traditional time series signal recognition technology mainly uses artificially designed features for recognition, which has the disadvantages of low efficiency and limited application scope. The current mainstream method is to convert time series signals into image signals and use deep convolutional neural networks for recognition. However, this method does not take into account the characteristics of time series signals and lacks clear physical meaning, resulting in a certain room for performance improvement. The present invention draws on wave interference theory to design a new neural network structure to realize time series signal recognition, which has a clearer physical meaning and also achieves better experimental results. This method can be applied in speech recognition, Internet of Things, communications and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 It is a flow chart of the method for identifying time series signals based on complex-valued interference neural network of the present invention;

[0097] Figure 2 These are two connection modes of the frequency domain fully connected layer and the interference convolutional layer in the complex-valued interference network of the present invention; (a) direct connection type; (b) dense connection type;

[0098] Figure 3 It is a dimensionality reduction distribution diagram, where (a) and (b) are the dimensionality reduction distribution diagrams of the features extracted by the traditional complex-valued ResNet network and the complex-valued interference neural network respectively;

[0099] Figure 4 This is a bar chart of the signal recognition accuracy of the traditional complex-valued ResNet network and the complex-valued interference neural network. DETAILED DESCRIPTION

[0100] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0101] Time series signals usually have a variety of manifestations, such as voice signals, communication signals, bioelectric signals, etc. The time series signal recognition method based on complex-valued interference network proposed in the present invention is applicable to any time series signal. The implementation process of the present invention is explained in detail below by taking the radio frequency fingerprint recognition of communication signals as an example. Radio frequency fingerprint recognition technology extracts the essential difference information of different sending terminal devices from the received radio frequency signal through signal processing methods, thereby distinguishing the sending terminal communication individuals. Therefore, the radio frequency fingerprint recognition problem is essentially a time series signal recognition problem.

[0102] like Figure 1 As shown, the specific implementation steps of the present invention are as follows:

[0103] Step S1: preprocess the communication signal and calculate the complex valued time-frequency diagram of the signal.

[0104] This step includes the following two sub-steps.

[0105] Step S101: pre-process the communication signal, such as burst detection, power normalization, linear filtering, etc. This step has strong flexibility, and any reasonable pre-processing method can be used.

[0106] Step S102: Calculate the complex-valued time-frequency diagram of the signal after preprocessing.

[0107] The embodiment of the present invention adopts the classic short-time Fourier transform method. The process of short-time Fourier transform is: multiply a time-limited window function w(n) before Fourier transforming the signal, and assume that the non-stationary signal is stationary within the short time interval of the analysis window. By moving the window function on the time axis, the signal is analyzed section by section to obtain a set of local "spectra" of the signal. The calculation formula is as follows:

[0108]

[0109] Where x(n) is the input communication signal, which can be a complex-valued IQ signal or a real-valued signal, w(n) is the window function, which can be a Hamming window, R is the number of signal sampling points of the adjacent signal window movement, and X m (ω) is the complex-valued time-frequency graph data, where m represents time (indicating the mth frame of short-time signal) and ω represents frequency. The complex-valued time-frequency graph data after frequency discretization is recorded as X, where X is a three-dimensional tensor, and its dimensions represent the real and imaginary parts (2D), the number of frequency dimensions, and the number of time dimensions, respectively, as follows:

[0110] X:=[2, Freq, Time].

[0111] Finally, the three-dimensional tensor X is transformed into a five-dimensional tensor X_ReSize, that is:

[0112] X_Resize:=[1, 2, Freq, 1, Time].

[0113] X_ReSize is the input of the complex-valued interference neural network.

[0114] Step S2: Construct a complex-valued interferometric neural network including an interferometric convolutional layer and a frequency-domain fully connected layer.

[0115] The present invention proposes a new neural network structure, namely a complex-valued interference network, which is a key step of the present invention. The present invention is improved from the traditional convolutional neural network, and is similar to the traditional convolutional neural network in principle, but the network replaces the convolution layer of the traditional convolutional network with a frequency domain fully connected layer and an interference convolution layer to achieve the purpose of considering the time domain and frequency domain separately, and has stronger interpretability and robustness.

[0116] Therefore, this step includes the following sub-steps.

[0117] Step S201: construct a frequency domain fully connected layer.

[0118] The input data of the frequency domain fully connected layer is recorded as FcIn, which is a five-dimensional tensor expressed as:

[0119] FcIn:=[Batch,2,Freq_FcIn,FeatureMaps,Time].

[0120] Among them, Batch represents the number of samples in each batch, 2 represents the real and imaginary parts of the complex value, Freq_FcIn represents the frequency dimension of the input data of the frequency domain fully connected layer, FeatureMaps represents the number of feature maps (the dimension of the initial time-frequency map is 1), and Time represents the time dimension, that is, the number of frames of the time-frequency map.

[0121] (1) Transform the dimension of the input FcIn so that the last dimension of the transformed data FcIn_ReSize is the frequency dimension, that is:

[0122] FcIn_ReSize:=[Batch*FeatureMaps*Time,2,Freq_FcIn].

[0123] Among them, Batch*FeatureMaps*Time represents the product of the number of samples in each batch, the number of feature maps, and the number of time dimensions.

[0124] The following steps (2)-(4) are the process of FcIn_ReSize passing through the equivalent complex-valued fully connected layer.

[0125] (2) Define the fully connected layers fc_re and fc_im, which represent the real and imaginary parts of the frequency domain fully connected layer, respectively. This step can be directly implemented using the torch.nn.Linear function.

[0126] (3) Calculate the equivalent complex-valued fully connected layer output:

[0127] FcOut_re=fc_re(FcIn_ReSize[:,0,:])-fc_im(FcIn_ReSize[:,1,:]),

[0128] FcOut_im=fc_re(FcIn_ReSize[:,1,:])+fc_im(FcIn_ReSize[:,0,:]).

[0129] For ease of description, the above formula uses the notation for array slicing in Python.

[0130] (4) Concatenate the dimensions of FcOut_re and FcOut_im, and transform the concatenated tensor into a five-dimensional tensor, keeping the meaning of each dimension of the transformed tensor FcOut the same as that of the input tensor FcIn:

[0131] FcOut:=[Batch,2,Freq_FcOut,FeatureMaps,Time].

[0132] Among them, Batch represents the number of samples in each batch, 2 represents the real and imaginary dimensions of the complex-valued output, Freq_FcOut represents the frequency dimension of the output data of the frequency domain fully connected layer, FeatureMaps represents the number of feature maps, and Time represents the time dimension, that is, the number of frames of the frequency map.

[0133] Step S202: construct an interference convolution layer.

[0134] The input data of the interference convolution layer is recorded as ConvIn. ConvIn is a five-dimensional tensor expressed as:

[0135] ConvIn:=[Batch,2,Freq,FeatureMaps_ConvIn,Time].

[0136] Among them, Batch represents the number of samples in each batch, 2 represents the real and imaginary parts of the complex value, Freq represents the frequency dimension, FeatureMaps_ConvIn represents the number of input feature maps of the interference convolution layer (the dimension of the initial time-frequency map is 1), and Time represents the time dimension, that is, the number of frames of the time-frequency map.

[0137] (1) Take the real and imaginary parts of the input data as ConvIn_re and ConvIn_im, that is:

[0138] ConvIn_re=ConvIn[:,0],

[0139] ConvIn_im=ConvIn[:,1].

[0140] Perform dimension transformation on ConvIn_re and ConvIn_im respectively so that the transformed tensors are three-dimensional tensors, namely:

[0141] ConvIn_re_ReSize:=[Batch, Freq*FeatureMaps_ConvIn, Time],

[0142] ConvIn_im_ReSize:=[Batch, Freq*FeatureMaps_ConvIn, Time].

[0143] Among them, Freq*FeatureMaps_ConvIn represents the frequency dimension Freq multiplied by the number of input feature maps FeatureMaps_ConvIn of the interference convolution layer.

[0144] The following steps (2)-(4) are the process of ConvIn_re_ReSize and ConvIn_im_ReSize passing through the equivalent complex-valued interference convolution layer.

[0145] (2) Define the optimizable parameters weight_re and weight_im, the three-dimensional tensor is expressed as weight_re: = [Freq*FeatureMaps_ConvOut, FeatureMaps_ConvIn, KernelSize], weight_im: = [Freq*FeatureMaps_ConvOut, FeatureMaps_ConvIn, KernelSize].

[0146] Among them, KernelSize, FeatureMaps_ConvIn, and FeatureMaps_ConvOut represent the kernel size of the interference convolution layer, the number of input feature maps, and the number of output feature maps. Freq*FeatureMaps_ConvOut represents the frequency dimension Freq multiplied by the number of output feature maps FeatureMaps_ConvIOut of the interference convolution layer. This step can be implemented using torch.nn.Parameter. The above parameters can all be initialized using the Kaiming normal initialization method.

[0147] (3) Calculate the equivalent complex-valued convolutional layer output:

[0148] ConvOut_re=F.conv1d(ConvIn_re_ReSize, weight=self.weight_re, bias=None, groups

[0149] =Freq)-F.conv1d(ConvIn_im_ReSize, weight=self.weight_im, bias

[0150] =None, groups=Freq)

[0151] ConvOut_im=F.conv1d(ConvIn_re_ReSize, weight=self.weight_im, bias=None, groups

[0152] =Freq)+F.conv1d(ConvIn_im_ReSize,weight=self.weight_re,bias

[0153] =None, groups=Freq)

[0154] In the formula, the variables ConvIn_re_ReSize and ConvIn_im_ReSize represent the results of the dimension transformation of the real and imaginary parts of the input of the interference convolution layer, respectively. The variables ConvOut_re and ConvOut_im represent the real and imaginary parts of the output of the interference convolution layer, respectively. F.conv1d is a one-dimensional convolution operation of torch.nn.functional. Weight, bias, and groups are three parameters of the function F.conv1d, representing the weight, bias, and number of groups of the convolution layer, respectively. Self.weight_re (or self.weight_im), None, and Freq are the values ​​of the three parameters. Among them, self.weight_re and self.weight_im are weight_re and weight_im after Kaiming normal initialization, respectively; bias = None means that the F.conv1d operation does not add bias, and groups = Freq means that the number of groups is equal to the number of frequency dimensions. The number of groups equal to the number of frequency dimensions is the most critical feature that distinguishes the interference convolution layer from the ordinary convolution layer.

[0155] (4) Concatenate the dimensions of ConvOut_re and ConvOut_im, and transform the concatenated tensor into a five-dimensional tensor, keeping the meaning of each dimension of the transformed tensor ConvOut the same as that of the input tensor ConvIn, that is:

[0156] ConvOut:=[Batch, 2, Freq, FeatureMaps_ConvOut, Time].

[0157] Among them, Batch represents the number of samples in each batch, 2 represents the real and imaginary dimensions of the complex-valued output, Freq represents the number of frequency dimensions, FeatureMaps_ConvOut represents the number of feature maps of the output data of the interference convolution layer, and Time represents the number of time dimensions, that is, the number of frames of the frequency map.

[0158] Step S203: construct a complete complex-valued interference neural network.

[0159] The construction method is roughly as follows:

[0160] (1) Repeatedly stack the frequency domain fully connected layer and the interference convolution layer several times to increase the network depth. The complex-valued interference network is mainly characterized by alternately stacking the frequency domain fully connected layer and the interference convolution layer. However, there are many specific ways to connect the frequency domain fully connected layer and the interference convolution layer. Figure 2 Two possible connection methods are given. (a) is a direct connection type, that is, the frequency domain fully connected layer and the interference convolution layer are directly connected alternately. This connection method is relatively simple to program, but this approach is usually only applicable to the case of a small number of network layers. As the number of network layers increases, the numerical instability of the calculation usually leads to optimization difficulties; (b) is a dense connection type, that is, the outputs of the interference convolution layers of different layers are finally connected through statistical pooling layers and dimensional splicing. In the case of a deeper network, the dense connection type is usually easier to optimize than the direct connection type.

[0161] The number of repetitions (i.e., the number of network layers) can be set freely (e.g., the number of repetitions is 5). At each repetition, the frequency domain fully connected layer and the interference convolution layer can be swapped, and activation layers and batch normalization layers can be added. The parameters of each frequency domain fully connected layer can be adjusted (e.g., the Freq_FcOut parameters of the 5 frequency domain fully connected layers are set to 256, 128, 64, 32, and 16, respectively), and the parameters of each interference convolution layer can also be set flexibly (e.g., the FeatureMaps_ConvOut parameter values ​​of the 5 interference convolution layers are 64, 64, 64, 128, and 128, respectively, and the KernelSize parameter values ​​are 3, 3, 3, 3, and 1, respectively).

[0162] (2) Access the statistical pooling layer.

[0163] The input PoolingIn of this layer is the following five-dimensional tensor,

[0164] PoolingIn:=[Batch, 2, Freq, FeatureMaps, Time].

[0165] Among them, Batch represents the number of samples in each batch, 2 represents the two dimensions of real and imaginary parts, Freq represents the number of frequency dimensions, FeatureMaps represents the number of feature maps, and Time represents the number of time dimensions, that is, the number of frames of the frequency map.

[0166] The output of the statistical pooling layer can be used as the time series signal feature extracted by the network, which is a two-dimensional tensor, namely:

[0167] Feature:=[Batch, FeatureMaps].

[0168] In this layer, the total signal energy at each moment is calculated along the frequency axis of the data, and then the average value is calculated along the time axis of the data to obtain the characteristics of each time series signal sample. The specific calculation formula is:

[0169]

[0170] The above formula uses the notation for array element indexing in Python, that is, Feature[i,n] represents the value of the two-dimensional tensor Feature when the first dimension is i and the second dimension is n, In[i,0,k,n,m] and In[i,1,k,n,m] represent the values ​​of the input tensor In when the dimensions are i,0,k,n,m and i,1,k,n,m respectively.

[0171] (3) The final two ordinary fully connected layers are used for feature transformation and output the probability that the sample belongs to each category, that is: Out: = [Batch, Num_Classes].

[0172] Among them, Out represents the final output of the complex-valued interference network, Batch represents the number of samples in each batch, and Num_Classes represents the number of communication terminals to be classified.

[0173] Step S3: Calculate the classification loss function and perform complex-valued interference neural network training based on the gradient descent method.

[0174] First, the network input complex-valued time-frequency graph data is passed through the complex-valued interference neural network layer by layer to obtain its output. Based on its output, the loss function of the network output relative to the true label of the sample can be calculated. For example, the cross entropy loss function:

[0175]

[0176] In the above formula, L represents the value of the loss function, N is the number of samples in each batch, and x i represents the input of the last fully connected layer, y i represents the true label of the sample, W, and b represents the weight parameter and bias parameter to be optimized in the last fully connected layer.

[0177] Then, the automatic differentiation mechanism in torch is used to calculate the gradient of the loss function relative to each parameter to be optimized in the complex-valued interference network, and the gradient descent method is used to update each parameter.

[0178]

[0179] Among them, θ is any parameter to be optimized in the complex-valued interference network, θ′ is the value of the parameter to be optimized θ after gradient descent update, It is the gradient of the loss function with respect to each parameter to be optimized in the complex-valued interference network. η is the learning rate, which is a hyperparameter and can be adjusted, such as 5e-4.

[0180] Finally, the above loss function calculation and gradient descent process are repeated several times until convergence.

[0181] Step S4: Complex-valued interference neural network model deployment and testing.

[0182] First, according to the method described in step S1, the complex-valued time-frequency diagram data to be tested is obtained from the time series signal to be tested. The complex-valued time-frequency diagram data to be tested uses the complex-valued interference neural network obtained in step S3 to obtain the probability that the time series signal to be tested belongs to each category. Finally, the category corresponding to the maximum probability is taken as the final prediction result of the complex-valued interference neural network.

[0183] like Figure 3 As shown in the figure, (a) and (b) are the dimensionality reduction distribution diagrams of the features extracted by the traditional complex-valued ResNet network and the complex-valued interference neural network, respectively. The dimensionality reduction method used is t-SNE. Figure 3 As can be seen from (a) and (b) in the figure, no matter it is a complex-valued ResNet network or a complex-valued interference network, the features of the same signal basically show the characteristics of clustered distribution; however, the intra-class distribution of the features learned by the complex-valued interference neural network of the present invention is more compact and regular than that of the traditional complex-valued ResNet network.

[0184] Figure 4 This is a bar chart of the signal recognition accuracy of the traditional complex-valued ResNet network and the complex-valued interference neural network. Figure 4 It can be seen that when the signal-to-noise ratio of the test set time series signal is equal to the signal-to-noise ratio of the training set (20dB), the recognition accuracy of the complex-valued interference network is slightly higher than that of the complex-valued residual; when the signal-to-noise ratio of the test set time series signal is lower than the signal-to-noise ratio of the training set (20dB), the recognition accuracy of the complex-valued residual network will drop sharply. Under the 10dB test environment, the recognition rate drops to 70.37%, while the interference network still has an accuracy rate of more than 80%. The above results show that compared with the traditional complex-valued ResNet network, the complex-valued interference network has stronger noise robustness.

[0185] The above is only an embodiment of the present invention in the application context of radio frequency fingerprint recognition of communication signals. The present invention is not limited to the above embodiments. The description of the present invention is for illustration and does not limit the scope of the claims. For those skilled in the art, it is obvious that there can be many substitutions, improvements and changes. Any technical solution formed by equivalent substitution or equivalent transformation falls within the scope of protection required by the present invention.

Claims

1. A method for identifying time series signals based on complex-valued interference neural network, characterized in that: The following steps are involved: The first step is to preprocess the time series signal and calculate the complex value time-frequency diagram of the time series signal, wherein the time series signal includes the fingerprint identification radio frequency signal; The second step is to construct a complex-valued interference neural network, which includes a plurality of interference convolution layers and a plurality of frequency domain fully connected layers, which are alternately connected in sequence to extract feature vectors and perform feature transformation, wherein the interference convolution layer is used to perform a complex-valued convolution operation in the time dimension on the complex-valued time-frequency graph to simulate the interference process of the wave, and the frequency domain fully connected layer is used to perform a complex-valued fully connected operation in the frequency dimension on the complex-valued time-frequency graph to simulate the nonlinear process of the wave; The third step is to calculate the classification loss function and train the complex-valued interference neural network based on the gradient descent method to obtain a trained complex-valued interference neural network model; The fourth step is to obtain the complex-valued time-frequency diagram data to be tested from the fingerprint identification radio frequency signal to be tested according to the first step, input the complex-valued time-frequency diagram data to be tested into the trained complex-valued interference neural network model, output the probability that the fingerprint identification radio frequency signal to be tested belongs to each transmitting terminal, and take the transmitting terminal corresponding to the maximum probability as the final prediction result.

2. The method for identifying time series signals based on complex-valued interference neural network according to claim 1, characterized in that: In the second step, the frequency domain fully connected layer includes: performing a dimension transformation on the input five-dimensional tensor so that the last dimension of the transformed five-dimensional tensor is the frequency dimension; defining two fully connected layers, which are equivalent to a complex-valued fully connected layer; calculating the real tensor and the imaginary tensor of the equivalent complex-valued fully connected layer output tensor based on the dimensionally transformed tensor and the two fully connected layers; performing dimension splicing on the real tensor and the imaginary tensor of the equivalent complex-valued fully connected layer output tensor, and transforming the spliced ​​tensor into a five-dimensional tensor with the same meaning as the input tensor, and inputting it into the interference convolution layer as the final output result of the frequency domain fully connected layer.

3. The method for identifying time series signals based on complex-valued interference neural network according to claim 1, characterized in that: In the second step, the interference convolution layer includes: taking the real tensor and the imaginary tensor of the final output result of the frequency domain fully connected layer input to the interference convolution layer, and performing dimension transformation on the real tensor and the imaginary tensor of the final output result of the frequency domain fully connected layer, so that the last dimension of the transformed five-dimensional tensor is the time dimension; defining two convolution layers, the two convolution layers are equivalent to a complex-valued convolution layer, wherein the weight parameters of the two convolution layers are initialized using the Kaiming normal initialization method; according to the tensor after dimension transformation and the two convolution layers, calculating the real tensor and the imaginary tensor of the equivalent complex-valued convolution layer output tensor; dimensionally splicing the real tensor and the imaginary tensor of the equivalent complex-valued convolution layer output tensor, and transforming the spliced ​​tensor into a five-dimensional tensor with the same meaning as the input tensor as the final output result of the interference convolution layer.

4. The method for identifying time series signals based on complex-valued interference neural network according to claim 1, characterized in that: The complex-valued interference neural network includes alternately stacking frequency domain fully connected layers and interference convolution layers for several times; a statistical pooling layer is connected after each interference convolution layer to convert complex-valued data into real-valued data and obtain the feature vector of each fingerprint identification radio frequency signal sample; the results of several statistical pooling layers are spliced ​​and connected to two layers of ordinary fully connected layers for feature transformation, and the probability that each fingerprint identification radio frequency signal sample belongs to each transmitting terminal is output.

5. The method for identifying time series signals based on complex-valued interference neural network according to claim 4, characterized in that: In the complex-valued interference neural network, when frequency-domain fully connected layers and interference convolutional layers are alternately stacked, the stacking is performed using a skip-layer connection or a dense connection method in a ResNet and DenseNet network architecture.

Images