Electromagnetic signal identification method based on selective knowledge transfer learning

Abstract

The invention discloses an electromagnetic signal identification method based on selective knowledge transfer learning. The method comprises the following steps: firstly, preprocessing collected electromagnetic signal data, and randomly dividing the data into data of a source domain and a target domain; and then training a network model for feature extraction. And designing a teacher model and a student model, inputting the extracted signal features into the teacher model, performing singular value decomposition on the training set, performing coefficient screening according to singular values, performing feature representation as a target domain, and obtaining a migrated model. Performing knowledge distillation on the output data of the teacher model, and calculating knowledge difference between the output data of the teacher model and the output data of the student model, distillation loss and cross entropy loss; and finally, network parameters of the student model are updated according to the distillation loss, a trained student model is obtained, and classification and identification of the electromagnetic signal data are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of signal processing, and in particular relates to an electromagnetic signal recognition method based on selective knowledge transfer learning. Background Art

[0002] With the rapid development of information technology and network technology, the methods and standards of electromagnetic signal classification are constantly updated and improved to meet the growing demand for signal processing. Electromagnetic signal classification is an important technology for electromagnetic environment situational awareness, but in practical applications, it is usually faced with the problem of insufficient labeled sample data. To address this problem, studying the electromagnetic signal classification method under small sample conditions has important theoretical significance and practical value.

[0003] Traditional machine learning methods are usually limited to solving problems in a single field. However, when the feature distributions of training data and test data are different, it is often necessary to retrain the model on the new data set, which will lead to data waste and reduced efficiency. In practical applications, data in many fields are difficult to obtain or label, such as medicine, biology, and security. Therefore, transfer learning methods have emerged to help learning problems in target fields with sparse or missing labeled samples by leveraging useful knowledge learned from the source field.

[0004] Transfer learning assumes that there are some common factors between different fields, such as features, concepts, and rules, which can be transferred and shared in some way to improve the effect and efficiency of learning. In recent years, with the development and maturity of transfer learning theory, it has provided new ideas and solutions to solve problems and challenges in fields such as object recognition and detection. It has achieved remarkable results in computer vision, natural language processing, recommendation systems, and other fields, and has also attracted widespread attention and research from academia and industry.

[0005] However, transfer learning is not a panacea, and it also faces some challenges and problems. For example, if the tasks in the source domain and the target domain are very different, then transfer learning will have a negative impact, a phenomenon known as "negative transfer". In addition, transfer learning also has problems such as overfitting, class imbalance, and cross-domain feature distribution differences. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes an electromagnetic signal recognition method based on selective knowledge transfer learning. According to the characteristics of the target task, the module parameters are selectively transferred, and knowledge distillation is used to reduce the calculation and storage overhead of the model.

[0007] A method for electromagnetic signal recognition based on selective knowledge transfer learning, the specific steps are as follows: Step 1: Collect electromagnetic signal data and select electromagnetic signals with a signal-to-noise ratio of 6 dB as samples. Use random sampling to divide the samples into source domain signals and target domain signals. Finally, divide the samples into training sets and test sets in a ratio of 8:2.

[0008] Step 2: Use the training set data in step 1 to train a MsmcNet neural network for feature extraction and save the optimal model parameters.

[0009] Step 3: Design two deep neural networks with the same structure, including three cascaded fully connected layers, as the teacher model and student model respectively.

[0010] Step 4: Use the MsmcNet neural network trained in step 2 to extract the features of the input electromagnetic signal, then input it into the teacher model for classification, and train and optimize the teacher model through stochastic gradient descent.

[0011] Step 5: Perform singular value decomposition on the training set data and transform the matrix The size of the elements in is used as the weight of the coefficient to screen the coefficient. The MsmcNet neural network trained in step 2 is used to extract the features of the input electromagnetic signal, which are input into the trained teacher model and student model respectively. The outputs of the teacher model and the student model are compared, and the distillation loss is calculated.

[0012] Step 6: Use the weighted sum of the distillation loss and the cross entropy loss as the total loss function to guide the update of the student model network parameters and complete the training of the student model. Use the trained MsmcNet neural network and the student model to extract and classify the electromagnetic signals in the data set.

[0013] The present invention has the following beneficial effects: 1. Use singular value decomposition to select the most useful weights from the teacher model so that the student model can highlight the most relevant features and improve accuracy; at the same time, filter out useless weights to improve the robustness of the model.

[0014] 2. Letting the student network learn the output distribution of the teacher network instead of directly learning the labels of the data can reduce the model's overhead and resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of the electromagnetic signal recognition method based on selective knowledge transfer learning; Figure 2 Schematic diagram of the MsmcNet neural network structure. DETAILED DESCRIPTION

[0016] The present invention will be further explained below with reference to the accompanying drawings; like Figure 1 As shown, the electromagnetic signal recognition method based on selective knowledge transfer learning specifically includes the following steps: Step 1. This embodiment directly uses electromagnetic signal data from the general data sets RML2016.04a and RML2016.10c. Both data sets contain signal samples of 11 different modulation modes, each sample contains two dimensions of real part and imaginary part, and the length of each dimension is 128 data points, which can provide a rich data basis for in-depth analysis.

[0017] Electromagnetic signals with a signal-to-noise ratio of 6dB were selected from the data set as samples. Six modulation signals were selected as source domain data using a random sampling method, and the remaining five modulation signals were divided into target domain data. In order to ensure the balance of sample distribution, for signals with different modulation modes under the same signal-to-noise ratio, the samples were divided into training sets and test sets in a ratio of 8:2.

[0018] Step 2: Build a Figure 2 The MsmcNet neural network shown in the figure includes 4 identical blocks, 1 normalization layer, and 1 fully connected layer. Each block includes a 2D convolutional layer of size 5×2, a 2D batch normalization layer, a ReLU activation function, and a 2D maximum pooling layer. Among them, the 2D batch normalization layer is used to improve the training speed and model generalization ability.

[0019] The training set and test set data are loaded through the loadNpy function, which returns four arrays: x_train, y_train, x_test, and y_test. Among them, x_train and x_test represent the signal data of the training set and test set respectively, while y_train and y_test correspond to the label data of the training set and test set respectively; construct variables named train_dataset and valid_dataset, encapsulate the training and test data and their labels into two dataset objects, and convert these objects into iterable data loaders to obtain data in batch order; input the training set data and test set data to the MsmcNet neural network, and save the optimal model parameters.

[0020] Step 3: Design two deep neural networks with the same structure, including three cascaded fully connected layers. ReLU activation function is used between layers to enhance nonlinear expression ability. The two deep neural networks with the same structure are used as teacher model and student model respectively.

[0021] Step 4: Use the MsmcNet neural network trained in step 2 to extract the features of the input electromagnetic signal, then input it into the teacher model for classification, set the learning rate and momentum value, and train and optimize the teacher model through stochastic gradient descent until the model converges.

[0022] Step 5: Obtain the matrix by performing singular value decomposition on the training set data .in It is a diagonal matrix. The elements on the diagonal are used as the weights of the coefficients. They are arranged in descending order according to the size of the elements. The first 10% of the coefficients are used as the feature representation of the target domain, and the last 10% of the coefficients are discarded to build a transfer learning model. The features of the input electromagnetic signal are extracted using the MsmcNet neural network trained in step 2, and input into the trained teacher model and student model respectively. The output of the teacher model is used as the "soft target", that is, the probability value of each category, as the knowledge output by the teacher model. The output of the student model is used as the "hard target", that is, the category corresponding to the maximum probability. Compare the KL divergence of the output of the teacher model and the student model as the distillation loss to measure the difference between the output of the student network and the output of the teacher network.

[0023] Step 6: Take the weighted sum of the distillation loss and the cross entropy loss as the total loss, back-propagate and update the network parameters of the student model, and finally obtain the trained student model. Use the final model to classify electromagnetic signals on the test set.

Claims

1. An electromagnetic signal recognition method based on selective knowledge transfer learning, characterized in that: The specific steps include: Step 1: Collect electromagnetic signal data as samples, and use random sampling to divide the samples into source domain signals and target domains; finally, randomly divide the samples into training sets and test sets; Step 2: Use the training set data in step 1 to train a neural network for feature extraction and save the optimal model parameters; Step 3: Design two deep neural networks with the same structure, including three cascaded fully connected layers, as the teacher model and the student model respectively; Step 4: Use the feature extraction neural network trained in step 2 to extract the features of the input electromagnetic signal, then input it into the teacher model for classification, and train and optimize the teacher model; Step 5: Perform singular value decomposition on the training set data and transform the matrix The size of the elements in is used as the weight of the coefficient to screen the coefficient; the feature extraction neural network trained in step 2 is used to extract the features of the input electromagnetic signal, and the features are input into the trained teacher model and student model respectively, the outputs of the teacher model and the student model are compared, and the distillation loss is calculated; Step 6: Use the weighted sum of the distillation loss and the cross entropy loss as the total loss function to guide the update of the student model network parameters and complete the training of the student model; use the trained feature extraction neural network and the student model to extract and classify the electromagnetic signals in the data set.

2. The electromagnetic signal recognition method based on selective knowledge transfer learning as claimed in claim 1, characterized in that: Electromagnetic signals with a signal-to-noise ratio of 6 dB were selected as samples, and the samples were divided into training sets and test sets in a ratio of 8:

2.

3. The electromagnetic signal recognition method based on selective knowledge transfer learning as claimed in claim 1, characterized in that: Construct a MsmcNet neural network for feature extraction.

4. The electromagnetic signal recognition method based on selective knowledge transfer learning as claimed in claim 3, characterized in that: The MsmcNet neural network includes 4 identical blocks, 1 normalization layer and 1 fully connected layer; each block includes a 2D convolutional layer of size 5×2, a 2D batch normalization layer, a ReLU activation function and a 2D maximum pooling layer.

5. The electromagnetic signal recognition method based on selective knowledge transfer learning as claimed in claim 3, characterized in that: The training set data is input into the MsmcNet neural network for model training, and the test set data is used to test the model performance and save the optimal model parameters.

6. The electromagnetic signal recognition method based on selective knowledge transfer learning as claimed in claim 1, characterized in that: The ReLU activation function is used between each linear layer of the teacher model and the student model to enhance the nonlinear expression ability.

7. The electromagnetic signal recognition method based on selective knowledge transfer learning as claimed in claim 1, characterized in that: The teacher model is trained and optimized by stochastic gradient descent.

8. The electromagnetic signal recognition method based on selective knowledge transfer learning as claimed in claim 1, characterized in that: Perform singular value decomposition on the training set data to obtain the matrix ; The matrix The elements on the diagonal are used as the weights of the coefficients and are arranged in descending order according to the size of the element values. The first 10% of the coefficients are used as the feature representation of the target domain, and the last 10% of the coefficients are discarded to construct a transfer learning model.

9. The electromagnetic signal recognition method based on selective knowledge transfer learning as claimed in claim 1, characterized in that: The teacher model outputs the probability value of each category, and the student model outputs the category corresponding to the maximum probability; the KL divergence between the teacher model output and the student model output is compared as the distillation loss to measure the difference between the student network output and the teacher network output.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 9.

Images