Method for individual identification of electromagnetic signals using open set semi-supervised learning technique

Abstract

The present application relates to a method for individual identification of electromagnetic signals using open set semi-supervised learning technology, belonging to the field of electromagnetic signal identification processing. The technical scheme of the present application trains a large classification network as a first model through self-supervised pre-training, so that it can more fully learn the internal relationship of unlabeled data and improve the network feature extraction capability; a small-scale classification network model closer to the characteristics of the identification sample is used as a second model, and in semi-supervised learning, the feature extraction capability of the pre-trained model is used to filter out unlabeled data other than the target classification, reducing the negative impact of out-of-distribution unlabeled data, and at the same time, model distillation is used to enable the small-scale classification network model to learn the generalization and robustness of the large network model for data features. Through the above three aspects, the influence of data other than the target classification on the semi-supervised learning network recognition accuracy in real situations is reduced, and robust identification of electromagnetic signals is achieved.

Description

Technical Field

[0001] This invention relates to electromagnetic signal recognition and processing, and more particularly to a method for individual electromagnetic signal recognition using open-set semi-supervised learning techniques. Background Technology

[0002] In the field of wireless communication, a large amount of useful information can be obtained by identifying various electromagnetic signals. However, due to the widespread use of different types of wireless devices, the electromagnetic environment has become increasingly complex (wide frequency domain coverage, long time domain duration, variable spatial distribution, and diverse signal characteristics). Traditional electromagnetic signal identification methods are not good at expressing complex signals and cannot effectively identify and classify different electromagnetic signals.

[0003] Leveraging deep learning or machine learning techniques can effectively address the difficulty of using traditional communication methods to process signal samples and gain a comprehensive understanding of the electromagnetic situation. However, limitations and contradictions still exist in utilizing deep learning for big data processing and deep sensing. In some military or security applications, due to non-cooperation and security concerns, only a very limited amount of sample data can be obtained. This leads to the problem of small sample size for labeled data, meaning that the number of labeled samples is very small, making it difficult to train a model with high generalization ability.

[0004] Semi-supervised learning is a learning method that uses a large number of unlabeled samples and a small number of labeled samples to train a model. This method can effectively solve the problem of small labeled samples and improve the model's recognition performance. However, since existing semi-supervised learning methods are all based on an ideal scenario, that is, all samples in the unlabeled data and the labeled data are ideally identically distributed, when data outside the target classification is mixed with unlabeled data and used together to train the classification model, the resulting machine learning model still does not have the robustness and reliability required for real-world applications. Summary of the Invention

[0005] To address the problem that existing technologies cannot robustly identify electromagnetic signals using semi-supervised learning in real-world scenarios, this invention proposes a method for individual electromagnetic signal identification using open-set semi-supervised learning. A large-scale classification network, used as the first model, is trained through self-supervised pre-training, allowing the larger model to more fully learn the intrinsic relationships in unlabeled data and improve its feature extraction capabilities. A smaller-scale classification network model, more closely resembling the characteristics of the identified samples, serves as the second model. During semi-supervised learning, unlabeled data outside the target category is filtered out, resulting in a more compact feature distribution among similar samples in the training data. Simultaneously, model distillation is incorporated into the semi-supervised learning process, enabling the smaller-scale classification network model to learn the generalization and robustness of the larger network model for data features. By addressing these three aspects simultaneously, the impact of data outside the target category on the accuracy of the semi-supervised learning network in real-world situations is reduced, achieving robust identification of electromagnetic signals. The specific technical solution is as follows:

[0006] A method for individual electromagnetic signal recognition using open-set semi-supervised learning technology, the method comprising the following steps:

[0007] Electromagnetic signal samples are obtained as the first training dataset. The first training dataset is an open set, consisting of labeled data and unlabeled data. The unlabeled data consists of unlabeled data within the target category and unlabeled data outside the target category.

[0008] The first model was trained using the SimCLR self-supervised learning algorithm on the first training dataset.

[0009] The first trained model is used to filter out unlabeled data outside the target category, and the unlabeled data and labeled data within the target category are used as the second training dataset.

[0010] Freeze the weights of the trained first model, add a soft label generation layer to the first model to form a third model, and input labeled data into the third model for fine-tuning training.

[0011] The trained third model is used to obtain the soft label of each data in the first training dataset. The labeled data with soft labels, the unlabeled data within the target category, and the unlabeled data outside the target category that meet the set conditions are used as the third training dataset.

[0012] Using the third training dataset, the second model learns the feature representation of the third model through model distillation, thus obtaining the distillation loss. The second training dataset is used to perform semi-supervised learning on the second model, and the semi-supervised learning loss is obtained. ;

[0013] Distillation loss and semi-supervised learning loss Weighted summation as loss error The second model is trained, and the trained second model can be used for individual electromagnetic signal recognition; wherein:

[0014]

[0015] In the formula: WarmUp is a linear auto-increment value.

[0016] In the above technical solution, the first model includes an encoder and a mapping end. The mapping end is used to assist the encoder in training convergence. The encoder consists of an input layer and a feature extraction layer, and the mapping end consists of a nonlinear fully connected network. The feature vector output by the feature extraction layer is input to the mapping end to obtain the data representation used by the self-supervised learning algorithm to calculate the loss.

[0017] In the above technical solution, the first model is trained using the SimCLR self-supervised learning algorithm on the first training dataset, including the following steps:

[0018] Data augmentation is performed on each data point in the first training dataset. Each data augmentation operation includes random flipping and random rotation of the complex domain, resulting in two sets of augmented data. x i1 and x N+i1 ;

[0019] For each data point in the first training dataset, perform data augmentation on two sets of data. x i1 and x N+i1 Input the first model to obtain two sets of nonlinear features. z i1 and z N+i1 Calculate the cosine similarity between these two sets of nonlinear features. , ;

[0020] Based on cosine similarity , Calculate the learning loss and Then, the average loss is calculated as the final loss error of the model's backpropagation. :

[0021]

[0022]

[0023]

[0024] In the formula: For temperature hyperparameters, This represents the total number of data points in the first training dataset.

[0025] In the above technical solution, the trained first model is used to filter out unlabeled data other than the target classification. The implementation steps include:

[0026] The feature vector is obtained by inputting labeled data into the first trained model.

[0027] By using feature vectors belonging to the same category, we can obtain the prototypical representations for each category. :

[0028]

[0029] In the formula: Indicates the number of categories of the identified target. This represents the number of data points belonging to category c of electromagnetic signals, where c is the category identifier. For the first k The feature vector corresponding to an electromagnetic signal belonging to category c;

[0030] Each unlabeled data point is input into the first trained model to obtain the corresponding feature vector. Calculate its cosine similarity with each category:

[0031] ,

[0032] Based on cosine similarity, the unlabeled dataset is divided into unlabeled data within the target category and unlabeled data outside the target category. In one implementation, the following partitioning principle is adopted:

[0033] Those above or equal to the threshold Unlabeled data is determined to be within the target category; otherwise, it is considered unlabeled data outside the target category. (Threshold) Set to:

[0034]

[0035] in: The mean of the cosine similarity scores for labeled data. The standard deviation of the cosine similarity scores for labeled data. This is a hyperparameter.

[0036] In the above technical solution, the soft label generation layer is a fully connected layer, and labeled data is input into the third model for fine-tuning training; the fine-tuning training is to optimize the parameters of the fully connected layer without changing the parameter values ​​of the trained first model during training, until the third model converges.

[0037] In the above technical solution, the set condition is that the confidence score is greater than or equal to the confidence threshold; the confidence score is the maximum value of the probability of each category corresponding to an electromagnetic signal data when it is input into the third model; the average confidence score of the labeled data is set as the confidence threshold.

[0038] In one implementation, distillation loss The calculation is as follows:

[0039]

[0040]

[0041] In the formula:

[0042] For calculation and The mean square error loss, The second model is based on the category label y on the input data. The predicted results obtained The third model is based on the category label y on the input data. The predicted results obtained; Calculate the mean of the mean squared error loss for all data.

[0043] In one implementation, semi-supervised learning loss The calculation steps include:

[0044] For each unlabeled data point, perform K data augmentation operations. Each data augmentation operation includes random flipping and random rotation of the complex field, resulting in K sets of augmented data.

[0045] For each unlabeled data point, its corresponding K augmented data sets are sequentially input into the second model, and the probabilities corresponding to the K predicted results are output. This allows us to obtain the consistency probability distribution corresponding to each unlabeled data point. :

[0046]

[0047] probability distribution Normalization using a power function yields the prediction results. The type of electromagnetic signal corresponding to the prediction result is used as a pseudo-label:

[0048] ,

[0049] In the formula: n is the total number of electromagnetic signal categories, T is a hyperparameter, and is the temperature value normalized by the power function;

[0050] Labeled data and unlabeled data with pseudo-labels are concatenated and rearranged to form mixed data. After that Take from front to back Data of the same length and Perform a mixing operation, and the resulting data will be used as new labeled data. Then The remaining data and the data with pseudo-labels Perform a mixing operation to obtain new pseudo-label data. ;

[0051] Using the obtained labeled data and pseudo-label data Calculate supervised loss separately and unsupervised loss The weighted sum of the two results in the final semi-supervised learning loss error. :

[0052]

[0053] In the formula: WarmUp is a linear auto-increment value.

[0054] In the above technical solution, one implementation of the hybrid operation is: combining a pair of data with labels or pseudo-labels. and Enhanced samples are obtained by mixing in the following manner. ,in:

[0055]

[0056]

[0057] In the formula:

[0058] , Electromagnetic signal sample, , For tags or pseudo tags, Through hyperparameters The random parameters that conform to the beta distribution are generated as follows:

[0059]

[0060] Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart illustrating an implementation of an electromagnetic signal individual identification method using open-set semi-supervised learning technology.

[0063] Figure 2 This is a flowchart of the SimCLR self-supervised learning process in one implementation.

[0064] Figure 3 This is a flowchart of the semi-supervised learning data augmentation and pseudo-label prediction algorithm in one implementation method.

[0065] Figure 4 This is a flowchart of the algorithm for the semi-supervised learning stochastic mixture and loss function calculation in one implementation method;

[0066] Figure 5 This is a flowchart of one implementation method;

[0067] Figure 6 This is a test accuracy curve of a trained model in one implementation and a test accuracy curve of a model trained using a general algorithm.

[0068] Figure 7 This is a test accuracy curve of the trained model and the trained model using a comparison method in the same domain, as shown in one implementation. Detailed Implementation

[0069] The terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0070] Operations in a flowchart do not have to be performed in sequence. Instead, they can be performed in reverse order or simultaneously. Furthermore, one or more additional operations can be added to the flowchart. One or more operations can be removed from the flowchart.

[0071] Significant progress has been made in semi-supervised learning techniques that train classification models using a small amount of labeled data and a large amount of unlabeled data. Modern semi-supervised learning methods typically assume that labeled and unlabeled data have the same class distribution. However, in reality, unlabeled data often exhibits an open-set paradigm, meaning the dataset may contain unknown samples outside the target class. When out-of-class samples are mixed with unlabeled data and used to train a classification model, the model's recognition accuracy is severely affected by this type of data because existing semi-supervised learning techniques lack robustness in handling open-set unlabeled data. This invention effectively alleviates the problem of decreased recognition accuracy caused by out-of-class samples in semi-supervised learning methods, enabling robust learning in open-set scenarios and improving the robustness and reliability of semi-supervised learning in real-world situations.

[0072] To address the aforementioned technical problems, one embodiment employs a technical solution for individual electromagnetic signal recognition using open-set semi-supervised learning technology. Combined with... Figure 1 As shown, the method includes the following steps:

[0073] S100. Obtain electromagnetic signal samples as the first training dataset. The first training dataset is an open set, consisting of labeled data and unlabeled data. The unlabeled data consists of unlabeled data within the target category and unlabeled data outside the target category.

[0074] In one implementation, electromagnetic spectrum detection equipment is used to acquire radio frequency signal data of 11 categories of UAVs. The data consists of a small amount of labeled data, a large amount of unlabeled data (including unlabeled data outside the target classification), and a small amount of labeled test data. "Small amount" and "large amount" are not limitations on quantity; they are relative concepts, and the specific quantity does not affect the method proposed in this invention.

[0075] The sampling rate of the electromagnetic spectrum acquisition equipment is 200KHz, the signal-to-noise ratio range of the acquisition environment is 0~20dB, the carrier frequency is 1090MHz, the acquired radio frequency signal is represented as IQ signal, the total data volume is 44900 groups, and the IQ signal data format is 2×128 (128 sampling points each for I channel and Q channel).

[0076] Six signal categories were used as recognition targets, totaling 24,900 sets; five signal categories were used as OOD (out-of-target, unlabeled data) data, totaling 20,000 sets. Of the six target categories: 24,000 sets were unlabeled data; and 900 sets were labeled data, of which 300 sets were used as training data and the remaining 600 sets as test data. All unlabeled data outside the target categories were used as training data, totaling 44,000 sets (24,000 sets of unlabeled data for target recognition and 20,000 sets of unlabeled data outside the target categories).

[0077] S102. Using the first training dataset, train the first model using the SimCLR self-supervised learning algorithm.

[0078] In one implementation: the first model is constructed as follows: first, a common neural network classification model is constructed, and the parameters such as the convolution kernel and stride in the classification model are modified to adapt to electromagnetic signal data. The classification model is then pruned, removing the fully connected layer (Dense) used for classification, and retaining the input layer and feature extraction layer as the encoder in self-supervised learning. After the feature extraction layer, a nonlinear mapping terminal is added to assist the model convergence.

[0079] The processing procedure is illustrated using an encoder as an example: The input layer augments the training data to 1×2×128 dimensions. The feature extraction layer then downsamples the augmented data to 16×1×128 after passing it through a convolutional layer. The convolutional kernels in the convolutional layer are 2×15 in size. After batch normalization and ReLU activation, the data is downsampled to 16×1×64 through a max pooling layer with a pooling size of 1×2 and a stride of 1×2. The data is then fed into four residual blocks with identical parameters. The number of residual block layers are [3, 4, 6, 3]. The data structures of all residual modules are upsampled to 32×1×32, 64×1×16, 128×1×8, and 256×1×4 respectively. The residual modules contain the following in sequence: (1) convolutional layer with a kernel size of 1×1; (2) batch normalization layer; (3) convolutional layer with a kernel size of 1×3; (4) convolutional layer with a kernel size of 1×1; (5) batch normalization layer; (6) downsampling layer (downsampled and summed with the input of the residual module); (7) ReLU activation layer. After passing through four residual modules, the data is downsampled to a one-dimensional feature vector of length 256 by a two-dimensional global average pooling layer (GlobalAveragePooling2D).

[0080] The processing procedure is illustrated using the mapping end as an example: the mapping end consists of a non-linear fully connected network, including two densely connected layers with 256×256 input / output channels. Between the two layers is a ReLU activation layer, used to provide non-linear features to the mapping end. Inputting the 256-length feature vector output from the feature extractor into this mapping end yields the final data representation used by the self-supervised learning algorithm to calculate the loss.

[0081] The encoder and mapping end training algorithm is the self-supervised learning SimCLR algorithm. This algorithm borrows the idea of ​​contrastive learning, aiming to encourage the model to learn how to distinguish between similar and dissimilar data. The overall algorithm process consists of four steps. The following section will combine... Figure 2 To elaborate further.

[0082] S102.1 Perform data augmentation on each data point in the first training dataset. Each data augmentation process includes random flipping and random rotation of the complex domain, resulting in two sets of augmented data. x i1 and x N+i1 .

[0083] In step S102.1 above, perturbation is added to the original data using random data augmentation. The data augmentation method is two methods designed by this invention for electromagnetic signal characteristics: random flipping of the complex domain and random rotation of the complex domain.

[0084] Complex domain random flipping involves mapping the sample points of the electromagnetic signal to complex space before performing the flipping operation. For a given electromagnetic signal data... In this invention, the operation of inverting the I-channel data is defined as a horizontal flip operation, and the operation of inverting the Q-channel data is defined as a vertical flip operation. "Random" refers to randomly selecting the horizontal and vertical flip operations. The following example shows an amplified signal sample obtained after performing a horizontal flip. :

[0085]

[0086] Complex-domain random rotation requires first mapping the IQ data to the complex domain, and then, based on the distribution of sample points in the complex domain, rotating them clockwise or counterclockwise around the origin to enhance the complex-domain rotation of the electromagnetic IQ data. This involves rotating the electromagnetic signal samples around the origin in the complex domain. By performing a rotation operation, an amplified signal sample can be obtained. as follows:

[0087]

[0088] In the above formula, For the rotation angle, in one embodiment, the rotation angle for electromagnetic signal enhancement is set to {0,} , , The random selection of}.

[0089] Remove the test cases from all the data collected in step 101, and perform two sets of data augmentation on all training data (labeled data refers to the data input to the model), so that one training data can be used to obtain two sets of perturbated samples. x i1 and x N+i1 .

[0090] S102.2 For each data point in the first training dataset, perform data augmentation on both sets of data. x i1 and x N+i1 Input the first model to obtain two sets of nonlinear features. z i1 and z N+i1 Calculate the cosine similarity between these two sets of nonlinear features. , .

[0091] For the exemplary structure of the encoder above, the data-enhanced perturbation samples x i1 and x N+i1 The input encoder will produce two sets of feature vectors, each with a length of 256. h i1 and h N+i1 , to feature vector h i1 and h N+i1 By inputting a nonlinear mapping terminator, nonlinearity can be introduced into the feature mapping process, resulting in two sets of nonlinear features of length 256. z i1 and z N+i1 .

[0092] Using feature representation z i1 and z N+i1 Calculate the cosine similarity between perturbed samples , They are used as a similarity measure between perturbation samples:

[0093] ,

[0094] Next, this metric needs to be transformed into a loss function for the convergent network. This self-supervised algorithm model uses the contrastive learning loss function, Normalized Temperature-Scaled Cross-Entropy Loss (NT-Xent loss):

[0095]

[0096]

[0097] In the formula: This is a temperature hyperparameter that controls the input and expands the cosine similarity; an example setting for this parameter is 0.5. This represents the total number of data points in the first training dataset.

[0098] Calculate the average loss of all augmented data pairs in each batch as the final loss error for model backpropagation:

[0099]

[0100] Train the first model until convergence using the final loss error L, and save the network parameters of the encoder and the mapping end for use in the following steps.

[0101] S103. Use the trained first model to filter out unlabeled data outside the target category, and use the unlabeled data and labeled data within the target category as the second training dataset.

[0102] Taking the electromagnetic signal data collected above as an example, the first model, after self-supervised pre-training, is used as a feature extractor. 300 sets of labeled data are input into the feature extractor to obtain corresponding feature vectors. Using feature vectors belonging to the same category, prototypical representations for each category are obtained. :

[0103]

[0104] In the formula: Indicates the number of categories of the identified target. This represents the number of data points belonging to category c of electromagnetic signals, where c is the category identifier. For the first k The feature vector corresponding to an electromagnetic signal belonging to category c.

[0105] Then, input all the unlabeled data into the feature extractor to obtain the corresponding feature vectors. Then, using the prototype representations of each category, the feature vectors of all unlabeled data are calculated. Cosine similarity with each target category As the confidence score of unlabeled data for the target category:

[0106]

[0107] The category with the highest confidence in the cosine similarity score is used as the predicted category for the unlabeled data. This also yields the maximum similarity score for the unlabeled data. By setting thresholds for the similarity scores of each category, the unlabeled dataset is divided into unlabeled data within the target category and unlabeled data outside the target category. The thresholds are set using the 3-sigma principle, with the thresholds set as follows:

[0108]

[0109] in: The mean of the cosine similarity scores for labeled data. The standard deviation of the cosine similarity scores for labeled data. For hyperparameters, An example value is 1.

[0110] Those above or equal to the threshold Unlabeled data is determined to be unlabeled data within the target category; otherwise, it is unlabeled data outside the target category.

[0111] S104. Freeze the weights of the trained first model, add a soft label generation layer to the first model to form a third model, and input labeled data into the third model for fine-tuning training.

[0112] One implementation of the software label generation layer is a fully connected layer. In practice, a fully connected layer is first initialized for the first model to form a third model, used to generate soft labels. Then, the third model is fine-tuned using labeled data.

[0113] During fine-tuning, all weights inherited from the first model are frozen, and only fully connected layers are trained, thus obtaining a complete third model of a large classification network with generalization capabilities.

[0114] S105. Use the trained third model to obtain the soft label for each data in the first training dataset. Use the labeled data with soft labels, the unlabeled data within the target category, and the unlabeled data outside the target category that meets the set conditions as the third training dataset.

[0115] The third model is used to calculate soft labels for the training data, which helps the second model learn the third model's ability to extract data features and its generalization ability through model distillation. The calculation method is to use the normalized exponential function (SoftMax) to calculate the probability distribution of the six prediction classes from the six confidence data output by the third model.

[0116] The soft labels obtained in the above method are divided into two categories: soft labels for labeled data and soft labels for unlabeled data. However, not all of these soft labels will be used to calculate the loss. In step 103, the unlabeled data is divided into two parts: unlabeled data within the target category and unlabeled data outside the target category. When calculating the distillation loss later, all the soft labels of the labeled data, all the unlabeled data within the target category, and a portion of the unlabeled data outside the target category will be used. The portion of unlabeled data outside the target category is selected based on the confidence score, where the confidence score is defined as the confidence score when an electromagnetic signal data... When input into the model, the model outputs the maximum probability of each category corresponding to that data:

[0117] .

[0118] In the formula: Electromagnetic signal data The corresponding confidence score, Electromagnetic signal data The probability of the corresponding category label being y. In the example above, there are 6 target categories, and 6 possible category labels for y. The category with the highest probability is selected as the electromagnetic signal data. The corresponding confidence score.

[0119] Considering the variability of data features and the uninterpretability of neural networks, and given the slight shortcomings of heuristic settings based on subjective experience in algorithm transferability, the mean confidence score of labeled data is used as the confidence threshold. Unlabeled data outside the target category with a confidence score greater than or equal to the confidence threshold are selected for distillation learning. Their soft labels are used to calculate the distillation loss, i.e.:

[0120]

[0121] S106. Using the third training dataset, the second model learns the feature representation of the third model through model distillation, and the distillation loss is obtained. The second training dataset is used to perform semi-supervised learning on the second model, and the semi-supervised learning loss is obtained. .

[0122] The second model for electromagnetic signal classification consists of an input layer, a feature extraction layer, and a fully connected layer. The input layer augments the dimensionality of the training sample data, increasing the number of channels to 1×2×128. In the feature extraction layer, the augmented data is first upsampled to 32×1×128 by a convolutional layer with a kernel size of 2×15, and then passed through a batch normalization layer and a ReLU activation layer. After activation, the data is downsampled to 32×1×64 by a max pooling layer with a pooling size of 1×2 and a stride of 1×2. The data is then upsampled to 64×1×64 by a convolutional layer with a kernel of 1×3. After batch normalization and ReLU activation, the data is downsampled to 64×1×32 by a max pooling layer with a pooling size of 1×2 and a stride of 1×2. The data is then fed into four residual modules with the same parameters. The data structure of all residual modules is maintained in the table. The residual modules contain the following in order: (1) input data, (2) convolutional layer with a kernel size of 1×3, (3) batch normalization layer, (4) convolutional layer with a kernel size of 1×3, (5) batch normalization layer, (6) matrix summation layer (summing with the input of the residual module), and (7) ReLU activation layer. After passing through four residual modules, the data is downsampled to 64×1×30 by a convolutional layer with a kernel size of 1×3, then reduced to one-dimensional feature data of length 64 by a two-dimensional global average pooling layer (GlobalAveragePooling2D), and finally upsampled to a length of 128 by a densely connected layer. A feature vector of length 128 is then calculated using the ReLU activation function. The fully connected layer is a fully connected network layer. After passing through the fully connected layer, the features are reduced to a predicted value of length 6. This predicted value is then fed into the SoftMax function to obtain the network's final predicted probability.

[0123] S106.1 Distillation Loss The calculation.

[0124] The soft labels for labeled data, unlabeled data within the distribution, and data outside the target category that passed the screening are recorded. This data is then fed into a second model, which serves as a small classification network, to obtain the predicted probability for each target category. The mean squared error loss is calculated using the predicted probability of each target category and the soft labels.

[0125]

[0126] In the formula: The second model is based on the category label y on the input data. The predicted results obtained The third model is based on the category label y on the input data. The predicted results obtained.

[0127] The final distillation loss is obtained by calculating the mean squared error loss for all samples. .

[0128] S106.2, Semi-supervised learning loss The calculation.

[0129] For example, the semi-supervised learning loss error is calculated using the MixMatch algorithm. The MixMatch algorithm can be replaced by algorithms such as Pi-Model and FixMatch.

[0130] Semi-supervised learning algorithms mainly consist of four parts: unlabeled data augmentation, pseudo-label result prediction, data mixing augmentation (MixUp), and semi-supervised loss function calculation.

[0131] S106.2.1, Unlabeled Data Augmentation.

[0132] For each unlabeled data point, perform K data augmentation operations. Each data augmentation operation includes random flipping and random rotation of the complex field, resulting in K sets of augmented data {x1,…,x}. K}

[0133] S106.2.2 Prediction of pseudo-label results.

[0134] Combination Figure 3 As shown. For each unlabeled data point, its corresponding K groups of augmented data {x1,…,x} are used to generate the augmented data {x1,…,x}. K Input the second model and output the probabilities corresponding to K sets of prediction results. By means operation, the consistency probability distribution corresponding to each unlabeled data point is obtained. :

[0135]

[0136] By sharpening, the probability distribution Normalization using a power function yields the prediction results. The type of electromagnetic signal corresponding to the prediction result is used as a pseudo-label:

[0137] ,

[0138] In the formula: n is the total number of electromagnetic signal categories, T is a hyperparameter, and T is the temperature value of the sharpening function. The sharpened probability distribution will be closer to the Dirac distribution (or the so-called "one-hot" distribution) expected in deep learning.

[0139] S106.2.3, Data MixUp Enhancement.

[0140] Labeled data and unlabeled data with pseudo-labels are concatenated and rearranged (e.g., the two sets are joined together and then shuffled) to form mixed data. After that Take from front to back Data of the same length and Perform a mixing operation, and the resulting data will be used as new labeled data. Then The remaining data and the data with pseudo-labels Perform a mixing operation to obtain new pseudo-label data. .

[0141] The mixed operation is implemented as follows:

[0142] A pair of data with labels or pseudo-labels and Enhanced samples are obtained by mixing in the following manner. ,in:

[0143]

[0144]

[0145] In the formula:

[0146] , Electromagnetic signal sample, , For tags or pseudo tags, Through hyperparameters The generated random parameters conform to a beta distribution. The example setting value is 0.75, and the generation method is as follows:

[0147]

[0148] .

[0149] S106.2.4 Calculation of semi-supervised loss function.

[0150] Combination Figure 4 As shown. Using the obtained labeled data and pseudo-label data The supervised loss and semi-supervised loss are calculated separately, and the weighted sum of the two is used to obtain the final semi-supervised learning loss error.

[0151] Specifically, the standard cross-entropy loss of labeled data is used as the supervised loss term:

[0152]

[0153] in, For data The number of samples in for The data in for The tag, For the second model, the data Predicting target types The probability, for and Cross-entropy.

[0154] Using a consistency regularization term as an unsupervised loss term ensures that the model outputs consistent values ​​for different perturbation versions of the same original data.

[0155]

[0156] In the above formula For data The number of samples in For data middle pseudo-tags The second model uses data. Predicting target types The probability of.

[0157] Using MSE as the loss function for unsupervised terms leverages its boundedness and insensitivity to incorrect predictions, which is beneficial for semi-supervised learning of unlabeled data and for measuring prediction uncertainty.

[0158] The weighted sum of the supervised and unsupervised project losses yields the final semi-supervised learning loss error. :

[0159]

[0160] In the formula: For custom parameters, WarmUp linear auto-increment is used as an example, which increases linearly from 0 to 1 over ten training rounds.

[0161] S107, Distillation loss and semi-supervised learning loss Weighted summation as loss error The second model is trained, and the trained second model can be used for individual electromagnetic signal recognition; wherein:

[0162]

[0163] In the formula: For example, WarmUp linear auto-increment is used, which means it grows linearly from 0 to 1 over ten training rounds.

[0164] See the overall block diagram of the above implementation method. Figure 5 As shown.

[0165] After obtaining the trained second model, the collected test data can be input to test its robustness against out-of-classification data. The predicted results are compared with the labels, the correct probability is calculated, and the performance is compared with that of directly using a semi-supervised learning network.

[0166] Specific results are as follows Figure 6 As shown, "MixMatch without OOD data" indicates the accuracy achieved on the test set using a semi-supervised learning algorithm when there are no OOD samples in the unlabeled data. After model convergence, the accuracy is around 80%. "MixMatch with OOD data" indicates the accuracy achieved on the test set using a semi-supervised learning algorithm when OOD unlabeled data is present. After model convergence, the recognition accuracy drops by approximately 10%, with a maximum recognition rate of 71%. This demonstrates that the model's recognition performance significantly decreases when OOD data is present, proving that conventional semi-supervised learning methods suffer from poor anti-interference capabilities and lack robust learning in real-world scenarios. "This invention with OOD data" indicates the accuracy achieved on the test set using the method of this invention when OOD unlabeled data is present. After model convergence, the recognition accuracy recovers by approximately 10%, with a maximum recognition rate of 80%. Furthermore, the model's convergence speed is faster than traditional semi-supervised algorithms, and the convergence curve is smoother.

[0167] Comparative experimental results with methods in the same field are as follows: Figure 7As shown in the figure, this diagram represents the comparative experimental results of the present invention with the DS3L and UASD algorithms under the same conditions. All three methods were tested with OOD samples present in the unlabeled data. "DS3L with OOD data" indicates the accuracy obtained using the DS3L algorithm on the test set, achieving a maximum accuracy of 77% after the classification model converges. However, the recognition rate curve still exhibits significant oscillations after model convergence. "UASD with OOD data" indicates the recognition accuracy obtained using the UASD algorithm on the test set after training the model, achieving a maximum accuracy of 74% after the classification model converges. The convergence curve also shows some oscillations. "The present invention with OOD data" indicates the accuracy obtained using the model trained by the present invention on the test set. After the classification model converges, the recognition accuracy reaches a maximum of 80%, and the convergence curve is smooth without oscillations. Compared with methods in the same field, it can be seen that the electromagnetic signal recognition model obtained by the technical solution of the present invention has better recognition performance in terms of recognition accuracy; in terms of robustness, the model trained by the present invention has a more stable and smoother convergence curve. Thus, it can be concluded that the present invention has better performance and robustness than the same technology in the same field.

[0168] The results show that the method of this invention significantly reduces the negative impact of OOD data on model accuracy in open set semi-supervised learning scenarios, provides stronger anti-interference performance for semi-supervised learning methods, enables the model to robustly mine and learn the features of the data, and thus achieves highly robust and reliable electromagnetic signal recognition through this robust semi-supervised learning technique.

[0169] In summary, this invention, in obtaining a model for electromagnetic signal recognition, fully leverages the advantages of pre-trained feature extraction, filters out unlabeled data with unknown interference, and retains the filtered unlabeled data within the target category for semi-supervised learning. This reduces the negative impact of learning out-of-target data during model training. Simultaneously, the SimCLR self-supervised algorithm utilizes a larger network to learn more potential relationships between features. To reduce potential overfitting in the classification process, model distillation replaces traditional model fine-tuning. This reduces the size of the classification model while still effectively learning the generalization performance of a large-scale classification model. Furthermore, by combining the model distillation algorithm with open-set semi-supervised learning, and based on the characteristics of out-of-distribution unlabeled data, selective distillation learning is performed on unlabeled data outside the target category that may have a positive gain for the model. This further enhances the model's generalization and stability while reducing the negative performance impact of out-of-target data.

[0170] It should be noted that the residual module in the above implementation can be replaced with other network models, such as densely connected networks, CNNs, etc.

[0171] Through the above description of the embodiments, those skilled in the art can clearly understand that this disclosure can be implemented using software plus necessary general-purpose hardware, or it can be implemented using dedicated hardware including dedicated integrated circuits, dedicated CPUs, dedicated memory, dedicated components, etc. Generally, any function performed by a computer program can be easily implemented using corresponding hardware, and the specific hardware structure used to implement the same function can be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for this disclosure, software implementation is more often a preferred implementation method.

[0172] Although embodiments of the present invention have been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments and application fields described above. The specific embodiments described above are merely illustrative and instructive, and not restrictive. Those skilled in the art can make many other forms based on the guidance of this specification and without departing from the scope of protection of the claims of the present invention, and all of these are within the scope of protection of the present invention.

Claims

1. A method for individual identification of electromagnetic signals using open-set semi-supervised learning techniques, characterized in that, The method includes the following steps: The first training dataset is obtained by using an electromagnetic spectrum detection device to acquire radio frequency electromagnetic signal samples. The first training dataset is an open set, consisting of labeled data and unlabeled data. The unlabeled data consists of unlabeled data within the target category and unlabeled data outside the target category. The first model was trained using the SimCLR self-supervised learning algorithm on the first training dataset. The first trained model is used to filter out unlabeled data outside the target category, and the unlabeled data and labeled data within the target category are used as the second training dataset. Freeze the weights of the trained first model, add a soft label generation layer to the first model to form a third model, and input labeled data into the third model for fine-tuning training. The trained third model is used to obtain the soft label of each data in the first training dataset. The labeled data with soft labels, the unlabeled data within the target category, and the unlabeled data outside the target category that meet the set conditions are used as the third training dataset. The set conditions are that the confidence score is greater than or equal to the confidence threshold. The confidence score is the maximum probability of each category corresponding to an electromagnetic signal data when it is input into the model. The average confidence score of the labeled data is set as the confidence threshold. The second model is trained by using the third training data set to obtain the feature expression of the third model through model distillation learning, and a distillation loss is obtained The second model is trained by using the second training data set to obtain the feature expression of the third model through model distillation learning, and a distillation loss is obtained ; losses due to distillation and semi-supervised learning losses weighted sum as loss error training a second model, the trained second model being capable of individual identification of radio frequency electromagnetic signals; wherein: ， where: is the WarmUp linear self-increment, is the calculation of the mean square error loss of , and is the prediction result obtained by the second model based on the category label y on the input data , is the prediction result obtained by the third model based on the category label y on the input data ; is the mean value of the mean square error loss calculation of all data.

2. The method according to claim 1, characterized in that: The first model includes an encoder and a mapping end, with the mapping end used to assist encoder training convergence. The encoder consists of an input layer and a feature extraction layer, and the mapping end consists of a non-linear fully connected network. The feature vectors output from the feature extraction layer are input to the mapping end to obtain the data representation used by the self-supervised learning algorithm to calculate the loss.

3. The method of claim 1, wherein, Using the first training dataset, the first model is trained using the SimCLR self-supervised learning algorithm, including the following steps: Data augmentation is performed on each data in the first training data set, and each data augmentation includes complex field random flipping and complex field random rotation, thereby obtaining two groups of data augmentation data x i1 and x N+i1 ; For each data in the first training data set, two groups of data after data augmentation processing x i1 and x N+i1 Input the first model to obtain two groups of nonlinear features z i1 and z N+i1 Calculate the cosine similarity of the two groups of nonlinear features 、 ; Based on cosine similarity , , compute learning loss and , and further compute average loss as final loss error for backpropagation of the first model : In the formulae: is a temperature hyperparameter, is the total number of data in the first training dataset.

4. The method of claim 1, wherein, The first trained model is used to filter out unlabeled data that does not belong to the target category. The steps include: The feature vector is obtained by inputting labeled data into the first trained model. Using eigenvectors belonging to the same category, a prototype representation of each category is obtained : In the formula: represents the number of categories of the identification target, represents the number of data of electromagnetic signals belonging to the cth category, c is a category identifier, is the cth category, k 1 is the feature vector corresponding to the electromagnetic signal belonging to the cth category. Inputting each unlabeled data into the trained first model to obtain a corresponding feature vector , and calculating cosine similarity of the feature vector with each category ， Based on cosine similarity, the unlabeled dataset is divided into unlabeled data within the target category and unlabeled data outside the target category.

5. The method of claim 4, wherein, The unlabeled dataset is divided into unlabeled data within the target category and unlabeled data outside the target category, based on the following criteria: determining unlabelled data above or equal to a threshold value as in-target unlabelled data, and otherwise as out-of-target unlabelled data, the threshold value is set to: where: is the mean of the cosine similarity scores of the labeled data, is the standard deviation of the cosine similarity scores of the labeled data, is a hyperparameter.

6. The method according to claim 1, characterized in that: The soft label generation layer is a fully connected layer that inputs labeled data into the third model for fine-tuning and training. The fine-tuning training involves optimizing the parameters of the fully connected layers without changing the parameter values ​​of the trained first model during training, until the third model converges.

7. The method of claim 1, wherein, Semi-supervised learning loss The calculation step comprises: For each unlabeled data point, perform K data augmentation operations. Each data augmentation operation includes random flipping and random rotation of the complex field, resulting in K sets of augmented data. For each unlabeled data, the corresponding K groups of data enhanced data are sequentially input into the second model, and the probability corresponding to the K groups of prediction results is output , thereby obtaining a consistency probability distribution corresponding to each unlabeled data : normalizing the probability distribution in a power function manner to obtain a prediction result taking the kind of the electromagnetic signal corresponding to the prediction result as a pseudo label ， In the formula: n is the total number of electromagnetic signal categories, T is a hyperparameter, and is the temperature value normalized by the power function; Splicing and rearranging the labeled data and the unlabeled data with pseudo labels to form mixed data , then taking data of the same length from front to back and mixing them to obtain new labeled data , and mixing the remaining data of with the data with pseudo labels to obtain new pseudo label data ;​​​ Using the obtained labeled data and pseudo-labeled data respectively compute supervised loss and unsupervised loss weight sum of both to obtain final semi-supervised learning loss error : In the formula: is the WarmUp linear self-increment value.

8. The method of claim 7, wherein, The mixed operation is implemented as follows: a pair of data with labels or pseudo labels and enhanced samples are obtained by mixing in the following way wherein: In the formula: , is an electromagnetic signal sample, , is a label or pseudo-label, is a random parameter following a beta distribution generated by hyperparameters generated by the following method: ， 。

Images