Small sample electromagnetic signal open set identification method based on multi-modal characteristics

Abstract

The invention relates to a small sample electromagnetic signal open set identification method based on multi-modal features, belongs to the technical field of communication signal processing, and solves the problem that an existing electromagnetic signal identification method based on deep learning depends on a large number of labeled samples and is based on closed set hypothesis. And the technical problems of difficulty in sample labeling in a complex dynamic electromagnetic environment and occurrence of unknown category signals are solved. Comprising the following steps: acquiring an electromagnetic signal in real time and performing power normalization to obtain a to-be-identified signal; inputting a known class signal data set and the to-be-recognized signal into a trained electromagnetic signal open set recognition model, and performing unsupervised classification recognition to obtain a recognition result; wherein the electromagnetic signal open set recognition model is used for judging based on the mahalanobis distance between the multi-modal semantic feature vector corresponding to the signal to be recognized and the central point vector of each known type signal in the known type signal data set, and obtaining the type of the signal to be recognized. And rapid and accurate identification of the electromagnetic signal type is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of communication signal processing, and in particular to an open set recognition method for small sample electromagnetic signals based on multimodal features. Background Art

[0002] Wireless communication electromagnetic signal recognition is crucial in modern communication systems, especially in complex electromagnetic environments. In civilian applications, electromagnetic signals can cause data transmission errors, call interruptions, or degraded service quality. Therefore, electromagnetic signal recognition plays an irreplaceable role in maintaining the efficient operation of modern wireless communication systems and is a key component in ensuring information transmission quality and network security.

[0003] Compared with traditional electromagnetic signal recognition algorithms, deep learning has begun to achieve good results in image processing, speech recognition and other fields. Therefore, scholars have begun to apply deep learning in the field of wireless communications to achieve fast and accurate recognition of electromagnetic signal types.

[0004] While deep learning-based methods have shown promising application prospects in electromagnetic signal recognition, they require sufficient labeled samples and a closed-set assumption. During training, various neural networks are generally able to perform electromagnetic signal recognition well if sufficient and well-labeled samples are provided. However, in complex, ever-changing, or non-cooperative electromagnetic environments, acquiring a large number of high-quality electromagnetic signal samples and annotating them in detail is an extremely challenging and costly task. Furthermore, most existing methods currently employ a closed-set recognition approach. This assumes that the dataset can be divided into N known, labeled categories, and expects the input to belong to one of these N categories. However, in real electromagnetic environments, various new electromagnetic signals constantly emerge, making it difficult to collect all possible electromagnetic signal types for model training. Summary of the Invention

[0005] In view of the above analysis, an embodiment of the present invention aims to provide an open set recognition method for small sample electromagnetic signals based on multimodal features, which is used to solve the technical problems that existing deep learning-based electromagnetic signal recognition methods rely on a large number of labeled samples and are mostly based on closed set assumptions, making it difficult to adapt to the difficulties in sample labeling and the emergence of unknown category signals in complex dynamic electromagnetic environments.

[0006] The purpose of the present invention is mainly achieved through the following technical solutions:

[0007] The present invention provides a method for open set recognition of small sample electromagnetic signals based on multimodal features, comprising the following steps:

[0008] Acquire electromagnetic signals in real time and perform power normalization to obtain the signal to be identified;

[0009] Inputting the known class signal data set and the signal to be identified into the trained electromagnetic signal open set recognition model to perform unsupervised classification and recognition to obtain a recognition result;

[0010] Among them, the electromagnetic signal open set recognition model is used to make a judgment based on the Mahalanobis distance between the multimodal semantic feature vector corresponding to the signal to be identified and the center point vector of each known class signal in the known class signal data set to obtain the category of the signal to be identified.

[0011] Furthermore, the electromagnetic signal open set recognition model includes a multimodal feature extractor and a distance-based classifier; wherein,

[0012] The modal feature extractor is used to extract the multimodal semantic feature vectors of the signal to be identified and each known class signal data in the known class signal data set, and calculate the center point vector of each known class signal based on the multimodal semantic feature vectors corresponding to the signal data in the known class signal data set;

[0013] The distance-based classifier is used to calculate the Mahalanobis distance between the multimodal semantic feature vector corresponding to the signal to be identified and the center point vector of each known class signal, so as to determine the category of the signal to be identified.

[0014] Furthermore, the multimodal feature extractor includes a manual extractor, an encoder, a feature fusion module and a category feature optimizer;

[0015] The artificial extractor is used to obtain corresponding artificial semantic features based on the low-dimensional discriminant feature vectors of the input signal to be identified and the signal data in the known class signal data set respectively;

[0016] The encoder is used to extract high-dimensional hidden features of the input signal to be identified and the signal data in the known class signal data set respectively;

[0017] The feature fusion module is used to fuse the artificial semantic features and high-dimensional hidden features of the signal to be identified and the signal data in the known class signal dataset to obtain a multimodal semantic feature vector corresponding to the signal to be identified and the signal data in the known class signal dataset;

[0018] The category feature optimizer is used to calculate the center point vector of each known class signal based on the multimodal semantic feature vector corresponding to the signal data in the known class signal data set.

[0019] Furthermore, the category of the signal to be identified is determined based on the Mahalanobis distance and a predetermined threshold as follows:

[0020]

[0021] Among them, k is the kth known category; γ c is a predetermined threshold; is the multimodal semantic feature vector z corresponding to the signal to be identified i To the center point vector of the k-th known class signal Mahalanobis distance;

[0022] When d min ≤γ c When y i is a known class, otherwise, determine y i It is an unknown class;

[0023] If it is a known class, select The known category corresponding to the minimum value is the category of the signal to be identified.

[0024] Furthermore, the center point vector of each known class signal in the known class signal data set is calculated as follows:

[0025]

[0026] in, N l is the total number of signal data in the known class signal dataset; Encode the vector of the true category; Signal to be identified Multimodal semantic features; N c is the total number of known categories.

[0027] Furthermore, the electromagnetic signal open set recognition model is obtained through the following training process:

[0028] Preprocessing the electromagnetic signals including multiple modulation types generated by simulation to obtain a labeled sample data set and an unlabeled sample data set; wherein the label is the modulation type;

[0029] Constructing a small sample electromagnetic signal open set recognition model; wherein the small sample electromagnetic signal open set recognition model sequentially comprises a multimodal feature extractor, a parallel labeled classifier and an unlabeled classifier, a relative entropy evaluator, and a parallel labeled decoder and an unlabeled decoder;

[0030] The labeled sample dataset and the unlabeled sample dataset are used to perform semi-supervised training on a small sample electromagnetic signal open set recognition model, including:

[0031] Loading the labeled sample dataset and the unlabeled sample dataset into the multimodal feature extractor;

[0032] Based on the joint loss function, the small sample electromagnetic signal open set recognition model is trained, and the model parameters are continuously adjusted and updated through back propagation and gradient descent optimization algorithms;

[0033] The labeled classifier and the unlabeled classifier share parameters; the labeled decoder and the unlabeled decoder share parameters;

[0034] Until the joint loss function converges or the training ends at a preset maximum number of iterations, the parameters of the small sample electromagnetic signal open set recognition model are saved to obtain a trained small sample electromagnetic signal open set recognition model;

[0035] A multimodal feature extractor is taken out from the trained small sample electromagnetic signal open set recognition model, and a distance-based classifier is added thereto to obtain a trained electromagnetic signal open set recognition model.

[0036] Furthermore, the unlabeled classifier obtains a corresponding multimodal semantic feature vector based on the unlabeled sample data in the unlabeled sample data set through the multimodal feature extractor, outputs a corresponding prediction probability vector, and uses the category index corresponding to the maximum probability in the prediction probability vector as a pseudo label for the unlabeled sample data. During the model training process, the pseudo label is regarded as the true label to participate in the cross entropy loss calculation;

[0037] The labeled classifier is used to improve the classification accuracy of the labeled sample data set by using cross entropy loss during model training;

[0038] The relative entropy evaluator performs data enhancement on the unlabeled sample dataset by minimizing the KL divergence loss based on the predicted probability vector output by the unlabeled classifier;

[0039] The labeled decoder and the unlabeled decoder are used to upsample the multimodal semantic feature vector to reconstruct labeled sample data and unlabeled sample data, respectively. During the model training process, the labeled sample data and the unlabeled sample data are reconstructed by minimizing the reconstruction loss.

[0040] Furthermore, the joint loss function L is as follows:

[0041] L=λ ce L ce +λ cl L cl +λ kl L kl +λ re L re

[0042] Among them, L ce is the cross entropy loss; L cl is the center loss; Lkl is the KL divergence loss; L re is the reconstruction loss; ce ,λ cl ,λ kl ,λ re L ce 、L cl 、L kl 、L re The weight of .

[0043] Furthermore, the relative entropy estimator performs data enhancement on the unlabeled sample data by minimizing the KL divergence loss, including:

[0044] In the unlabeled sample data set, the unlabeled sample data And the corresponding unlabeled sample data with added noise perturbation After passing through the multimodal feature extractor and the unlabeled classifier, the probability vector and

[0045] calculate and The KL divergence loss is as follows:

[0046]

[0047] Among them, N u is the total number of signals in the unlabeled sample dataset; is the KL divergence loss function, y i for for For unlabeled sample data The probability value of belonging to the jth category; Represents unlabeled sample data with added noise perturbation The probability value of belonging to the jth category; H is the total number of categories.

[0048] Furthermore, the pre-processing of the electromagnetic signals including multiple modulation types generated by simulation includes:

[0049] intercepting the electromagnetic signal according to a preset length and performing power normalization to obtain a first electromagnetic signal set;

[0050] Dividing the first electromagnetic signal set into a labeled sample data set and an unlabeled sample data set according to a preset ratio;

[0051] Noise is injected into samples in the unlabeled sample dataset to perform data enhancement.

[0052] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0053] 1. This invention adopts a semi-supervised learning method, which combines a small number of labeled samples with a large number of unlabeled samples for joint training. This solves the problem that traditional methods require a large amount of labeled data and significantly reduces the dependence on labeled samples.

[0054] 2. The multimodal feature extractor of the present invention integrates the low-dimensional discriminant features of the manual extractor (such as time domain moment kurtosis, frequency domain moment kurtosis, etc.) and the high-dimensional hidden features learned by the encoder, overcoming the limitations of a single feature, enhancing the model's ability to characterize complex signals, and fully exploring the multi-dimensional features of electromagnetic signals. Multimodal feature fusion improves feature discriminability and enhances the recognition accuracy of known categories;

[0055] 3. This invention optimizes category features through a category center optimizer and a relative entropy evaluator, and uses data augmentation technology to explore data invariance, thereby improving the model's generalization ability for unknown categories and noisy data. It also constrains the prediction consistency between original samples and noise-perturbed samples through the KL divergence loss, and combines the reconstruction loss to preserve the essential characteristics of the signal, significantly improving the model's stability in low signal-to-noise ratio environments.

[0056] 4. In the present invention, the unlabeled classifier generates pseudo labels for unlabeled samples and participates in the cross-entropy loss calculation. The relative entropy evaluator performs data enhancement by minimizing the KL divergence loss, making full use of unlabeled data and improving model performance. A four-element joint loss function of cross entropy loss, center loss, KL divergence loss, and reconstruction loss is designed to balance classification accuracy, intra-class compactness, data enhancement, and feature integrity, achieving efficient multi-objective optimization and reducing training resource consumption.

[0057] 5. The present invention breaks through the limitation of closed set assumption and realizes open set recognition. The classification strategy based on Mahalanobis distance combined with a predetermined threshold can effectively identify known category signals and detect unknown category signals, thereby enhancing the model's open set recognition capability in actual complex environments and adapting to scenarios where new signals dynamically appear in real electromagnetic environments.

[0058] In the present invention, the above-mentioned technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of the present invention will be described in the following description, and some advantages will become apparent from the description or be learned through practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like parts throughout the drawings.

[0060] Figure 1 This is a flow chart of a method for open set recognition of small sample electromagnetic signals based on multimodal features in an embodiment of the present invention;

[0061] Figure 2 Schematic diagram of semi-supervised training and unsupervised classification recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0063] In order to solve the above problems, it is necessary to transform the traditional electromagnetic signal closed set recognition problem into a small sample open set recognition problem using a small amount of labeled sample data and a large amount of unlabeled sample data. A specific embodiment of the present invention discloses a small sample electromagnetic signal open set recognition method based on multimodal features, such as Figure 1 As shown, the following steps are included:

[0064] Step S1: acquiring electromagnetic signals in real time and performing power normalization to obtain signals to be identified;

[0065] Step S2: inputting the known class signal data set and the signal to be identified into a trained electromagnetic signal open set recognition model to perform unsupervised classification and recognition to obtain a recognition result;

[0066] Among them, the electromagnetic signal open set recognition model is used to make a judgment based on the Mahalanobis distance between the multimodal semantic feature vector corresponding to the signal to be identified and the center point vector of each known class signal in the known class signal data set to obtain the category of the signal to be identified.

[0067] The electromagnetic signal open set recognition model includes a multimodal feature extractor and a distance-based classifier.

[0068] Step S1, specifically.

[0069] For example, a high-performance RF front-end module is used to capture electromagnetic signals in the communication environment in real time; if the real-time electromagnetic signal is continuous, a data caching mechanism is designed to temporarily store the collected electromagnetic signal data in a high-speed cache area, and then read it on demand by subsequent recognition, to ensure the synchronization and smoothness of the electromagnetic signal collection and recognition process.

[0070] The goal of power normalization is to eliminate the amplitude differences of electromagnetic signals and ensure that electromagnetic signals of different intensities are processed at the same scale.

[0071] For example, peak normalization is used for transient electromagnetic signals collected in real time; average power normalization is used for continuous electromagnetic signals collected in real time.

[0072] For electromagnetic signals collected in real time, power normalization is performed in real time to ensure the consistency and stability of signal recognition.

[0073] Step S1 captures electromagnetic signals in real time and performs power normalization processing on them to eliminate the amplitude differences of electromagnetic signals and ensure that electromagnetic signals of different intensities are processed at a unified scale, thereby ensuring the consistency and stability of the electromagnetic signal open set recognition model in signal category recognition.

[0074] Step S2 includes steps S21-S22.

[0075] Step S21: using the electromagnetic signal open set recognition model to classify and recognize the signal to be recognized, and obtain a recognition result.

[0076] The electromagnetic signal open set recognition model includes a multimodal feature extractor and a distance-based classifier; wherein,

[0077] The modal feature extractor is used to extract the multimodal semantic feature vectors of the signal to be identified and each known class signal data in the known class signal data set, and calculate the center point vector of each known class signal based on the multimodal semantic feature vectors corresponding to the signal data in the known class signal data set;

[0078] The distance-based classifier is used to calculate the Mahalanobis distance between the multimodal semantic feature vector corresponding to the signal to be identified and the center point vector of each known class signal to determine the category of the signal to be identified;

[0079] The multimodal feature extractor includes a manual extractor, an encoder, a feature fusion module and a category feature optimizer;

[0080] The artificial extractor is used to obtain corresponding artificial semantic features based on the low-dimensional discriminant feature vectors of the input signal to be identified and the signal data in the known class signal data set respectively;

[0081] The encoder is used to extract high-dimensional hidden features of the input signal to be identified and the signal data in the known class signal data set respectively;

[0082] The feature fusion module is used to fuse the artificial semantic features and high-dimensional hidden features of the signal to be identified and the signal data in the known class signal dataset to obtain a multimodal semantic feature vector corresponding to the signal to be identified and the signal data in the known class signal dataset;

[0083] The category feature optimizer is used to calculate the center point vector of each known class signal based on the multimodal semantic feature vector corresponding to the signal data in the known class signal data set.

[0084] The electromagnetic signal open set recognition model is used to classify and recognize the electromagnetic signal after power normalization acquired in real time to obtain a corresponding recognition result; the recognition result is an unknown class or a known modulation type.

[0085] The recognition and classification of electromagnetic signals is unsupervised recognition and classification, which requires a data set of known class signals.

[0086] Exemplarily, for example, simulation software MATLAB is used to generate communication electromagnetic signal samples of 9 common modulation types; the 9 modulation types include: ASK (Amplitude Shift Keying), BPSK (Binary Phase Shift Keying), 4PSK (Quadrature Phase Shift Keying), QPSK (Quadrature Phase Shift Keying, another quaternary phase shift keying), 16APSK (16-Amplitude Phase Shift Keying), 64APSK (64-Amplitude Phase Shift Keying, 64 Amplitude Phase Shift Keying), 16QAM (16-QuadratureAmplitude Modulation, 16 orthogonal amplitude modulation), 32QAM (32-QuadratureAmplitude Modulation, 32 orthogonal amplitude modulation) and 64QAM (64-QuadratureAmplitude Modulation, 64 orthogonal amplitude modulation).

[0087] In practical applications, additional modulation types can be added based on specific needs. The known signal dataset consists of a small amount of electromagnetic signal sample data with modulation type labels. Before being used for electromagnetic signal recognition, the electromagnetic signals in the known signal dataset are power normalized.

[0088] The low-dimensional discriminant feature vector of the signal to be identified or the signal data in the known class signal data set is as follows:

[0089]

[0090] Among them, a4, b4, b3, ρ f 、P flat They are time domain moment kurtosis, frequency domain moment kurtosis, frequency domain moment skewness coefficient, single frequency energy concentration, frequency domain parameters and average spectrum flatness coefficient.

[0091] The six characteristics are specifically defined as follows:

[0092] F1) Time domain moment kurtosis a4

[0093] The time domain moment kurtosis is used to calculate the steepness of the time domain distribution of the electromagnetic signal, as follows:

[0094]

[0095] Among them, x i (n) is the i-th electromagnetic signal x i The nth value of μ. T is the mean value of the electromagnetic signal amplitude, σ T is the standard deviation of the electromagnetic signal amplitude, It means to find the mathematical expectation.

[0096] F2) Frequency domain moment kurtosis b4

[0097] Frequency domain moment kurtosis is used to calculate the steepness of the frequency domain distribution of electromagnetic signals, as follows:

[0098]

[0099] in, is x i (n) The kth spectrum value after discrete Fourier transform, μ F and σ F They are The mean and standard deviation of .

[0100] F3) Frequency domain moment skewness coefficient b3

[0101] The frequency domain moment skewness coefficient is used to calculate the degree of deviation of the electromagnetic signal from the normal distribution, as follows:

[0102]

[0103] F4) Single frequency energy concentration

[0104] Single-frequency energy concentration is used to calculate the energy of the electromagnetic signal at a single frequency point, as follows:

[0105]

[0106] Wherein, m is the subscript index corresponding to the maximum spectrum value, k0 is a constant, illustratively, k0 is set to 1, and N is the length of the electromagnetic signal.

[0107] F5) Frequency domain parameter ρ f

[0108] Frequency domain parameters are used to calculate the degree of envelope change of the electromagnetic signal spectrum, as follows:

[0109]

[0110] F6) Average spectrum flatness coefficient

[0111] The average spectrum flatness coefficient is used to calculate the degree of change in the electromagnetic signal spectrum amplitude as follows:

[0112]

[0113] in, is x i Power spectrum The mean of Refers to the normalized power spectrum The impulse components of are shown in equations (7) and (8) respectively.

[0114]

[0115] Among them, L s Is the length of the sliding window, usually L s =0.03N, where N is the signal length (number of sampling points).

[0116] A classification strategy based on Mahalanobis distance is used to detect the signal to be identified based on a dataset of known class signals.

[0117] Input the labeled electromagnetic signal in the known class signal dataset into the multimodal feature extractor to obtain the center point vector of each known class signal

[0118] Calculate the center point vector of each known class signal in the known class signal data set as follows:

[0119]

[0120] in, N l is the total number of signal data in the known class signal dataset; Encode the vector of the true category; Signal to be identified Multimodal semantic features; N c is the total number of known categories.

[0121] For δ(.), when the condition When true, δ(.) = 1; otherwise, δ(.) = 0;

[0122] The numerator is the sum of the multimodal semantic feature vectors of all k-th electromagnetic signals in the sample; the denominator is the total number of k-th electromagnetic signals.

[0123] The category of the signal to be identified is determined based on the Mahalanobis distance and a predetermined threshold as follows:

[0124]

[0125] Among them, k is the kth known category; γ c is a predetermined threshold; is the multimodal semantic feature vector z corresponding to the signal to be identified i To the center point vector of the k-th known class signal Mahalanobis distance;

[0126] When d min ≤γ c When y i is a known class, otherwise, determine y i It is an unknown class;

[0127] If it is a known class, select The known category corresponding to the minimum value is the category of the signal to be identified.

[0128] in, The calculation is as follows:

[0129]

[0130] Among them, Ω k is the covariance matrix corresponding to the multimodal semantic feature vector of the k-th type of signal data, Ω k -1 Ω k The inverse matrix of .

[0131] For example, γ c Set to 3.0 and change according to specific needs in actual application.

[0132] like Figure 2 As shown, K1, K2, ..., K n is a known modulation type.

[0133] Step S22: Based on the semi-supervised trained small sample electromagnetic signal open set recognition model, a trained electromagnetic signal open set recognition model is obtained.

[0134] The electromagnetic signal open set recognition model is obtained through the following training process:

[0135] Preprocessing the electromagnetic signals including multiple modulation types generated by simulation to obtain a labeled sample data set and an unlabeled sample data set; wherein the label is the modulation type;

[0136] Constructing a small sample electromagnetic signal open set recognition model; wherein the small sample electromagnetic signal open set recognition model sequentially comprises a multimodal feature extractor, a parallel labeled classifier and an unlabeled classifier, a relative entropy evaluator, and a parallel labeled decoder and an unlabeled decoder;

[0137] The labeled sample dataset and the unlabeled sample dataset are used to perform semi-supervised training on a small sample electromagnetic signal open set recognition model, including:

[0138] Loading the labeled sample dataset and the unlabeled sample dataset into the multimodal feature extractor;

[0139] Based on the joint loss function, the small sample electromagnetic signal open set recognition model is trained, and the model parameters are continuously adjusted and updated through back propagation and gradient descent optimization algorithms;

[0140] The labeled classifier and the unlabeled classifier share parameters; the labeled decoder and the unlabeled decoder share parameters;

[0141] Until the joint loss function converges or the training ends at a preset maximum number of iterations, the parameters of the small sample electromagnetic signal open set recognition model are saved to obtain a trained small sample electromagnetic signal open set recognition model;

[0142] A multimodal feature extractor is taken out from the trained small sample electromagnetic signal open set recognition model, and a distance-based classifier is added thereto to obtain a trained electromagnetic signal open set recognition model.

[0143] Based on the trained small sample electromagnetic signal open set recognition model, an electromagnetic signal open set recognition model is obtained.

[0144] Step S22 includes steps S221-S222.

[0145] Step S221 : pre-processing the electromagnetic signals including multiple modulation types generated by simulation to obtain a labeled sample data set and an unlabeled sample data set.

[0146] (1) Simulate and generate electromagnetic signal samples.

[0147] The simulation software MATLAB is used to generate nine common communication electromagnetic signal types, including ASK, BPSK, 4PSK, QPSK, 16APSK, 64APSK, 16QAM, 32QAM, and 64QAM. These signals cover low- to high-order modulation schemes, simulating the diversity of electromagnetic signals in actual communication systems.

[0148] For each type of electromagnetic signal with a SNR (Signal-to-Noise Ratio) ranging from -6dB to 12dB and an interval of 2dB (a total of 10 SNR points), 2000 samples with a sampling length of 128 points are generated to form a data set. Ensure that the model can learn robustly under different noise intensities. Simulate and generate electromagnetic signal sample sets with different modulation types. as follows:

[0149] 10 SNRs × 9 types of electromagnetic signals × 2000 = 180,000 samples Formula (13)

[0150] (2) Preprocessing of electromagnetic signal samples including multiple modulation types.

[0151] The preprocessing of the electromagnetic signals including multiple modulation types generated by simulation includes:

[0152] intercepting the electromagnetic signal according to a preset length and performing power normalization to obtain a first electromagnetic signal set;

[0153] Dividing the first electromagnetic signal set into a labeled sample data set and an unlabeled sample data set according to a preset ratio;

[0154] Noise is injected into samples in the unlabeled sample dataset to perform data enhancement.

[0155] The numerical simulation uses Gaussian white noise injection to simulate random factors such as noise and interference in the actual environment;

[0156] The power of all noise-injected electromagnetic signal samples is normalized to eliminate signal amplitude differences.

[0157] (3) The sample data set is divided into: labeled sample data set and unlabeled sample data set.

[0158] Construct labeled sample datasets and unlabeled sample datasets that meet the requirements of open set recognition.

[0159] Will Divide into a labeled sample dataset and an unlabeled sample dataset

[0160] A dataset containing a small number of labeled samples and a dataset containing a large number of unlabeled samples constitute;

[0161] N l is the total number of labeled sample data, N u is the total number of unlabeled sample data. and Represent a labeled and unlabeled received time domain sample signal respectively, Generate a sample set of electromagnetic signals for simulation. yes The one-hot vector encoding of the true category, and is the set of sample labels, N c is the total number of known class labels.

[0162] Assuming that the total number of known class styles (such as ASK, BPSK, etc.) is 9, the one-hot vector encoding of each class is as follows:

[0163] ASK: The category index is 0, and the corresponding one-hot encoding is [1,0,0,0,0,0,0,0,0];

[0164] BPSK: The class index is 1, and the corresponding one-hot encoding is [0,1,0,0,0,0,0,0,0];

[0165] And so on up to Category 9.

[0166] The labeled sample dataset contains a small amount of sample data, for example, 10 to 100 electromagnetic signals of each modulation type;

[0167] Unlabeled sample dataset, containing a large amount of sample data; for example, 1000 electromagnetic signals of each type, used for semi-supervised learning of small-sample electromagnetic signal open set recognition models.

[0168] Through precise signal simulation, strict noise control and reasonable data set division, it provides a high-quality and diverse sample data foundation for subsequent semi-supervised training.

[0169] The present invention fully considers the complexity and dynamics of the actual electromagnetic environment, ensuring that the model can still effectively distinguish between known and unknown signals under small sample conditions.

[0170] Step S222: construct a small sample electromagnetic signal open set recognition model and perform model training.

[0171] like Figure 2 The upper part shows a schematic diagram of the semi-supervised training stage. The small-sample electromagnetic signal open set recognition model includes a multimodal feature extractor, a labeled classifier, an unlabeled classifier, a relative entropy evaluator, a labeled decoder, and an unlabeled decoder.

[0172] A multimodal feature extractor is used to extract multimodal semantic features of electromagnetic signal sample data to fully utilize the advantages of different features and improve the recognition rate and scalability of the model. The multimodal feature extractor includes a manual extractor, an encoder, a feature fusion module, and a category feature optimizer. The details are as follows:

[0173] Extracting low-dimensional discriminant feature vectors of electromagnetic signals using artificial extractors As shown in formula (1); the vector elements include time domain moment kurtosis a4, frequency domain moment kurtosis b4, frequency domain moment skewness coefficient b3, single frequency energy concentration Frequency domain parameter ρ f and the average spectrum flatness coefficient As shown in formulas (2)-(9).

[0174] The low-dimensional discriminant feature vectors corresponding to the sample data in the labeled sample data set and the unlabeled sample data set Input into the artificial extractor to obtain the artificial semantic feature z corresponding to the sample data 1,i ,as follows:

[0175]

[0176] Here, φ(·) is the artificial extractor mapping function, which consists of two fully connected layers and an activation function.

[0177] The network structure of the manual extractor is shown in Table 1.

[0178] Table 1. Network structure of manual extractor

[0179]

[0180] First fully connected layer:

[0181] Receive low-dimensional discriminant feature combination vector as input.

[0182] The low-dimensional discriminant feature combination vector is converted into a 32-dimensional feature vector through linear transformation; the feature vector after linear transformation is activated by ReLU. The role of the ReLU function is to introduce nonlinearity, set the negative values ​​in the feature vector after linear transformation to zero, and keep the positive values ​​unchanged, which helps the model learn more complex feature representations and can increase the model's nonlinear fitting ability.

[0183] The feature vector after ReLU activation is still 32-dimensional, but it already contains the feature information after nonlinear transformation.

[0184] Second fully connected layer:

[0185] Receive the 32-dimensional feature vector after ReLU activation of the first fully connected layer as input.

[0186] The input 32-dimensional feature vector is linearly transformed into a 128-dimensional feature vector; the 128-dimensional feature vector after linear transformation is activated by ReLU and nonlinearly transformed to obtain the artificial semantic feature z 1,i ;

[0187] Further enhance the feature expression ability and the nonlinear fitting ability of the model, so that the extracted artificial semantic features z 1,i Able to better capture complex patterns and regularities in electromagnetic signals.

[0188] After processing by the first and second fully connected layers, the final result is a 128-dimensional feature vector z1,i, which is the artificial semantic feature of the electromagnetic signal. z1,i fully integrates the information of the six low-dimensional discriminant features and extracts high-dimensional artificial semantic features through nonlinear transformation, which can more effectively characterize the characteristics of the electromagnetic signal.

[0189] The encoder feature extractor is used to extract the high-dimensional hidden features z of the electromagnetic signal. 2,i ,as follows:

[0190] z 2,i =E(x i ) Formula (15)

[0191] Where E(.) is the encoder mapping function.

[0192] The network structure of the encoder is shown in Table 2.

[0193] Table 2 Encoder network structure

[0194]

[0195]

[0196] Input layer: The input dimension is 2×128 (each sample has 128 sampling points, and each sampling point has two feature vectors, I and Q); the input data is passed directly;

[0197] The first convolutional block consists of a convolutional layer with a kernel size of 3, 64 kernels, a convolution step of 1, and a ReLU activation function. After the convolutional layer performs a convolution operation on the input data, the ReLU activation function is applied to perform a nonlinear transformation on the convolution result.

[0198] First pooling layer: Pool_size:2, indicating that the pooling layer window size is 2;

[0199] The second convolutional block consists of a convolution layer with a kernel size of 3, 128 kernels, a convolution step of 1, and a ReLU activation function;

[0200] Second pooling layer: The pooling layer window size is 2;

[0201] The third convolutional block: consists of a convolution layer with a kernel size of 3,256 kernels, a convolution step of 1, and a ReLU activation function;

[0202] The third pooling layer: the pooling layer window size is 2;

[0203] The fourth convolutional block consists of a convolutional layer with a kernel size of 3,512 kernels, a convolution step of 1, and a ReLU activation function;

[0204] Fourth pooling layer: The pooling layer window size is 2;

[0205] Flattening layer: Flatten the features of the fourth pooling layer output dimension of 6×512 into a one-dimensional feature vector of 6×512=3072;

[0206] Then, the first, second, and third fully connected blocks are sequentially passed to obtain a high-dimensional hidden feature z with an output dimension of 128. 2,i ; Among them, the first, second and third fully connected blocks all include ReLU activation functions.

[0207] The feature fusion module combines the artificial semantic features z extracted by the artificial extractor 1,i , and the high-dimensional hidden features z extracted by the encoder 2,i Fusion is performed to obtain the multimodal semantic feature vector z of the signal i ,as follows:

[0208] z i =z 1,i +z 2,i Formula (16)

[0209] The class center optimizer is used to make the model learn how to gather the features of samples of the same class towards their class center while maintaining the feature differences between different classes. The details are as follows:

[0210] During model training, labeled sample data Input into the multimodal feature extractor to extract its multimodal semantic features as The corresponding categories are Calculate the corresponding category The center point is The calculation method is as shown in formula (10).

[0211] The center point is the average of the multimodal semantic features of the same known class signal in the same batch in the labeled sample dataset.

[0212] In order to make the multimodal semantic feature vector as close to the center of the same electromagnetic signal type as possible and at the same time away from the centers of other electromagnetic signal types, the category center optimizer is used to calculate the center loss as follows:

[0213]

[0214] Where M represents the total number of labeled samples contained in a training batch.

[0215] The category center optimizer mainly calculates the category center and optimizes the center loss for the multimodal semantic features of labeled known class sample data to enhance the model's ability to recognize known categories.

[0216] The unlabeled classifier obtains a corresponding multimodal semantic feature vector based on the unlabeled sample data in the unlabeled sample data set through the multimodal feature extractor, outputs a corresponding prediction probability vector, and uses the category index corresponding to the maximum probability in the prediction probability vector as a pseudo label for the unlabeled sample data. During the model training process, the pseudo label is regarded as the true label to participate in the cross entropy loss calculation;

[0217] The labeled classifier is used to improve the classification accuracy of the labeled sample data set by using cross entropy loss during model training;

[0218] The relative entropy evaluator performs data enhancement on the unlabeled sample dataset by minimizing the KL divergence loss based on the predicted probability vector output by the unlabeled classifier;

[0219] The labeled decoder and the unlabeled decoder are used to upsample the multimodal semantic feature vector to reconstruct labeled sample data and unlabeled sample data, respectively. During the model training process, the labeled sample data and the unlabeled sample data are reconstructed by minimizing the reconstruction loss.

[0220] The labeled classifier is used to improve the classification accuracy of labeled sample data, while the unlabeled classifier is used to make the predicted probability vector of unlabeled sample data as sparse as possible.

[0221] The network structure of the labeled classifier is consistent with that of the unlabeled classifier, and they share parameters. During the model training process, when the network parameters of one classifier are updated, the network parameters of the other classifier are also updated at the same time.

[0222] The network structures of the labeled classifier and the unlabeled classifier are shown in Table 3.

[0223] The details are as follows:

[0224] Calculate cross entropy loss for labeled and unlabeled classifiers Not only focusing on labeled samples The cross entropy loss also focuses on unlabeled samples The cross entropy loss.

[0225] Unlabeled samples The semantic features extracted by the multimodal feature extractor are In order to give it a false label Assume the predicted probability vector is correct, where C(·) is the unlabeled classifier mapping function. The index corresponding to the maximum element in is used as the pseudo label Similarly, the label classifier obtains The index corresponding to the largest participating element in Category.

[0226] The cross entropy loss is as follows:

[0227]

[0228] in,

[0229] Table 3 Classifier network structure

[0230]

[0231] The relative entropy estimator performs data enhancement on unlabeled sample data by minimizing the KL (Kullback-Leibler) divergence loss, including:

[0232] In the unlabeled sample data set, the unlabeled sample data And the corresponding unlabeled sample data with added noise perturbation After passing through the multimodal feature extractor and the unlabeled classifier, the probability vector and

[0233] calculate and The KL divergence loss is as follows:

[0234]

[0235] Among them, N u is the total number of signals in the unlabeled sample dataset; is the KL divergence loss function, y i for for For unlabeled sample data The probability value of belonging to the jth category; Represents unlabeled sample data with added noise perturbation The probability value of belonging to the jth category; H is the total number of categories.

[0236] Exemplarily, the noise disturbance is Gaussian noise or other types of random noise.

[0237] In real-world scenarios, unlabeled data is often easier to obtain than labeled data. We use a relative entropy estimator to perform data augmentation on unlabeled samples. We use the KL divergence loss to measure the difference in probability distribution between the original unlabeled samples and the noise-perturbed samples, learning the inherent invariance of the data. This not only improves the model's recognition accuracy for known categories, but also enhances its ability to detect unknown categories, while fully leveraging the value of unlabeled sample data.

[0238] The labeled decoder and the unlabeled decoder upsample the multimodal semantic features corresponding to the labeled samples and the multimodal semantic features corresponding to the unlabeled samples, respectively, and gradually reconstruct the original input labeled electromagnetic signal samples and the unlabeled electromagnetic signal samples. The details are as follows:

[0239] For multimodal semantic features z i Upsampling is performed to gradually reconstruct the original input D(·) is the decoder mapping function, and its network structure is shown in Table 4.

[0240] The network structure of the labeled decoder and the unlabeled decoder is consistent and they share parameters.

[0241] Table 4 Decoder network structure

[0242]

[0243]

[0244] The training process aims to minimize the reconstruction error as follows:

[0245]

[0246] The labeled decoder reconstructs the sample data in the labeled sample data set; the unlabeled decoder reconstructs the sample data in the unlabeled sample set.

[0247] The joint loss function is used to train the open set recognition model of small sample electromagnetic signals.

[0248] The joint loss function as follows:

[0249]

[0250] in, is the cross entropy loss; For the center loss; is the KL divergence loss; is the reconstruction loss; ce ,λ cl ,λ kl ,λ re They are The weight of .

[0251] λ ce ,λ cl ,λ kl and λ re is a constant used to balance the proportions of the four loss functions.

[0252] For example, λ ce =1.0,λ cl =0.5,λ kl = 0.1 and λ re =0.1. In practical applications, it can be revised according to specific needs.

[0253] The gradient descent method is used to update the network parameters Θ of the small sample electromagnetic signal open set recognition model as follows:

[0254]

[0255] Where η is the step factor, which is generally set to a constant; is the joint loss function The network parameters Θ are all the network parameters in the semi-supervised training phase, including the artificial extractor, encoder, labeled classifier, unlabeled classifier, labeled decoder and unlabeled decoder network parameters.

[0256] When the joint loss function converges or reaches a predetermined number of iterations, a trained small sample electromagnetic signal open set recognition model is obtained.

[0257] A multimodal feature extractor is taken out from the trained small sample electromagnetic signal open set recognition model, and a distance-based classifier is added thereto to obtain a trained electromagnetic signal open set recognition model, and classify the electromagnetic signals collected in real time.

[0258] The function of step S2 is to obtain a trained electromagnetic signal open set recognition model, and use the trained electromagnetic signal open set recognition model to perform unsupervised classification and recognition on the electromagnetic signal acquired in real time and power normalized, and determine whether its category is a known modulation type or an unknown class.

[0259] In summary, the open set recognition method for small sample electromagnetic signals based on multimodal features according to an embodiment of the present invention has the following beneficial effects:

[0260] 1. This invention adopts a semi-supervised learning method, which combines a small number of labeled samples with a large number of unlabeled samples for joint training. This solves the problem that traditional methods require a large amount of labeled data and significantly reduces the dependence on labeled samples.

[0261] 2. The multimodal feature extractor of the present invention integrates the low-dimensional discriminant features of the manual extractor (such as time domain moment kurtosis, frequency domain moment kurtosis, etc.) and the high-dimensional hidden features learned by the encoder, overcoming the limitations of a single feature, enhancing the model's ability to characterize complex signals, and fully exploring the multi-dimensional features of electromagnetic signals. Multimodal feature fusion improves feature discriminability and enhances the recognition accuracy of known categories;

[0262] 3. This invention optimizes category features through a category center optimizer and a relative entropy evaluator, and uses data augmentation technology to explore data invariance, thereby improving the model's generalization ability for unknown categories and noisy data. It also constrains the prediction consistency between original samples and noise-perturbed samples through the KL divergence loss, and combines the reconstruction loss to preserve the essential characteristics of the signal, significantly improving the model's stability in low signal-to-noise ratio environments.

[0263] 4. In the present invention, the unlabeled classifier generates pseudo labels for unlabeled samples and participates in the cross-entropy loss calculation. The relative entropy evaluator performs data enhancement by minimizing the KL divergence loss, making full use of unlabeled data and improving model performance. A four-element joint loss function of cross entropy loss, center loss, KL divergence loss, and reconstruction loss is designed to balance classification accuracy, intra-class compactness, data enhancement, and feature integrity, achieving efficient multi-objective optimization and reducing training resource consumption.

[0264] 5. The present invention breaks through the limitation of closed set assumption and realizes open set recognition. The classification strategy based on Mahalanobis distance combined with a predetermined threshold can effectively identify known category signals and detect unknown category signals, thereby enhancing the model's open set recognition capability in actual complex environments and adapting to scenarios where new signals dynamically appear in real electromagnetic environments.

[0265] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0266] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A small sample electromagnetic signal open set recognition method based on multimodal features, characterized by: include: Acquire electromagnetic signals in real time and perform power normalization to obtain the signal to be identified; Inputting the known class signal data set and the signal to be identified into the trained electromagnetic signal open set recognition model to perform unsupervised classification and recognition to obtain a recognition result; Among them, the electromagnetic signal open set recognition model is used to judge based on the Mahalanobis distance between the multimodal semantic feature vector corresponding to the signal to be identified and the center point vector of each known class signal in the known class signal data set to obtain the category of the signal to be identified.

2. The open set recognition method for small sample electromagnetic signals based on multimodal features according to claim 1 is characterized in that: The electromagnetic signal open set recognition model includes a multimodal feature extractor and a distance-based classifier; wherein, The modal feature extractor is used to extract the multimodal semantic feature vectors of the signal to be identified and each known class signal data in the known class signal data set, and calculate the center point vector of each known class signal based on the multimodal semantic feature vectors corresponding to the signal data in the known class signal data set; The distance-based classifier is used to calculate the Mahalanobis distance between the multimodal semantic feature vector corresponding to the signal to be identified and the center point vector of each known class signal, so as to determine the category of the signal to be identified.

3. The open set recognition method for small sample electromagnetic signals based on multimodal features according to claim 2, characterized in that: The multimodal feature extractor includes a manual extractor, an encoder, a feature fusion module and a category feature optimizer; The artificial extractor is used to obtain corresponding artificial semantic features based on the low-dimensional discriminant feature vectors of the input signal to be identified and the signal data in the known class signal data set respectively; The encoder is used to extract high-dimensional hidden features of the input signal to be identified and the signal data in the known class signal data set respectively; The feature fusion module is used to fuse the artificial semantic features and high-dimensional hidden features of the signal to be identified and the signal data in the known class signal dataset to obtain a multimodal semantic feature vector corresponding to the signal to be identified and the signal data in the known class signal dataset; The category feature optimizer is used to calculate the center point vector of each known class signal based on the multimodal semantic feature vector corresponding to the signal data in the known class signal data set.

4. The open set recognition method for small sample electromagnetic signals based on multimodal features according to claim 2, characterized in that: The category of the signal to be identified is determined based on the Mahalanobis distance and a predetermined threshold as follows: Among them, k is the kth known category; γ c is a predetermined threshold; is the multimodal semantic feature vector z corresponding to the signal to be identified i To the center point vector of the k-th known class signal Mahalanobis distance; When d min ≤γ c When y i is a known class, otherwise, determine y i It is an unknown class; If it is a known class, select The known category corresponding to the minimum value is the category of the signal to be identified.

5. The open set recognition method for small sample electromagnetic signals based on multimodal features according to claim 4 is characterized in that: Calculate the center point vector of each known class signal in the known class signal data set as follows: in, N l is the total number of signal data in the known class signal dataset; Encode the vector of the true category; Signal to be identified Multimodal semantic features; N c is the total number of known categories.

6. The open set recognition method for small sample electromagnetic signals based on multimodal features according to claim 1, characterized in that: The electromagnetic signal open set recognition model is obtained through the following training process: Preprocessing the electromagnetic signals including multiple modulation types generated by simulation to obtain a labeled sample data set and an unlabeled sample data set; wherein the label is the modulation type; Constructing a small sample electromagnetic signal open set recognition model; wherein the small sample electromagnetic signal open set recognition model comprises a multimodal feature extractor, a parallel labeled classifier and an unlabeled classifier, a relative entropy evaluator, a parallel labeled decoder and an unlabeled decoder; The labeled sample dataset and the unlabeled sample dataset are used to perform semi-supervised training on a small sample electromagnetic signal open set recognition model, including: Loading the labeled sample dataset and the unlabeled sample dataset into the multimodal feature extractor; Based on the joint loss function, the small sample electromagnetic signal open set recognition model is trained, and the model parameters are continuously adjusted and updated through back propagation and gradient descent optimization algorithms; The labeled classifier and the unlabeled classifier share parameters; the labeled decoder and the unlabeled decoder share parameters; Until the joint loss function converges or the training ends at a preset maximum number of iterations, the parameters of the small sample electromagnetic signal open set recognition model are saved to obtain a trained small sample electromagnetic signal open set recognition model; A multimodal feature extractor is taken out from the trained small sample electromagnetic signal open set recognition model, and a distance-based classifier is added thereto to obtain a trained electromagnetic signal open set recognition model.

7. The open set recognition method for small sample electromagnetic signals based on multimodal features according to claim 6, characterized in that: The unlabeled classifier obtains a corresponding multimodal semantic feature vector based on the unlabeled sample data in the unlabeled sample data set through the multimodal feature extractor, outputs a corresponding prediction probability vector, and uses the category index corresponding to the maximum probability in the prediction probability vector as a pseudo label for the unlabeled sample data. During the model training process, the pseudo label is regarded as the true label to participate in the cross entropy loss calculation; The labeled classifier is used to improve the classification accuracy of the labeled sample data set by using cross entropy loss during model training; The relative entropy evaluator performs data enhancement on the unlabeled sample dataset by minimizing the KL divergence loss based on the predicted probability vector output by the unlabeled classifier; The labeled decoder and the unlabeled decoder are used to upsample the multimodal semantic feature vector to reconstruct labeled sample data and unlabeled sample data, respectively. During the model training process, the labeled sample data and the unlabeled sample data are reconstructed by minimizing the reconstruction loss.

8. The open set recognition method for small sample electromagnetic signals based on multimodal features according to claim 7, characterized in that: The joint loss function L is as follows: L=λ ce L ce +λ cl L cl +λ kl L kl +λ re L re Among them, L ce is the cross entropy loss; L cl is the center loss; L kl is the KL divergence loss; L re is the reconstruction loss; ce ,λ cl ,λ kl ,λ re L ce 、L cl 、L kl 、L re The weight of .

9. The open set recognition method for small sample electromagnetic signals based on multimodal features according to claim 8, characterized in that: The relative entropy estimator performs data augmentation on unlabeled sample data by minimizing the KL divergence loss, including: In the unlabeled sample data set, the unlabeled sample data And the corresponding unlabeled sample data with added noise perturbation After passing through the multimodal feature extractor and the unlabeled classifier, the probability vector and calculate and The KL divergence loss is as follows: Among them, N u is the total number of signals in the unlabeled sample dataset; l kl (.) is the KL divergence loss function, y i for for For unlabeled sample data The probability value of belonging to the jth category; Represents unlabeled sample data with added noise perturbation The probability value of belonging to the jth category; H is the total number of categories.

10. The open set recognition method for small sample electromagnetic signals based on multimodal features according to claim 6, characterized in that: The preprocessing of the electromagnetic signals including multiple modulation types generated by simulation includes: intercepting the electromagnetic signal according to a preset length and performing power normalization to obtain a first electromagnetic signal set; Dividing the first electromagnetic signal set into a labeled sample data set and an unlabeled sample data set according to a preset ratio; Noise is injected into samples in the unlabeled sample dataset to perform data enhancement.

Images