A cross-domain few-shot specific emitter recognition method based on self-supervised learning and transfer learning
By employing self-supervised learning and transfer learning methods, and using von Neumann entropy and cosine similarity to train the model, the accuracy problem of identifying specific radiation sources in cross-domain scenarios was solved, achieving efficient identification under conditions of few samples and low cost.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DALIAN MARITIME UNIVERSITY
- Filing Date
- 2026-02-12
- Publication Date
- 2026-06-23
Smart Images
Figure CN122262784A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of radio signal identification, and more particularly to a cross-domain few-sample specific radiation source identification method based on self-supervised learning and transfer learning. Background Technology
[0002] With the rapid growth of IoT devices in critical infrastructure applications, security vulnerabilities in wireless networks are also increasing. Specific Source Identification (SEI) technology utilizes radio frequency fingerprints (RFF) for device authentication. These fingerprints originate from inherent physical layer defects in the transmitter and are difficult to forge, thus providing a reliable device authentication scheme. Traditional SEI methods rely on prior knowledge of specific signals, making it difficult to uncover the signal's latent features. In contrast, deep learning, as a data-driven method, has shown significant advantages in RFF identification. However, in real-world scenarios, the number of available samples for each type of device is often limited, easily leading to overfitting in these data-driven methods and consequently, performance degradation. In recent years, research on specific source identification has increasingly focused on data-constrained scenarios, incorporating methods such as metric learning, meta-learning, and unsupervised learning into its technical framework. Compared to supervised pre-training methods that require large amounts of labeled data and meta-learning methods with complex training processes, unsupervised learning offers advantages such as no labeled pre-training and low training overhead, demonstrating greater development potential.
[0003] Unsupervised learning is used in the upstream task pre-training stage of transfer learning, where unlabeled data is used as an auxiliary dataset to reduce data acquisition and labeling costs. However, most existing methods assume that the auxiliary dataset and the target dataset are in the same domain, requiring separate collection of auxiliary data for different SEI tasks, which is difficult to achieve in some non-cooperative SEI scenarios. To address these issues, this invention focuses on data reuse in cross-domain SEI tasks, specifically addressing the domain shift challenge caused when auxiliary and target data originate from different signal types. Compared to extractors trained using existing methods, the RF fingerprint extractor pre-trained based on the framework proposed in this invention exhibits superior recognition accuracy in cross-domain few-shot SEI tasks. Summary of the Invention
[0004] To address the technical challenges of existing few-shot radiation source identification methods, such as overfitting due to limited available data, the need for costly labeled data during pre-training, and difficulty in obtaining target domain data in non-cooperative scenarios, this invention proposes a cross-domain few-shot specific radiation source identification method based on self-supervised learning and transfer learning. In the model pre-training process, unlabeled data is used, and the model is trained in reverse using von Neumann entropy and cosine similarity. Labeled data is then used to fine-tune the model through a knowledge transfer encoder. This directly controls the eigenvalue distribution of the representation matrix during training, improving matrix rank and isotropy. The model retains more effective information during training, resulting in stronger adaptability in cross-domain generalization scenarios, reduced requirements for auxiliary datasets, and good identification accuracy with limited samples and low cost.
[0005] The technical means employed in this invention are as follows:
[0006] A method for identifying cross-domain few-shot specific radiation sources based on self-supervised learning and transfer learning includes the following steps: The wireless signal is collected and divided into an upstream task pre-training dataset and a downstream task fine-tuning dataset. The upstream task pre-training dataset is unlabeled data, and the unlabeled signal is generated by a device that is different from the downstream radiation source. A self-supervised joint embedding method is adopted, and the first model and the second model are pre-trained using the upstream task pre-training dataset. During the pre-training process, the autocorrelation matrix output by the first model is regularized using von Neumann entropy, the cosine similarity between the feature projections output by the first model and the second model is calculated, and the parameters of the first model are updated based on the regularized autocorrelation matrix and cosine similarity. The first model includes a first encoder and a first projection head connected together, and the second model includes a second encoder and a second projection head connected together. The pre-trained first encoder is transferred to the feature extractor. The downstream task fine-tuning dataset is input into the connected feature extractor and classification head to fine-tune the feature extractor and classification head. The downstream task fine-tuning dataset is labeled data. The wireless signal is input into the finely tuned feature extractor and classification head to obtain the predicted transmitter category of the signal.
[0007] Furthermore, the difference between upstream and downstream missions lies in the fact that they are different transmitter devices for the same signal type and transmitter devices for different signal types.
[0008] Further, the pre-training of the first model and the second model using the upstream task pre-training dataset includes: Two different data augmentation methods were used to augment the upstream task pre-training dataset to obtain a first augmented view and a second augmented view. The first enhanced view and the second enhanced view are input into the first encoder and the second encoder, respectively, to obtain the first feature vector and the second feature vector. Based on the first feature vector, an autocorrelation matrix is constructed. The first feature vector and the second feature vector are respectively input into the first projection head and the second projection head to obtain the first feature projection and the second feature projection. In each training batch, the autocorrelation matrix is updated based on the first feature vector and the first feature projection to maintain the exponential moving average of the autocorrelation matrix; Calculate the von Neumann entropy of the updated autocorrelation matrix to obtain the regularization loss term; Calculate the cosine similarity between the first feature projection and the second feature projection; A self-supervised training loss is constructed based on the regularization loss term and the cosine similarity, and the first encoder and the first projection head are optimized by minimizing the self-supervised training loss; The second encoder and the second projection head are updated from the first encoder and the first projection head via a momentum mechanism.
[0009] Furthermore, the data augmentation method includes rotation, flipping, and adversarial data augmentation, and the calculation formula for the rotated view is:
[0010] in, This is the rotated view. For rotation angle, It is an in-phase signal. Orthogonal signals For upstream task pre-training dataset, The formula for calculating the flipped view is:
[0011] in, To toggle the I-channel signal, This is to toggle the Q-channel signal.
[0012] Furthermore, the formula for calculating the regularization loss term is as follows:
[0013] in, This is the regularization loss term. These are the eigenvalues of the updated autocorrelation matrix. The updated autocorrelation matrix, This is the sample number.
[0014] Furthermore, the formula for calculating the self-supervised training loss is as follows:
[0015] in, For self-supervised training loss, The first feature projection after normalization. The second feature projection after normalization. Let be the weight of the von Neumann entropy regularization term. This is the regularization loss term. This is the expected operation.
[0016] Furthermore, the momentum parameter update formulas for the second encoder and the second projection head are as follows:
[0017] in, The target network parameters are given at time step t. The weights of the parameters of the second model itself. The target network parameters are given at time step t-1. These are the network parameters for the first model.
[0018] Furthermore, the formula for calculating the updated autocorrelation matrix is as follows:
[0019] in, Here is the updated autocorrelation matrix at time step t. Here is the updated autocorrelation matrix at time step t-1. For time steps, For the weight hyperparameters during the update, For batch size, The first feature projection is the result of normalization.
[0020] Compared with the prior art, the present invention has the following advantages: 1. This invention achieves high recognition accuracy with a small number of samples even when there are significant differences between pre-training and fine-tuning sample data. This can greatly reduce the requirements for auxiliary data and reduce the cost of data collection and labeling.
[0021] 2. This invention utilizes von Neumann entropy to control the eigenvalue distribution of the representation matrix, thereby improving the rank and isotropy of the matrix, which in turn improves the generalization and recognition accuracy of the model in downstream tasks.
[0022] 3. The method of this invention can directly control the distribution of eigenvalues of the matrix during training, which is beneficial to improving the matrix rank and isotropy, so that the model retains more effective information during training. Therefore, it exhibits stronger adaptability in cross-domain generalization scenarios, reduces the requirements for auxiliary datasets, and achieves better recognition accuracy with fewer samples and lower cost.
[0023] Based on the above reasons, this invention can be widely applied in fields such as radio signal identification. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 This is a flowchart of the method of the present invention.
[0026] Figure 2 This diagram illustrates the impact of different hyperparameters on the recognition accuracy of the method in this embodiment of the invention.
[0027] Figure 3 This is a schematic diagram comparing the accuracy of the method in this embodiment of the invention with existing methods. Detailed Implementation
[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0030] like Figure 1 As shown, this invention provides a cross-domain few-shot specific radiation source identification method based on self-supervised learning and transfer learning: the dataset is divided according to the type of the collected wireless signal. It includes a pre-training dataset from an upstream task. Downstream task fine-tuning dataset A transfer learning framework is established. Two data augmentation methods are randomly selected from the set of data augmentation methods to augment the signal. Processing is performed to obtain an enhanced view. and Input the enhanced views into the encoder separately. , and projector head , Generate corresponding representation and Before calculating the loss, the output of the projection head is normalized to a unit vector. and By minimizing the self-supervised training loss To optimize and Network parameters ,and and parameters Then, momentum encoder technology is used for updates. Simultaneously, during training, the encoder... and projector head The generated representation maintains the autocorrelation matrix. The exponential moving average; during the fine-tuning phase, the encoder This will be used as the initial RFF feature extractor. Moving to the target task, the extractor already contains prior signal information obtained during the pre-training phase. A new classification head is then added. The entire network is then fine-tuned using a small number of labeled samples. Finally, the data is input into the network to obtain the predicted class distribution of the input signal. The predicted transmitter category of the signal is obtained.
[0031] Specifically, the present invention includes the following steps: S1. Collect wireless signals and divide the wireless signals into upstream task pre-training datasets. Fine-tuning the dataset with downstream tasks The upstream task pre-training dataset is unlabeled data, and the unlabeled signals are generated by devices different from the downstream radiation sources.
[0032] Next, a transfer learning framework was established. Unlabeled signals from radiation source devices different from the downstream radiation sources were used when selecting pre-training data, indicating a domain difference between the two.
[0033] As a preferred embodiment of the present invention, the difference between the upstream and downstream tasks lies in the following: different transmitter devices for the same signal type, and transmitter devices for different signal types.
[0034] Cross-domain, few-sample, specific radiation source identification scenario: The basic task includes a large amount of unlabeled auxiliary data. The target task only contains a limited number of labeled samples. Used for fine-tuning, among which and They represent the first Each input signal and its corresponding label. The goal of specific radiation source identification is to find a mapping function. To make it in the target task Unknown category data This invention achieves maximized classification performance by implementing the mapping function using a deep neural network. Its optimization objective can be expressed as:
[0035] in, yes and The joint probability distribution, The cross-entropy loss function is typically used to measure the difference between the predicted and true classes of a sample. Because... Often unknown, therefore a training dataset is typically used. To replace it. However, when Number of samples In finite time, the model is prone to overfitting, leading to poor training performance. To alleviate this problem, an auxiliary dataset is introduced. Self-supervised learning is performed on the basic task to provide better initial model parameters for training the target task. The optimization objective in the pre-training phase can be expressed as:
[0036] in, This represents the initial model for migration to downstream tasks. This represents the loss function for self-supervised learning. Let represent the objective function of the basic task. Then, the model is fine-tuned on the objective task using a small training dataset to obtain the final model for radiation source identification.
[0037] S2. A self-supervised joint embedding method is adopted. The first model and the second model are pre-trained using the upstream task pre-training dataset. During the pre-training process, the autocorrelation matrix output by the first model is regularized using von Neumann entropy. The cosine similarity between the feature projections output by the first model and the second model is calculated. The parameters of the first model are updated based on the regularized autocorrelation matrix and the cosine similarity. The first model includes a first encoder and a first projection head connected together. The second model includes a second encoder and a second projection head connected together.
[0038] Specifically, the pre-training steps include: The first step involves augmenting the upstream task's pre-training dataset using two different data augmentation methods, specifically for the signal. Processing is performed to obtain the first enhanced view. Second Enhanced View .
[0039] Data augmentation methods include rotation, flipping, and adversarial data augmentation. The formula for calculating the rotated view is:
[0040] in, This is the rotated view. For rotation angle, It is an in-phase signal. Orthogonal signals This is a pre-training dataset for upstream tasks.
[0041] The formula for calculating the flipped view is:
[0042] in, To toggle the I-channel signal, This is to toggle the Q-channel signal.
[0043] Step 2: Input the first enhanced view and the second enhanced view into the first encoder respectively. Second encoder We obtain the first eigenvector and the second eigenvector, and construct the autocorrelation matrix based on the first eigenvector.
[0044] Step 3: Input the first feature vector and the second feature vector into the first projection head respectively. Second projection head The first feature projection is obtained. Second feature projection (in , ),and ).
[0045] Fourth step: In each training batch, update the autocorrelation matrix based on the first feature vector and the first feature projection to maintain the autocorrelation matrix. Exponential moving average.
[0046] Maintaining the autocorrelation matrix The exponential moving average, weighted by the current autocorrelation matrix and historical autocorrelation matrices, implicitly stores data, allowing the model optimization process to move beyond the limitations of current data. At time step... hour, Update using the following formula:
[0047] in, Here is the updated autocorrelation matrix at time step t. Here is the updated autocorrelation matrix at time step t-1. For time steps, For the weight hyperparameters during the update, For batch size, The first feature projection is the result of normalization.
[0048] Step 5: Calculate the von Neumann entropy of the updated autocorrelation matrix to obtain the regularization loss term.
[0049] Before calculating the loss, the first feature projection and the second feature projection are normalized using unit vectors to obtain the results. and The formula is:
[0050] in, The first feature projection after normalization. For the first feature projection, The normalized second feature projection, This is the projection of the second feature.
[0051] A self-supervised joint embedding method is used to pre-train the model, generating multiple augmented views for each sample and training the model to produce similar representations across different views. To prevent the model from collapsing into trivial constant solutions, the autocorrelation matrix of the neural network output is regularized using von Neumann entropy (VNE).
[0052] The von Neumann entropy is defined as the autocorrelation matrix. Eigenvalues The formula for calculating the Shannon entropy and the regularization loss term is:
[0053] in, This is the regularization loss term. These are the eigenvalues of the updated autocorrelation matrix. The updated autocorrelation matrix, This is the sample number.
[0054] Step 6: Calculate the cosine similarity between the first feature projection and the second feature projection.
[0055] Self-supervised training loss Using cosine similarity This measures the similarity between outputs and includes a regularization term. The mathematical expression is as follows:
[0056] in, For self-supervised training loss, The first feature projection after normalization. The second feature projection after normalization. Let be the weight of the von Neumann entropy regularization term. This is the regularization loss term. This is the expected operation.
[0057] Step 7: Construct a self-supervised training loss based on the regularization loss term and cosine similarity, and optimize the first encoder and the first projector by minimizing the self-supervised training loss.
[0058] Online network and The parameters are obtained through loss. Optimize using backpropagation.
[0059] Step 8: The second encoder and the second projection head are updated from the first encoder and the first projection head via a momentum mechanism.
[0060] Target Network and Updated using momentum encoding techniques. Given weights... In the case of time step At that time, the momentum parameter update formula for the second encoder and the second projection head is:
[0061] in, The target network parameters are given at time step t. The weights of the parameters of the second model itself. The target network parameters are given at time step t-1. These are the network parameters for the first model.
[0062] S3. Transfer the pre-trained first encoder to the feature extractor, and input the downstream task fine-tuning dataset into the connected feature extractor and classification head to fine-tune the pre-feature extractor and classification head. The downstream task fine-tuning dataset is labeled data.
[0063] The first pre-trained encoder serves as the initial RFF feature extractor. Moving to the target task, the extractor already contains prior signal information obtained during the pre-training phase. A new classification head is then added. and using a small number of labeled samples Fine-tune the entire network.
[0064] During the fine-tuning phase, the encoder This will be used as the initial RFF feature extractor. Moving to the target task, the extractor already contains prior signal information obtained during the pre-training phase. A new classification head is then added. and using a small number of labeled samples Fine-tune the entire network. The classifier's output is the predicted class distribution of the input signal. This stage employs the cross-entropy (CE) loss function to ensure that the model output fits the distribution of the real data in the target task. This loss function can be expressed as:
[0065] in, For the true category distribution, Let cross-entropy (CE) be the loss function. To predict the category distribution, This is the sample size.
[0066] S4. This model can be used to infer the transmitter category of a received signal. The wireless signal is input into a finely tuned feature extractor and classification head to obtain the predicted transmitter category of the signal (i.e., the predicted category distribution of the input signal). ).
[0067] The above steps S1 to S4 are executed in sequence.
[0068] like Figure 2 As shown, the effects of different hyperparameters on the model's recognition performance were explored, and a combination of hyperparameters that achieved better performance was selected.
[0069] like Figure 3 As shown, compared with several existing self-supervised training methods, the present invention achieves the best results.
[0070] This embodiment also includes a cross-domain few-shot specific radiation source identification method based on self-supervised learning and transfer learning, including: Module setup: Collect wireless signals of different types and from different transmitters to build a pre-trained dataset and a fine-tuning dataset, which are used for upstream and downstream tasks, respectively.
[0071] Generation module: used for generating wireless signal data Two data augmentation views are generated, and the robustness of the model training is enhanced by methods such as rotation and flipping.
[0072] Forward module: Two data augmentation views are input into the online network and the target network respectively for forward propagation to generate feature representations of the signal.
[0073] Update module: used to calculate the cosine similarity and von Neumann entropy between two outputs, obtain the self-supervised training loss, and perform backpropagation to update the parameters of the online network. The target network parameters are updated using momentum coding technology.
[0074] Fine-tuning module: Used to fine-tune the pre-trained network model using downstream datasets to make the model fit the target domain.
[0075] The module that generates the predicted class distribution of the input signal. This allows us to determine the type of transmitter.
[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for identifying specific radiation sources across domains with few samples based on self-supervised learning and transfer learning, characterized in that, Includes the following steps: The wireless signal is collected and divided into an upstream task pre-training dataset and a downstream task fine-tuning dataset. The upstream task pre-training dataset is unlabeled data, and the unlabeled signal is generated by a device that is different from the downstream radiation source. A self-supervised joint embedding method is adopted, and the first model and the second model are pre-trained using the upstream task pre-training dataset. During the pre-training process, the autocorrelation matrix output by the first model is regularized using von Neumann entropy, the cosine similarity between the feature projections output by the first model and the second model is calculated, and the parameters of the first model are updated based on the regularized autocorrelation matrix and cosine similarity. The first model includes a first encoder and a first projection head connected together, and the second model includes a second encoder and a second projection head connected together. The pre-trained first encoder is transferred to the feature extractor. The downstream task fine-tuning dataset is input into the connected feature extractor and classification head to fine-tune the feature extractor and classification head. The downstream task fine-tuning dataset is labeled data. The wireless signal is input into the finely tuned feature extractor and classification head to obtain the predicted transmitter category of the signal.
2. The method for cross-domain few-shot specific radiation source identification based on self-supervised learning and transfer learning according to claim 1, characterized in that, The difference between upstream and downstream missions lies in the fact that they are different transmitters for the same signal type and transmitters for different signal types.
3. The method for cross-domain few-shot specific radiation source identification based on self-supervised learning and transfer learning according to claim 1, characterized in that, The pre-training of the first and second models using the upstream task pre-training dataset includes: Two different data augmentation methods were used to augment the upstream task pre-training dataset to obtain a first augmented view and a second augmented view. The first enhanced view and the second enhanced view are input into the first encoder and the second encoder, respectively, to obtain the first feature vector and the second feature vector. Based on the first feature vector, an autocorrelation matrix is constructed. The first feature vector and the second feature vector are respectively input into the first projection head and the second projection head to obtain the first feature projection and the second feature projection. In each training batch, the autocorrelation matrix is updated based on the first feature vector and the first feature projection to maintain the exponential moving average of the autocorrelation matrix; Calculate the von Neumann entropy of the updated autocorrelation matrix to obtain the regularization loss term; Calculate the cosine similarity between the first feature projection and the second feature projection; A self-supervised training loss is constructed based on the regularization loss term and the cosine similarity, and the first encoder and the first projection head are optimized by minimizing the self-supervised training loss; The second encoder and the second projection head are updated from the first encoder and the first projection head via a momentum mechanism.
4. The method for cross-domain few-shot specific radiation source identification based on self-supervised learning and transfer learning according to claim 3, characterized in that, The data augmentation methods include rotation, flipping, and adversarial data augmentation. The calculation formula for the rotated view is as follows: in, This is the rotated view. For rotation angle, It is an in-phase signal. Orthogonal signals For upstream task pre-training dataset, The formula for calculating the flipped view is: in, To toggle the I-channel signal, This is to toggle the Q-channel signal.
5. The method for cross-domain few-shot specific radiation source identification based on self-supervised learning and transfer learning according to claim 3, characterized in that, The formula for calculating the regularization loss term is as follows: in, This is the regularization loss term. These are the eigenvalues of the updated autocorrelation matrix. The updated autocorrelation matrix, This is the sample number.
6. The method for cross-domain few-shot specific radiation source identification based on self-supervised learning and transfer learning according to claim 3, characterized in that, The formula for calculating the self-supervised training loss is as follows: in, For self-supervised training loss, The first feature projection after normalization. The second feature projection after normalization. Let be the weight of the von Neumann entropy regularization term. This is the regularization loss term. This is the expected operation.
7. The method for cross-domain few-shot specific radiation source identification based on self-supervised learning and transfer learning according to claim 3, characterized in that, The momentum parameter update formulas for the second encoder and the second projection head are as follows: in, The target network parameters are given at time step t. The weights of the parameters of the second model itself. The target network parameters are given at time step t-1. These are the network parameters for the first model.
8. The method for cross-domain few-shot specific radiation source identification based on self-supervised learning and transfer learning according to claim 3, characterized in that, The formula for calculating the updated autocorrelation matrix is as follows: in, Here is the updated autocorrelation matrix at time step t. Here is the updated autocorrelation matrix at time step t-1. For time steps, For the weight hyperparameters during the update, For batch size, The first feature projection is the result of normalization.