Radio frequency fingerprint identification general architecture and method based on multi-modal large model knowledge distillation, and electronic equipment

Abstract

The invention relates to the field of wireless sensing and monitoring identification, and particularly provides a radio frequency fingerprint identification general architecture and method based on multi-mode large model knowledge distillation, which are used for extracting and identifying radio frequency fingerprint features of a transmitter, improving identification accuracy and reducing reasoning delay, and comprises the following steps: firstly, collecting a baseband signal of a wired mode through optical fiber connection; the method comprises the steps of preprocessing multi-modal signal data through a data preprocessing mode of data slicing, domain transformation and RGB image generation, then inputting preprocessed wired modal and wireless modal baseband signals into a multi-modal large model, and extracting radio frequency fingerprint features through unsupervised learning; then, knowledge of the multi-modal large model is transmitted to the lightweight network model through knowledge distillation; and finally, carrying out fine tuning on the lightweight network model for a wireless scene, and deploying the lightweight network model to an edge Internet of Things device for identification and authentication of the Internet of Things device. According to the scheme, the accuracy, robustness and model expandability of radio frequency fingerprint identification are improved.

Description

Technical Field

[0001] The present invention belongs to the field of physical layer security of wireless communication, and particularly relates to an identity recognition and authentication of a transmitter based on a general architecture of radio frequency fingerprint recognition by multi-modal large model knowledge distillation. Background Art

[0002] Due to the openness and broadcast nature of wireless networks, electromagnetic signals are not easily protected by traditional cryptography. Malicious Internet of Things (IoT) device terminals such as drones and smart car keys held by illegal users can invade wireless networks at any time by modifying the Media Access Control (MAC) address, transmitting high-power forged signals, etc., and further launch Denial Of Service (DoS) / Distributed DenialOf Service (DDoS) attacks, form DDoS botnets, obtain device control permissions, execute malicious code, etc.

[0003] Although such threats can be mitigated by upper-layer identity authentication, the influx of a large number of heterogeneous IoT devices into wireless networks has brought huge pressure to traditional identity verification methods (such as IoT SIM cards), and largely cannot meet the requirements of edge intelligent applications for low latency and lightweight. Radio Frequency Fingerprint (RFF) originates from the inherent hardware defects of devices. In terms of radio frequency components, RFF can be divided into frequency offset, in-phase and quadrature (I / Q) imbalance, and power amplifier nonlinearity, etc. RFF exists at the beginning of device manufacturing and is an identity credential beyond the bit level, with the advantages of randomness, uniqueness, and non-clonability. RFF is directly extracted from the physical layer signals of wireless communication without going through upper-layer communication protocol processing, and has the significant advantages of low latency and lightweight. Therefore, radio frequency fingerprint recognition provides a promising zero-trust solution for IoT identity authentication. However, during the wireless propagation of signals, multipath fading and Doppler frequency shift of the wireless channel are usually mixed with RFF, seriously affecting the accuracy of radio frequency fingerprint recognition. It is urgent to mitigate the impact of channel effects on radio frequency fingerprint recognition through effective methods to improve device recognition accuracy and robustness in complex wireless environments.

[0004] As the latest artificial intelligence technology, the large language model (LLM) is highly versatile and is regarded as the next wave of AI innovator, attracting extensive attention from academia and industry. Currently, the LLM has achieved remarkable success in fields such as AI assistants, text and image generation, software development, and healthcare. Combining radio frequency fingerprinting (RFF) with the LLM can utilize the LLM's powerful automatic feature extraction and recognition capabilities to extract and identify the RFF of wireless Internet of Things (IoT) devices, thereby effectively improving the accuracy of device identification. Meanwhile, knowledge distillation technology can compress the parameters of complex LLMs, enabling the RFF model based on the LLM to be effectively deployed on edge IoT devices, achieving lightweight and low-latency device identification and authentication. Summary of the Invention

[0005] Objective of the Invention: The technical problem to be solved by the present invention is to provide a general architecture for radio frequency fingerprint recognition based on multi-modal large model knowledge distillation, aiming to solve the problems of low accuracy, poor robustness, and weak model scalability of radio frequency fingerprint recognition in different wireless communication scenarios through a general training method.

[0006] Technical Solution:

[0007] To achieve the above objective, the technical solution of the present invention is as follows. A general architecture for radio frequency fingerprint recognition based on multi-modal large model knowledge distillation includes several types of wireless devices, a multi-modal signal data acquisition module, a data preprocessing module, a large language model (LLM) training module, a knowledge distillation module, a lightweight model fine-tuning module, and a device authentication module. The several types of wireless devices are several communication nodes in a wireless communication scenario. The multi-modal signal data acquisition module is located behind the several types of wireless devices. The data preprocessing module is located behind the multi-modal signal data acquisition module. The LLM model training module is located behind the data preprocessing module. The knowledge distillation module is located behind the LLM model training module. The lightweight model fine-tuning module is located behind the knowledge distillation module. The device authentication module is located behind the lightweight model fine-tuning module. This architecture can be applied to many wireless communication physical layer security protection scenarios, including IoT device identity authentication, satellite GPS signal authentication, broadcast ADS-B signal authentication, connected vehicle key authentication, identity recognition and monitoring of drones or intelligent vehicles, wireless network intrusion detection, and smart home security protection.

[0008] A method for radio frequency fingerprint recognition based on multi-modal large model knowledge distillation, the method comprising the following steps:

[0009] Step 1. Collect baseband signals in the wired mode through optical fiber connection and simultaneously collect baseband signals in the wireless mode through wireless connection to construct a multi-modal signal data set;

[0010] Step 2. Preprocess the multi-modal signal data through data slicing, domain transformation, and RGB image generation-based data preprocessing methods. Convert the baseband signal into semantics through word segmentation for network model training;

[0011] Step 3. Input the preprocessed wired and wireless modal baseband signals into the multi-modal LLM, and extract radio frequency fingerprint features through unsupervised learning;

[0012] Step 4. Transfer the knowledge of the multi-modal LLM to a lightweight network model through knowledge distillation;

[0013] Step 5. Fine-tune the lightweight network model for specific wireless scenarios, and deploy the fine-tuned lightweight model to edge Internet of Things devices for device identification and authentication

[0014] In this architecture, the signal r(t) received by the authentication gateway is

[0015] r(t) = h(t) * f RFF (s(t)) + n(t),

[0016] where h(t) represents the channel effect including multipath fading and Doppler shift, f RFF represents the non-linear distortion caused by device hardware defects, s(t) represents the transmitted signal, and n(t) represents the Gaussian white noise in the wireless channel.

[0017] The signal data in the dataset includes the baseband signals of the wired mode and the wireless mode. The same data preprocessing method is used for both signals. By slicing the data, each data segment has the same length. After data slicing, the wired mode dataset D wired and the wireless mode dataset D wireless can be obtained, where the wired baseband signal is x (x ∈ D wired ), and the wireless baseband signal is During training, preprocess the multi-modal signal data through data preprocessing methods such as domain transformation and RGB image generation. Convert the baseband signal into semantics through word segmentation for network model training.

[0018] The model training module can be expressed as

[0019]

[0020] where W LLM represents the weight parameters of the LLM, D LLM represents the dataset of the LLM, which is composed of pairs of wired baseband signals and wireless baseband signals, denotes the self-supervised contrastive loss (SCL), f LLM denotes the mapping from the data of the LLM to features. The purpose of the model training module is to update the parameters when the input is x and to minimize the SCL.

[0021] When the LLM training is completed, knowledge distillation can be expressed as

[0022]

[0023] where, W Light denotes the weight parameters of the lightweight model, D Light denotes the dataset used for lightweight model training, which is composed of wireless baseband signals, denotes the mean squared error loss function, f Light denotes the mapping from the data of the lightweight model to RFF features. The purpose of the knowledge distillation module is to update the lightweight model parameters when the input is to minimize the mean squared error loss between the features extracted by the lightweight model and those of the LLM.

[0024] Through the knowledge distillation module, the knowledge of the LLM is transferred to the lightweight model, and then it enters the lightweight model fine-tuning module. By adding a classifier at the end of the lightweight model, the model can be fine-tuned. The mathematical expression is

[0025]

[0026] where, D fine-tuning denotes the fine-tuning dataset, which is composed of wireless baseband signals and their corresponding device labels y, denotes the cross entropy loss (CEL). The purpose of the fine-tuning module is to perform a small amount of training adjustment on the lightweight model according to the current specific wireless environment, so that the lightweight model has better performance on the current dataset.

[0027] The fine-tuned lightweight model can be deployed on edge IoT devices to complete the construction of the final device authentication module. The radio frequency fingerprint features extracted in the device authentication module will be automatically recognized as device numbers. Deploying the device authentication module on edge IoT devices, the model has a low number of parameters, fast inference speed, and at the same time has the feature extraction performance of the LLM. This architecture is not limited to a specific wireless communication protocol and can support multiple communication protocols such as LoRa, WIFI, and 5G NR.

[0028] The LLM and lightweight model described in the architecture can be selected and replaced according to the actual application scenario, and are not limited to a specific model structure.

[0029] This architecture takes into account the insufficiency of single-modal data information. In the design of the LLM training module, joint training of multi-modal data is considered, and the LLM is used to effectively extract features, improving the device recognition accuracy and robustness in complex wireless environments.

[0030] The unsupervised learning used in LLM training includes self-supervised contrastive learning. The applicable scope of this architecture includes radio frequency fingerprint recognition tasks, as well as source-channel coding and decoding, channel estimation, and channel state information feedback tasks in the field of wireless communication.

[0031] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation described above.

[0032] A computer-readable storage medium stores computer instructions, which, when executed by a processor, implement the radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation described above.

[0033] Advantages: This architecture is not limited to a specific wireless communication protocol and can support multiple communication protocols such as LoRa, WIFI, and 5GNR. The LLM and lightweight models in the architecture can be replaced according to actual scenario requirements and are not limited to a specific model structure. The self-supervised contrastive learning method of the LLM can also be flexibly replaced with other unsupervised learning methods as needed, with strong model scalability. This architecture takes into account the insufficiency of single-modal data information, and joint training of multi-modal data is considered in the design of the model training module, enabling the LLM to effectively extract RFF features under the influence of channel effects, improving the device recognition accuracy and robustness in complex wireless environments. Knowledge distillation uses a combination of soft labels and hard labels to fully transfer the knowledge of the teacher large model to the student lightweight model. In a specific scenario, the parameter quantity of the lightweight model after knowledge distillation can be reduced by more than 80%, the inference speed is significantly accelerated, device recognition at the millisecond level can be achieved, and at the same time, it is close to the feature extraction performance of the LLM, with a performance loss of less than 3%. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is an application scenario diagram provided by an embodiment of the present invention;

[0035] Figure 2 is a diagram of the general architecture and model training method provided by an embodiment of the present invention.

[0036] Figure 3 is a diagram of the radio frequency fingerprint recognition device provided by an embodiment of the present invention.

[0037] Figure 4It is a schematic diagram of the lightweight device for radio frequency fingerprint recognition provided by an embodiment of the present invention. Detailed implementation manners

[0038] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] Embodiment: In an embodiment of the present invention, as Figure 1 shown in the application scenario diagram of the general architecture for radio frequency fingerprint recognition based on multi-modal large model knowledge distillation, which includes a number of fixed devices, a number of mobile devices, and an authentication gateway for accessing the core network. The fixed devices and mobile devices send network access requests to the authentication gateway through the ALOHA protocol. The authentication gateway, fixed devices, and mobile devices are generally resource-constrained edge Internet of Things devices. In a wireless propagation environment, due to the reflection and occlusion of electromagnetic waves by objects such as buildings and trees, the wireless channel h(t) includes channel effects of multipath fading and Doppler frequency shift. In this case, the signal r(t) received by the authentication gateway is

[0040] r(t) = h(t) * f RFF (s(t)) + n(t),

[0041] where h(t) represents the channel effect including multipath fading and Doppler frequency shift, f RFF represents the non-linear distortion caused by device hardware defects, s(t) represents the transmitted signal, and n(t) represents the Gaussian white noise in the wireless channel.

[0042] In the training stage, the baseband signals of the fixed devices and mobile devices are collected. Among them, the baseband signals in the wired mode are collected through optical fiber connections, and at the same time, the baseband signals in the wireless mode are collected through wireless connections to construct a multi-modal signal dataset. For the two signals in the multi-modal signal dataset, the same data preprocessing method is used. By slicing the data, each data segment maintains the same length. After the data slicing process, the wired mode dataset D wired and the wireless mode dataset D wireless can be obtained, where the wired baseband signal is x (x ∈ D wired ), and the wireless baseband signal is The two constructed datasets are used for the subsequent training of the model.

[0043] The multi-modal signal data is preprocessed through data preprocessing methods such as domain transformation and RGB image generation. The baseband signal is converted into semantics for model training through word segmentation, and its loss function can be expressed as

[0044]

[0045] Among them, W LLM represents the weight parameters of the LLM, and D LLM represents the dataset of the LLM, which consists of pairs of wired baseband signals and wireless baseband signals. represents the SCL loss, and f LLM represents the mapping from the data of the LLM to features. The purpose of the model training module is to update the parameters when the input is x and to achieve the minimum SCL loss.

[0046] In order to design a lightweight model to meet the requirements of edge Internet of Things devices and retain the feature extraction performance of the LLM as much as possible, knowledge distillation is carried out after the LLM training is completed. The optimization problem can be expressed as

[0047]

[0048] Among them, W Light represents the weight parameters of the lightweight model, and D Light represents the dataset used for lightweight model training, which consists of wireless baseband signals. represents the mean squared error loss function, and f Light represents the mapping from the data of the lightweight model to features. The purpose of the knowledge distillation module is to update the lightweight model parameters when the input is to minimize the mean squared error loss between the features extracted by the lightweight model and those of the LLM.

[0049] Through the knowledge distillation module, the knowledge of the LLM is transferred to the lightweight model designed according to the actual scenario requirements. Next, it enters the lightweight model fine-tuning module. By adding a classifier at the end of the lightweight model, the model can be fine-tuned. The mathematical expression is

[0050]

[0051] Among them, D fine-tuning represents the fine-tuning dataset, which consists of wireless baseband signals and their corresponding device labels y. represents the CEL loss. The purpose of the fine-tuning module is to perform a small amount of training adjustment on the lightweight model according to the current specific wireless environment, so that the lightweight model has better performance on the current dataset.

[0052] The fine-tuned lightweight model is the device authentication module, which can realize the lightweight deployment of the gateway. Through software and hardware integration, the development of lightweight authentication devices can be completed, such as Figure 4Compared with LLM models, the fine-tuned lightweight model has extremely few model parameters, extremely low memory occupancy, faster inference speed, and equivalent or even better feature extraction performance than LLM. When an unknown device accesses the authentication gateway, the gateway receives the wireless signal of the unknown device and extracts the radio frequency fingerprint features, and the extracted radio frequency fingerprint features will be automatically recognized as the device number.

[0053] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A general architecture for radio frequency fingerprint recognition based on knowledge distillation of multimodal large models, characterized in that, It includes wireless devices of several models, a multi-modal signal data acquisition module, a data preprocessing module, a large language model (LLM) training module, a knowledge distillation module, a lightweight model fine-tuning module, and a device authentication module. The several models of wireless devices are several communication nodes in a wireless communication scenario. The multi-modal signal data acquisition module is located at the back end of the several models of wireless devices. The data preprocessing module is located at the back end of the multi-modal signal data acquisition module. The LLM model training module is located at the back end of the data preprocessing module. The knowledge distillation module is located at the back end of the LLM model training module. The lightweight model fine-tuning module is located at the back end of the knowledge distillation module. The device authentication module is located at the back end of the lightweight model fine-tuning module.

2. A radio frequency fingerprint recognition method based on knowledge distillation of multi-modal large models, characterized in that Adopt the general architecture of radio frequency fingerprint recognition based on multi-modal large model knowledge distillation described in claim 1. The method includes the following steps: Step 1. Collect the baseband signals of the wired mode through optical fiber connection and collect the baseband signals of the wireless mode through wireless connection at the same time to construct a multi-modal signal data set. Step 2. Preprocess the multi-modal signal data through data slicing, domain transformation, and RGB image generation data preprocessing methods. Convert the baseband signal into semantics through word segmentation for network model training. Step 3. Input the preprocessed wired-mode and wireless-mode baseband signals into the multi-modal LLM, and extract radio frequency fingerprint features through unsupervised learning. Step 4. Transfer the knowledge of the multi-modal LLM to a lightweight network model through knowledge distillation. Step 5. Fine-tune the lightweight network model for a specific wireless scenario, and deploy the fine-tuned lightweight model to edge Internet of Things devices for device identification and authentication.

3. The radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation according to claim 2, wherein In this architecture, the signal r(t) received by the authentication gateway is r(t) = h(t) * f RFF (s(t)) + n(t), where h(t) represents the channel effect including multipath fading and Doppler shift, f RFF represents the non-linear distortion caused by device hardware defects, s(t) represents the transmitted signal, and n(t) represents the Gaussian white noise in the wireless channel.

4. The radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation according to claim 3, wherein, The signal data in the dataset includes the baseband signal in the wired mode and the baseband signal in the wireless mode. The same data preprocessing method is used for both signals. By slicing the data, each data segment has the same length. After the data slicing process, the wired-mode dataset D wired and the wireless-mode dataset D wireless are obtained, where the wired baseband signal is x (x ∈ D wired ), and the wireless baseband signal is During training, the multi-modal signal data is preprocessed through data preprocessing methods such as domain transformation and RGB image generation. The baseband signal is converted into semantics through word segmentation for network model training. The model training module is represented as Among them, W LLM represents the weight parameters of the LLM, and D LLM represents the dataset of the LLM, which consists of pairs of wired baseband signals and wireless baseband signals. represents the Self-supervised Contrastive Loss (SCL), and f LLM represents the mapping from the data of the LLM to features. The purpose of the model training module is to update the parameters when the input is x and to achieve the minimum SCL.

5. The radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation according to claim 4, wherein When the LLM training is completed, knowledge distillation is performed, which is represented as Among them, W Light represents the weight parameters of the lightweight model, D Light represents the dataset used for lightweight model training, which is composed of wireless baseband signals, represents the mean squared error loss function, f Light represents the mapping of the data of the lightweight model to the RFF features. The purpose of the knowledge distillation module is to update the lightweight model parameters when the input is to minimize the mean squared error loss between the features extracted by the lightweight model and the LLM.

6. The radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation according to claim 2, characterized in that Through the knowledge distillation module, the knowledge of the LLM is transferred to the lightweight model. Next, enter the lightweight model fine-tuning module. By adding a classifier at the end of the lightweight model, the model is fine-tuned. The mathematical expression is Among them, D fine-tuning represents the fine-tuning dataset, which consists of wireless baseband signals and their corresponding device labels y, represents the Cross Entropy Loss (CEL). The purpose of the fine-tuning module is to perform a small amount of training adjustment on the lightweight model according to the current specific wireless environment, so that the lightweight model has better performance on the current dataset. The fine-tuned lightweight model can be deployed on resource-constrained edge Internet of Things devices to complete the construction of the final device authentication module. The radio frequency fingerprint features extracted in the device authentication module will be automatically recognized as device numbers.

7. The radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation according to claim 2, wherein, Deploy the device authentication module on edge Internet of Things devices. The model has a low number of parameters and a fast inference speed, and at the same time is close to the feature extraction performance of the LLM.

8. The radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation according to claim 2, wherein The unsupervised learning used in LLM training includes self-supervised contrast learning. The applicable scope of this architecture includes radio frequency fingerprint recognition tasks, as well as source-channel coding and decoding, channel estimation, and channel state information feedback tasks in the field of wireless communication.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation described in any one of claims 2 to 8 above.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instruction is executed by the processor, it implements the radio frequency fingerprint recognition method based on multi-modal large model knowledge distillation described in any one of claims 2-8.

Images