Construction method of radio frequency fingerprint identification model, identification method, system and equipment

Abstract

The invention provides a radio frequency fingerprint identification model construction method, identification method, system and device, and the method comprises the steps: carrying out the preprocessing of radio frequency signals in a radio frequency signal set, and obtaining a sample spectrum image set; performing feature extraction and feature shuffling on the sample spectrograms in the sample spectrogram set to obtain a feature vector set; performing batch normalization processing on the feature vectors in the feature vector set to obtain a first feature representation set; performing linear transformation and category mapping on first feature representations in the first feature representation set to obtain a second feature representation set; and based on the first feature representation set and the second feature representation set, training by adopting a multi-task learning mode combining metric learning and representation learning to obtain a radio frequency fingerprint identification model. According to the invention, the signal identification accuracy and robustness of the constructed radio frequency fingerprint identification model can be improved.

Description

Technical Field

[0001] The present invention relates to the field of neural networks and wireless communications, and in particular to a construction method, an identification method, a system and a device for a radio frequency fingerprint identification model. Background Art

[0002] With the rapid development of artificial intelligence and machine learning technologies, the application of the Internet of Things (IoT) has become increasingly popular. IoT security involves the policies and procedures adopted in this environment to protect devices, networks, and data from unauthorized access, use, disclosure, destruction, or tampering. Among them, secure access and authentication of devices is a key link in information security and the first line of defense for information protection. However, how to accurately identify a large number of terminal devices of the same model still faces major challenges.

[0003] In the related technologies, due to the limitations of traditional device authentication methods, multi-device interference and security issues of low-cost devices, Radio Frequency Fingerprint Identification (RFFI) has become an urgently needed technology. The current research on RFFI based on physical signals is mainly divided into two schools: traditional feature engineering-based methods and deep learning-based methods. Among them: Traditional feature engineering-based methods often fail to meet the requirements of high precision and high real-time performance when facing complex wireless environments. With the advancement of artificial intelligence technology, models based on deep learning can automatically extract signal features and improve the accuracy of signal classification and recognition, and are gradually becoming a research hotspot. However, the differences between features of devices of the same model are very small, and it is difficult to extract effective features with sufficient discrimination, resulting in increased recognition difficulty and reduced accuracy. In addition, many previous studies regard RFFI as a single-task end-to-end classification task. This method cannot accurately capture the subtle differences between signals during the training process, nor can it conduct a deeper analysis of RF fingerprints. Summary of the invention

[0004] In order to solve the problems of the prior art, the present invention proposes a construction method, an identification method, a system and a device for a radio frequency fingerprint recognition model, aiming to improve the accuracy and robustness of the recognition signal of the constructed radio frequency fingerprint recognition model.

[0005] The purpose of the present invention is achieved by adopting the following technical solutions:

[0006] In one aspect, the present invention provides a method for constructing a radio frequency fingerprint recognition model, the method comprising:

[0007] Preprocessing the radio frequency signals in the radio frequency signal set to obtain a sample spectrum atlas; wherein the radio frequency signal set includes: a radio frequency signal emitted by each sample device in a plurality of groups of sample devices of the same model;

[0008] Performing feature extraction and feature shuffling on the sample spectrum graphs in the sample spectrum graph set to obtain a feature vector set;

[0009] Performing batch normalization processing on the feature vectors in the feature vector set to obtain a first feature representation set;

[0010] Performing linear transformation and category mapping on the first feature representation in the first feature representation set to obtain the second feature representation set; wherein each second feature representation in the second feature representation set corresponds to a predicted label;

[0011] Based on the first feature representation set and the second feature representation set, a multi-task learning method combining metric learning and representation learning is adopted to perform training to obtain a radio frequency fingerprint recognition model.

[0012] Optionally, the preprocessing of the radio frequency signals in the radio frequency signal set to obtain a sample spectrum atlas includes:

[0013] Preprocessing the received radio frequency signals in the radio frequency signal set to obtain a standardized signal set;

[0014] By using short-time Fourier transform, time-frequency analysis is performed on the standardized signals in the standardized signal set to obtain the sample frequency spectrum set with two-dimensional channels being independent.

[0015] Optionally, the extracting and shuffling features of the sample spectrograms in the sample spectrogram set to obtain a feature vector set includes:

[0016] Using a feature extraction layer in an N-layer residual convolutional neural network with a fully connected layer deleted, feature extraction is performed on the sample spectrograms in the sample spectrogram set to obtain an initial feature set; wherein N is an integer greater than 1;

[0017] Using a shuffling strategy, shuffling the initial features in the initial feature set to obtain a shuffled feature set;

[0018] The average pooling layer of the N-layer residual convolutional neural network is used to average pool the shuffled features in the shuffled feature set to obtain the feature vector set.

[0019] Optionally, each initial feature in the initial feature set is a multi-dimensional feature, and the shuffling strategy is used to shuffle the initial features in the initial feature set to obtain a shuffled feature set, including:

[0020] Determine a first eigenvalue, a plurality of intermediate eigenvalues, and a plurality of eigenvalues ​​to be moved that sequentially constitute each initial feature in the initial feature set;

[0021] The multiple intermediate eigenvalues ​​and the multiple eigenvalues ​​to be moved are moved respectively, and the shuffled features of each initial feature formed in the order of the first eigenvalue, the multiple eigenvalues ​​to be moved and the multiple intermediate eigenvalues ​​are determined to obtain the shuffled feature set.

[0022] Optionally, the training is performed based on the first feature representation set and the second feature representation set by adopting a multi-task learning method combining metric learning and representation learning to obtain a radio frequency fingerprint recognition model, including:

[0023] Respectively obtaining a first loss function for characterizing the metric learning and a second loss function for characterizing the representation learning and carrying label smoothing;

[0024] Using the first loss function, determining the first loss corresponding to the similarity between multiple first feature representations in the first feature representation set;

[0025] Using the second loss function, determine the second loss corresponding to the cosine similarity between the predicted label corresponding to the second feature representation in the second feature representation set and the sample label corresponding to the second feature representation;

[0026] The first loss and the second loss are used for training to obtain the radio frequency fingerprint recognition model.

[0027] Correspondingly, the present invention also provides a system for constructing a radio frequency fingerprint recognition model, the system comprising:

[0028] A preprocessing module is used to preprocess the radio frequency signals in the radio frequency signal set to obtain a sample spectrum atlas; wherein the radio frequency signal set includes: a radio frequency signal emitted by each sample device in a plurality of groups of sample devices of the same model;

[0029] A feature extraction module, used for performing feature extraction and feature shuffling on the sample spectrum graphs in the sample spectrum graph set to obtain a feature vector set;

[0030] An embedding head module, used for performing batch normalization processing on the feature vectors in the feature vector set to obtain a first feature representation set;

[0031] A classification head module, configured to perform linear transformation and category mapping on the first feature representation in the first feature representation set to obtain the second feature representation set; wherein each second feature representation in the second feature representation set corresponds to a predicted label;

[0032] The multi-task learning module is used to perform training based on the first feature representation set and the second feature representation set by adopting a multi-task learning method combining metric learning and representation learning to obtain a radio frequency fingerprint recognition model.

[0033] Optionally, the preprocessing module includes:

[0034] A preprocessing unit, used for preprocessing the received radio frequency signal set to obtain a standardized signal set;

[0035] The time-frequency analysis unit is used to perform time-frequency analysis on the standardized signals in the standardized signal set by using short-time Fourier transform to obtain the sample spectrum atlas set independent of the two-dimensional channel.

[0036] Optionally, the feature extraction module includes:

[0037] The feature extraction layer in the N-layer residual convolutional neural network of the fully connected layer is deleted to extract features from the sample spectrograms in the sample spectrogram set to obtain an initial feature set; wherein N is an integer greater than 1;

[0038] A shuffling unit, configured to perform feature shuffling on the initial features in the initial feature set using a shuffling strategy to obtain a shuffled feature set;

[0039] The average pooling layer of the N-layer residual convolutional neural network is used to average pool the shuffled features in the shuffled feature set to obtain the feature vector set.

[0040] Optionally, each initial feature in the initial feature set is a multi-dimensional feature, and the shuffling unit is further used to determine a leading eigenvalue, multiple intermediate eigenvalues, and multiple eigenvalues ​​to be moved that sequentially constitute each initial feature in the initial feature set; the multiple intermediate eigenvalues ​​and the multiple eigenvalues ​​to be moved are moved respectively, and the shuffled features of each initial feature constituted in the order corresponding to the leading eigenvalue, the multiple eigenvalues ​​to be moved, and the multiple intermediate eigenvalues ​​are determined to obtain the shuffled feature set.

[0041] Optionally, the multi-task learning module includes:

[0042] A loss function acquisition unit, used to respectively acquire a first loss function for characterizing the metric learning and a second loss function for characterizing the representation learning and carrying label smoothing;

[0043] a loss determination unit, configured to use the first loss function to determine the first loss corresponding to the similarity between multiple first feature representations in the first feature representation set; and use the second loss function to determine the second loss corresponding to the cosine similarity between the predicted label corresponding to the second feature representation in the second feature representation set and the sample label corresponding to the second feature representation;

[0044] A training unit is used to use the first loss and the second loss to perform training to obtain the radio frequency fingerprint recognition model.

[0045] On the other hand, the present invention also provides a method for identifying a radio frequency signal, the method comprising:

[0046] Acquire a radio frequency signal to be identified;

[0047] The radio frequency signal to be identified is input into the radio frequency fingerprint recognition model constructed by any of the above-mentioned construction methods to identify the device to which the radio frequency signal to be identified belongs.

[0048] Correspondingly, the present invention also provides a radio frequency signal recognition system, the system comprising:

[0049] An acquisition module, used for acquiring a radio frequency signal to be identified;

[0050] The identification module is used to input the radio frequency signal to be identified into the radio frequency fingerprint identification model constructed by any of the above-mentioned construction systems to identify the device to which the radio frequency signal to be identified belongs.

[0051] In another aspect, the present application further provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0052] The memory is used to store one or more programs;

[0053] When the one or more programs are executed by the at least one processor, any of the above-described methods for constructing a radio frequency fingerprint recognition model or the above-described method for recognizing a radio frequency signal is implemented.

[0054] On the other hand, the present invention also provides a readable storage medium having an execution program stored thereon, and when the execution program is executed, the method for constructing a radio frequency fingerprint recognition model as described above or the method for recognizing a radio frequency signal as described above is implemented.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The method and system for constructing a radio frequency fingerprint recognition model provided by the present invention firstly perform preprocessing, feature extraction and feature shuffling on a plurality of radio frequency signal sets emitted by sample devices of the same model in sequence to obtain a feature vector set; then, batch normalization, linear transformation and category mapping are performed on the obtained feature vector set to obtain a first feature representation set and a second feature representation set for training; finally, based on the first feature representation set and the second feature representation set, training is performed by means of a multi-task learning method combining metric learning and representation learning to obtain a radio frequency fingerprint recognition model. In this way, on the one hand, based on feature shuffling, perturbation processing can be performed on the extracted features, thereby increasing the difference between features, and thus improving the generalization ability of the subsequently constructed radio frequency fingerprint recognition model; on the other hand, a multi-task learning framework is constructed by combining metric learning and representation learning, which can realize that the distribution of the feature vectors output by the subsequently obtained radio frequency fingerprint recognition model in the feature space is more compact, and at the same time, the ability to distinguish between class features is enhanced, that is, the slight differences between signals are accurately captured in the model training link, thereby improving the accuracy and robustness of the constructed radio frequency fingerprint recognition model in recognizing signals.

[0057] The radio frequency signal recognition method and system provided by the present invention can identify the radio frequency signal to be identified by using a radio frequency fingerprint recognition model that has feature shuffling and is constructed with the help of metric learning and representation learning, and can more accurately and efficiently determine the device to which the radio frequency signal to be identified belongs; that is, it can achieve: optimizing signal feature extraction, that is, automatically extracting effective features in the radio frequency signal to be identified through the radio frequency fingerprint recognition model constructed by the deep learning model, thereby enhancing the accuracy and robustness of the recognition of the radio frequency signal to be identified; improving system adaptability, and being able to dynamically adjust the radio frequency fingerprint recognition model according to different wireless environments and application requirements by means of a flexible neural network architecture to cope with complex and changeable signal interference; improving transmission efficiency: through intelligent signal processing, the transmission rate and reliability of the wireless communication system can be improved to meet the high performance requirements of future communication technologies; promoting interconnection between devices: realizing efficient connection and communication of various wireless devices, and promoting the development of application scenarios such as the Internet of Things and smart homes.

[0058] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions provided by the embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0060] Figure 1 The spectrum diagram corresponding to the RF signals emitted by two devices of the same model;

[0061] Figure 2 A schematic diagram of a flow chart of a method for constructing a radio frequency fingerprint recognition model provided by an embodiment of the present invention;

[0062] Figure 3 A schematic diagram of an execution flow of an N-layer residual convolutional neural network with fully connected layers deleted for feature extraction and feature shuffling provided in an embodiment of the present invention;

[0063] Figure 4 A schematic diagram of a flow chart of a method for identifying a radio frequency signal provided by an embodiment of the present invention;

[0064] Figure 5 A schematic diagram of the structure of a radio frequency fingerprint recognition model provided by an embodiment of the present invention;

[0065] Figure 6 A schematic diagram of the visualization effect corresponding to signal recognition when the radio frequency fingerprint recognition model provided by an embodiment of the present invention is applied;

[0066] Figure 7 A schematic diagram of the composition of a system for constructing a radio frequency fingerprint recognition model provided by an embodiment of the present invention;

[0067] Figure 8 A schematic diagram of the composition of a radio frequency signal recognition system provided by an embodiment of the present invention;

[0068] Fig. 9 A schematic diagram of the composition of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0070] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0071] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0072] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present invention. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the embodiments of the present invention.

[0073] With the rapid development of artificial intelligence and machine learning technologies, the application of IoT has become increasingly popular, and its potential and value are constantly being developed and realized. IoT security involves the policies and procedures adopted in this environment to protect devices, networks, and data from unauthorized access, use, disclosure, destruction, or tampering. Among them, secure access and authentication of devices are key links in information security and the first line of defense for information protection. However, with the widespread deployment of IoT devices and the continuous advancement of security technology, how to accurately identify a large number of terminal devices of the same model still faces major challenges. Therefore, it is necessary to conduct in-depth research on the accurate identification and security authentication of a large number of terminal devices of the same model to ensure the overall security of the IoT environment.

[0074] In the related technologies, the authentication methods of devices include password authentication, public key infrastructure, two-factor authentication and biometric authentication. However, these methods have some problems in the identification process, such as passwords are easy to be cracked and forgotten, public key infrastructure management is complex, two-factor authentication requires additional equipment or means, and biometric authentication requires special hardware support and may involve privacy issues. Therefore, the limitations of these authentication methods, multi-device interference and security issues of low-cost devices make RFFI an urgently needed technology.

[0075] At the same time, the current research on RFFI based on physical signals is mainly divided into two schools: traditional feature engineering-based methods and deep learning-based methods. Traditional feature engineering-based methods often fail to meet the requirements of high precision and high real-time performance when facing complex wireless environments. With the advancement of artificial intelligence technology, models based on deep learning can automatically extract signal features and improve the accuracy of signal classification and recognition, and are gradually becoming a research hotspot. However, Figure 1 As shown, the spectrum graphs 101 and 102 corresponding to the RF signals emitted by the same type of devices a and b are shown. It can be concluded that the feature differences between the RF signals emitted by the same type of devices (device a and device b) are very small, so it is difficult to extract effective features with sufficient discrimination, which leads to increased difficulty and reduced accuracy in identifying the same type of devices. In addition, many previous studies regard RFFI as a single-task end-to-end classification task, which has the problem of not being able to accurately capture the slight differences between signals during the training process, and it is impossible to conduct a deeper analysis of the RF fingerprint.

[0076] Based on the above problems, the present invention provides a method for constructing a radio frequency fingerprint recognition model, a recognition method, a system and a device, aiming to improve the accuracy and robustness of the recognition signal of the constructed radio frequency fingerprint recognition model.

[0077] Embodiment 1:

[0078] like Figure 2 As shown, it is a flow chart of a method for constructing a radio frequency fingerprint recognition model provided by an embodiment of the present invention. Figure 2 The steps shown in the following instructions are:

[0079] Step 101: pre-process the radio frequency signals in the radio frequency signal set to obtain a sample spectrum atlas set.

[0080] The radio frequency signal set includes: a radio frequency signal emitted by each sample device in a plurality of groups of sample devices of the same model.

[0081] In some embodiments of the present invention, multiple groups of sample devices of the same model include: a group of sample devices of model A, a group of sample devices of model B, etc.; wherein the number of sample devices included in a group of sample devices may be determined according to actual needs, such as: 2, 3 or more. Here, a radio frequency signal emitted by a sample device may refer to a basic frequency band transmission signal emitted by the sample device, such as: a long-range wireless transmission (Long Range Radio, LoRa) signal.

[0082] It should be noted that the multiple groups of sample devices of the same model involved in the present invention may all be devices in an Internet of Things environment.

[0083] In some embodiments of the present invention, each RF signal in the RF signal set can be preprocessed first, and time-frequency analysis can be performed on each preprocessed RF signal to obtain an independent spectrum diagram of the two-dimensional channel corresponding to each RF signal, thereby obtaining the sample spectrum diagram set.

[0084] In some embodiments of the present invention, the above step 101 may be implemented by the following steps 1011 and 1012 (not shown in the figure):

[0085] Step 1011: pre-process the received radio frequency signals in the radio frequency signal set to obtain a standardized signal set.

[0086] Step 1012: Use short-time Fourier transform to perform time-frequency analysis on the standardized signals in the standardized signal set to obtain the sample frequency spectrum set with two-dimensional channels being independent.

[0087] In some embodiments of the present invention, the received RF signal set (for example, including: RF signal 1, RF signal 2) can be preprocessed to obtain a standardized signal set (for example, including: standardized signal 1, standardized signal 2); wherein the preprocessing includes but is not limited to: carrier frequency offset (CFO) compensation, normalization processing, etc. Further, the standardized signal 1 and the standardized signal 2 in the standardized signal set can be subjected to time-frequency analysis using short-time Fourier transform, respectively, to obtain a two-dimensional channel-independent sample spectrum corresponding to the standardized signal 1. Figure 1 , the two-dimensional channel-independent sample spectrum corresponding to the standardized signal 2 Figure 2 , thus obtaining the sample spectrum atlas.

[0088] For example, the RF signal set includes: the baseband transmission signal s(t) emitted by the sample device 1. The baseband transmission signal s(t) is modulated and up-converted, propagated through the wireless channel, and finally received by the receiver. Then, the received signal y(t) can be expressed as follows:

[0089] y(t)=h(τ,t)*f(s(t))+n(t) formula (1);

[0090] where h(τ,t) is the time-varying channel impulse response, f(·) represents the hardware distortion function, n(t) is the additive white Gaussian noise, and * denotes the convolution operation.

[0091] It should be noted that since the transmitted RF signal will be significantly affected by the wireless channel effects, such as multipath fading, channel noise, hardware distortion, etc., in order to effectively offset these effects, the transmitted RF signal needs to be preprocessed to reduce the impact of the wireless channel effects on the transmitted RF signal.

[0092] In some embodiments of the present invention, the RF signals in the received RF signal set can be preprocessed, that is, the received signal y(t) corresponding to the baseband transmission signal s(t) can be preprocessed to obtain a corresponding standardized signal, and so on, each RF signal in the RF signal set is preprocessed, so that a standardized signal set can be obtained.

[0093] Here, the process of preprocessing the received signal y(t) may refer to the following description:

[0094] First, the preamble sequence is used for synchronization processing to ensure that the starting position of the received signal y(t) is aligned. The received signal y(t) is then compensated for CFO, and the frequency offset is estimated and removed from the received signal y(t), thereby obtaining the intermediate signal in the preprocessing process, thereby solving the CFO problem caused by the inconsistency between the transmitter and the receiver frequencies.

[0095] Secondly, the intermediate signal in the preprocessing process is normalized to standardize the signal amplitude to eliminate the amplitude changes caused by factors such as transmission distance and antenna gain, thereby ensuring the consistency of signal reception under different conditions. At the same time, the phase information of the signal is made consistent through phase correction to eliminate signal distortion caused by phase offset, and a filter is used to remove noise from the signal and adjust the signal sampling rate.

[0096] After the above preprocessing steps, the corrected and standardized standardized signal A is finally obtained. Finally, the short-time Fourier transform (STFT) is used to perform time-frequency analysis on the standardized signal A, and the time domain signal, that is, the standardized signal A, is converted into the time-frequency domain representation, so as to observe the changes of the standardized signal A in time and frequency. The corresponding description is as follows:

[0097] S can be set to the STFT complex matrix corresponding to the standardized signal A, and the absolute values ​​of the elements of the S matrix are taken, and only the amplitude information of S is used. This is because when processing wireless signals, the frequency domain signal after Fourier transform is usually in complex form, and the absolute value of the complex number can represent its amplitude. In this way, the phase information in the S matrix is ​​ignored, and only the amplitude information is retained, thereby reducing the impact of factors such as phase noise, carrier frequency offset, and sampling time deviation. Correspondingly, the STFT complex matrix can be mathematically arranged as follows:

[0098]

[0099] Among them, the column vector X m =[X 1,m ,X 2,m ,···,X N,m ] T represents the ideal spectrum of the mth segment of the standardized signal A, and the column vector H m =[H 1,m ,H 2,m ,···,H N,m ] T represents the channel frequency response experienced by the mth segment of the standardized signal A; wherein T represents the transpose of the correlation vector, and the value range of m is [1, M], represents the element-wise product, F(X m ) represents the spectrum of the mth signal, |F(X m )| represents the amplitude of the corresponding spectrum. Assuming that the wireless channel will not change significantly in such a short time, the channel frequency response H of adjacent segments m |≈|H m+1 |, divide the mth column by the m+1th column, thereby eliminating or reducing the impact of the wireless channel and hardware distortion on the spectrum graph. After the above division operation, the matrix Q that eliminates channel related information is as follows:

[0100]

[0101] Finally, the amplitude of Q is expressed in dB scale as follows, is the obtained two-dimensional channel-independent spectrum of the standardized signal A. The pixel size of the spectrum may be 102×62, as shown in the following formula (4):

[0102]

[0103] In this way, by preprocessing the transmitted RF signal, such as the LoRa signal, the influence of the wireless channel effect on the transmitted RF signal can be effectively offset, and the preprocessed standardized signal is converted into a two-dimensional channel-independent spectrum diagram, which can retain the unique characteristics of the sample device, thereby providing data support for subsequent RF signal identification.

[0104] Step 102: Perform feature extraction and feature shuffling on the sample spectrograms in the sample spectrogram set to obtain a feature vector set.

[0105] In some embodiments of the present invention, a feature extraction module that introduces a shuffling strategy can be used to perform feature extraction and feature shuffling on each sample spectrum graph in the sample spectrum graph set to obtain the feature vector set; wherein the feature extraction module can be a feature extraction network.

[0106] It should be noted that the feature shuffling operation is to move the features along a certain dimension and rearrange the patch embedding to introduce additional perturbations.

[0107] In some embodiments of the present invention, a radio frequency fingerprint feature extraction network that introduces a shuffling strategy can be designed. For example, the deep network model can be adjusted accordingly. Further, for example, the residual network (ResidualNetwork, ResNet) can be adjusted accordingly, the last fully connected layer deployed in ResNet can be removed, and the shuffling strategy can be introduced in the adjusted ResNet, thereby obtaining a feature extractor that introduces a shuffling strategy to achieve feature extraction and feature shuffling of the sample spectrum graph.

[0108] In some embodiments of the present invention, step 102 provided in the above embodiment may be implemented by the following steps 1021 to 1023 (not shown in the figure):

[0109] Step 1021: Use the feature extraction layer in the N-layer residual convolutional neural network with the fully connected layer deleted to extract features from the sample spectrograms in the sample spectrogram set to obtain an initial feature set.

[0110] Wherein, N is an integer greater than 1.

[0111] In some embodiments of the present invention, a feature extraction layer in an N-layer residual convolutional neural network with a fully connected layer deleted is used as a feature extractor to extract features from each sample spectrum in the sample spectrum set to obtain initial features corresponding to each sample spectrum, thereby obtaining the initial feature set. Here, N can be 18, 34, 50, etc., and its specific value can be determined according to actual needs.

[0112] Step 1022: Utilize a shuffling strategy to perform feature shuffling on the initial features in the initial feature set to obtain a shuffled feature set.

[0113] In some embodiments of the present invention, feature shuffling can be performed on each initial feature in the initial feature set, such as: the order of the feature quantities included in each initial feature is disrupted and rearranged to achieve feature shuffling of each initial feature, thereby obtaining a shuffled feature corresponding to each initial feature, and then obtaining a shuffled feature set.

[0114] Step 1023: Use the average pooling layer of the N-layer residual convolutional neural network to average pool the shuffled features in the shuffled feature set to obtain the feature vector set.

[0115] In some embodiments of the present invention, an N-layer residual convolutional neural network with a fully connected layer deleted can be further used, and the average pooling layer inside it can be used to perform average pooling, that is, feature aggregation, on each shuffled feature in the shuffled feature set to obtain a feature vector corresponding to each shuffled feature, thereby obtaining the feature vector set.

[0116] In some embodiments of the present invention, in order to flexibly run the deep learning model on a mobile device or embedded system, the N-layer residual convolutional neural network can be a lighter version of ResNet-18, that is, N is 18. Correspondingly, the N-layer residual convolutional neural network with the fully connected layer deleted may include: a feature extraction layer (which further includes: an initial convolutional layer, a maximum pooling layer, a residual block) and an average pooling layer, and the corresponding execution process can be referred to Figure 3 Here, taking a sample spectrum graph 301 for feature extraction and feature shuffling as an example, firstly, the feature extraction layer of ResNet-18 with the fully connected layer removed is used to extract features from the sample spectrum graph 301, as shown in Figure 3 Here, 302 specifically includes the following:

[0117] 1. Initial convolution layer (this layer uses a larger convolution kernel of 7×7, a stride of 2, a padding of 3, and an output channel of 64). This initial convolution layer is used to extract preliminary features of the input sample spectrogram 301, aiming to cover a larger area while reducing the dimension and the amount of calculation. The corresponding output data (i.e., feature) has a size of 64*112*112.

[0118] 2. Maximum pooling layer (the size of the convolution kernel of this layer is 3*3, the step size is 2, and the padding is 1). This maximum pooling layer does not change the number of channels of the data, but it will halve the size of the data. The final output data size is 64*56*56.

[0119] 3. The residual block has four 3×3 convolutional layers, that is, a sequence of four residual blocks, so that the input is not only passed directly through the convolutional layer, but also directly connected to the subsequent layer through a "shortcut". The gradient in ResNet-18 without the fully connected layer is allowed to flow directly through the shortcut, avoiding the problem of gradient vanishing in the deep network, so that ResNet-18 without the fully connected layer can be safely deepened. And each residual block has connections inside to help ResNet-18 without the fully connected layer remember the input information, preventing information loss in the deeper part of ResNet-18 without the fully connected layer, effectively allowing ResNet-18 without the fully connected layer to learn the difference between input and output instead of directly learning the mapping function.

[0120] In some embodiments of the present invention, each initial feature in the initial feature set is a multi-dimensional feature, and the above step 1022 can be implemented by the following process:

[0121] First, a first eigenvalue, a plurality of intermediate eigenvalues ​​and a plurality of eigenvalues ​​to be moved which sequentially constitute each initial feature in the initial feature set are determined.

[0122] Secondly, the multiple intermediate eigenvalues ​​and the multiple eigenvalues ​​to be moved are moved respectively, and the shuffled features of each initial feature formed in the order corresponding to the first eigenvalue, the multiple eigenvalues ​​to be moved and the multiple intermediate eigenvalues ​​are determined to obtain the shuffled feature set.

[0123] In some embodiments of the present invention, the initial feature 1 is: [A, B1, B2, B3, C1, C2, C3, C4], its corresponding first eigenvalue is A, multiple intermediate eigenvalues ​​are: B1, B2, B3, and multiple eigenvalues ​​to be moved are: C1, C2, C3, C4, then the multiple intermediate eigenvalues ​​and the multiple eigenvalues ​​to be moved are moved respectively to obtain the shuffled features of the initial feature 1, such as: [A, C1, C2, C3, C4, B1, B2, B3].

[0124] Here, you can continue to refer to Figure 3 As shown, 303 is the feature shuffling of the initial features obtained by extracting features from the sample spectrum 301; wherein, the features output by the last layer of the feature extraction layer of the sample spectrum 301 in 302, that is, the feature extraction layer is used to extract features from the sample spectrum 301, and the initial features Z i-1 , and Z i-1 It can be expressed as Its corresponding description can also be described as Here, N is the dimension of the initial feature, L is the number of convolutional layers in the feature extraction layer, h ranges from [1, N], and i represents the i-th initial feature in the initial feature set, which is Zi-1 .

[0125] Here, the patches at the first h positions in the initial feature can be moved to the last position, that is, the order of the entire initial feature is moved in h steps, and the feature arrangement is changed from becomes and The shared tags remain unchanged.

[0126] In this way, through the feature shuffling operation, the subsequent model can better adapt to various input situations, break the fixed connection between features, and thus enhance the learning and generalization capabilities of the model.

[0127] Continue with the above description and refer to Figure 3 As shown, the obtained feature shuffle set is input to the average pooling layer of 304, which can be the average pooling layer of the N-layer residual convolutional neural network without the fully connected layer, which can generate a fixed-length feature vector by calculating the average value of each channel of the shuffled feature. Where:

[0128] In order to obtain a fixed-length feature vector for further similarity measurement tasks, the global average pooling is used, i.e. Figure 3 The average pooling layer shown in is used to aggregate the shuffled features. Here, the average pooling layer takes the tensor V∈R of the shuffled features W×H×C As input, its corresponding shape is: W×H×C, R represents the real number field. Correspondingly, a vector f∈R is generated 1×1×C As the output of the average pooling process. Where W, H, C represent the width, height and number of channels of the extracted feature vector, the output global vector f = [f1,...,f c ,...,f C ] can be expressed as the following formula:

[0129]

[0130] Among them, f c is the cth element in the output global vector f, V c | represents the number of all elements in channel c, ν∈V c Represents an element on channel c in the feature map tensor V.

[0131] It should be noted that the deep network model, such as the last fully connected layer of ResNet, is removed, and only ResNet is used as a feature extractor. Secondly, since the feature differences between devices of the same model are small, in order to make the extracted features more interactive, a feature shuffling operation can be used after extracting the basic features of the input RF signal. In this way, feature extraction and feature shuffling of the input sample spectrum atlas are implemented to obtain a high-dimensional feature vector set, which can effectively represent the key feature information in the RF signal of the same model of device, and at the same time enable the model to focus on feature extraction rather than the final decision output, generating more compact and diverse robust features.

[0132] In this way, the ResNet deep learning model without the fully connected layer is used as a feature extractor to automatically extract high-dimensional features from the original RF signal (sample spectrum atlas) using only the deep network. At the same time, a shuffling strategy is introduced into the network framework corresponding to the ResNet deep learning model without the fully connected layer to achieve perturbation processing on the extracted features, thereby increasing the differences between features and improving the generalization ability of the subsequent training model.

[0133] Step 103: Perform batch normalization processing on the feature vectors in the feature vector set to obtain a first feature representation set.

[0134] In some embodiments of the present invention, an embedding head module (Embedding Head) may be used to perform batch normalization processing on each feature vector in the feature vector set, thereby obtaining a first feature representation set.

[0135] Step 104: Perform linear transformation and category mapping on the first feature representation in the first feature representation set to obtain the second feature representation set.

[0136] Each second feature representation in the second feature representation set corresponds to a predicted label.

[0137] In some embodiments of the present invention, a classification head module (Classification Head) may be used to sequentially perform linear transformation and category mapping on each first feature representation in the first feature representation set obtained in the previous step to obtain a second feature representation set.

[0138] In some embodiments of the present invention, in order to adapt to the multi-task learning strategy involved in the subsequent construction of the radio frequency fingerprint recognition model, an embedding head and a classification head may be introduced after feature extraction and feature shuffling. The embedding head and the classification head further process and transform the feature vector after average pooling, such as batch normalization, linear transformation, and category mapping, to generate the final feature representation corresponding to the training data set (the first feature representation set and the second feature representation set) and the test data set suitable for the subsequent construction of the radio frequency fingerprint recognition model.

[0139] In some embodiments of the present invention, batch normalization can be used to standardize the feature vectors in the feature vector set to obtain a corresponding first feature representation set, so that the first feature representation set can be used for metric learning to calculate the similarity between training samples (such as cosine similarity). At the same time, the first feature representation set is further linearly transformed, and the features after the linear transformation are mapped to the category space to obtain a corresponding second feature representation set, so that the second feature representation set can be used for representation learning to achieve discriminative feature representations through classification tasks.

[0140] Here, assuming that a batch of feature vectors f∈R ο×c ; where ο is the number of samples in a batch, C represents the dimension of each feature vector f, that is, the number of channels contained in f, where f = {f1,..,fi,...,fo}, and i is [1,ο], then f after batch normalization bn ∈R ο×c The corresponding eigenvector can be expressed as shown in the following formula (6):

[0141]

[0142] where γ and β are trainable scale and shift parameters, ε is a constant for the mini-batch variance for numerical stability, and μ is the mean of f, σ 2 is the variance of f, f bn is the mean-averaged variance of f.

[0143] In some embodiments of the present invention, in each training batch corresponding to the subsequent construction of the RF fingerprint recognition model, the model in training normalizes the input data before passing through a certain layer of the network to ensure that the input received by the layer has a fixed mean and variance, thereby effectively alleviating the problems of increased learning complexity and slow speed caused by changes in input distribution during training. At the same time, by mapping features to specific classification labels (each label corresponds to a unique identity), the final feature representation is generated for subsequent efficient training and accurate testing, ultimately improving the training stability and generalization performance of the final constructed RF fingerprint recognition model.

[0144] In this way, by combining the embedding head (batch normalization processing) and the classification head (linear transformation and category mapping), the distribution of features in the embedding space and the distinguishability in the category space can be optimized simultaneously, thereby improving the performance of the subsequently constructed RF fingerprint recognition model.

[0145] Step 105: Based on the first feature representation set and the second feature representation set, a multi-task learning method combining metric learning and representation learning is adopted to perform training to obtain a radio frequency fingerprint recognition model.

[0146] In some embodiments of the present invention, in order to effectively improve the performance of the subsequently constructed RF fingerprint recognition, a more advanced loss function SoftTriple for deep metric learning and an enhanced margin cosine loss function (Large Margin Cosine Loss, LMCL) can be used, which can optimize the feature space from the perspectives of metric learning and representation learning, respectively, so as to enhance the discrimination ability and generalization ability of the constructed RF fingerprint recognition model.

[0147] In some embodiments of the present invention, the above step 105 may be implemented by the following steps 1051 to 1054 (not shown in the figure):

[0148] Step 1051, respectively obtain a first loss function for characterizing the metric learning and a second loss function for characterizing the characterization learning and carrying label smoothing.

[0149] Step 1052: Use the first loss function to determine the first loss corresponding to the similarity between multiple first feature representations in the first feature representation set.

[0150] Step 1053: Use the second loss function to determine the second loss corresponding to the cosine similarity between the predicted label corresponding to the second feature representation in the second feature representation set and the sample label corresponding to the second feature representation.

[0151] Step 1054: Use the first loss and the second loss to perform training to obtain the radio frequency fingerprint recognition model.

[0152] In some embodiments of the present invention, a first loss function for characterizing metric learning, such as the loss function SoftTriple, is obtained respectively, and a second loss function for characterizing label smoothing technology and characterizing learning, such as the loss function LMCL, is obtained.

[0153] In some embodiments of the present invention, first, a first loss function can be used to determine the first loss corresponding to the similarity (such as cosine similarity) between multiple first feature representations in the first feature representation set, and a second loss function can be used to determine the second loss of the cosine similarity between the predicted label corresponding to each second feature representation in the second feature representation set and the corresponding sample label; then, the RF fingerprint recognition model to be trained is trained based on the first loss and the second loss at the same time, such as adjusting the parameters of the RF fingerprint recognition model to be trained until the loss of its output converges to obtain the RF fingerprint recognition model.

[0154] In some embodiments of the present invention, a first loss function, such as the loss function SoftTriple, is used to perform metric learning on the first feature representation set obtained by the embedding head module, which can magnify the original slight differences between feature representations in the feature space, so that the model can better capture the changes within the class and the separation between the classes, and more effectively learn the fine-grained structure of the category in the feature space; at the same time, a second loss function, such as the loss function LMCL, is used to perform representation learning on the second feature representation set obtained by the classification head module, which can further enhance the model's ability to distinguish different features, increase the distance between features of different categories, make them easier to distinguish in the feature space, and reduce the distance between features of the same category. In this way, the model can effectively capture and amplify the subtle differences in the features of the same model of equipment.

[0155] It should be noted that the loss function SoftTriple uses a metric learning method based on multiple center points rather than a single triplet, introduces multiple agents to represent each class K, and optimizes the relationship between these agents during the training process. Since the loss function SoftTriple combines the idea of ​​soft classification cross entropy, its training process is more stable and converges faster. All class center points are considered during the calculation process, so it has a low dependence on hard negative samples (negative samples that are very similar to positive sample nodes and difficult to distinguish from positive samples in the latent space), is not easy to fall into a difficult training situation, and can obtain better performance without complex sample selection. Therefore, the loss determined by the loss function SoftTriple can be placed in the feature output layer, allowing each class to have multiple centers to better handle the diversity within the class, and the features of each class are represented as the weighted sum of multiple agents through the soft assignment mechanism, so that the originally small differences are amplified in the feature space, so as to better capture the changes within the class and the separation between classes, and then more effectively learn the fine-grained structure of the category in the feature space. Among them, the objective function corresponding to the loss function SoftTriple can be defined as follows:

[0156]

[0157] in, Represents sample x i With the y i cosine similarity of class agents; It is the yth i The proxy vector of the class, cos(θ j , i) represents the similarity between sample i and the prototype vector (or class center) of category j, which is quantified by calculating the cosine of the angle between them; j represents the category index, which refers to other categories except the true category of the current sample; j≠y i It means excluding the true category yi of the current sample i, so as to sum up other categories; N represents the number of samples, that is, the total number of samples in the current batch; s is a scaling parameter used to enlarge the numerical range of cosine similarity and enhance the stability of training.

[0158] Correspondingly, the loss function LMCL uses a representation learning method. On the basis of metric learning, in order to improve the feature learning effect of the subsequently constructed RF fingerprint recognition model, the large margin cosine loss LMCL can be placed behind the classification head (i.e., used to obtain the second feature representation set) of the constructed RF fingerprint recognition model to improve the representation ability of the model by maximizing the intra-class similarity and inter-class difference. The loss function LMCL introduces an additional margin parameter based on the standard softmax loss, thus forming a new objective function. Assume that the output feature of the deep learning model is x, the corresponding class label is y, the weight matrix of the model is W, and the weight vector of the jth class is W j By introducing a larger edge in the feature space, samples of the same type are made closer and samples of different types are separated, thereby improving the distinguishability of the features. The objective function corresponding to the loss function LMCL can be defined as:

[0159]

[0160] Among them, cos(θ j ,i) represents the cosine similarity between the i-th sample and the j-th class weight vector, that is, s is a scaling parameter used to enlarge the value range of cosine similarity and enhance the stability of training; N represents the number of samples, that is, the total number of samples in the current batch. Represents the similarity between sample i and the prototype vector (or class center) of category j, which is quantified by calculating the cosine of the angle between them; m is the edge parameter used to adjust the distance between classes; j≠y i It means excluding the true category yi of the current sample i, thereby summing up other categories.

[0161] In some embodiments of the present invention, in order to solve the problem that sample features cannot be fully represented due to insufficient training data, thereby causing network overfitting, label smoothing technology can be applied in the representation learning process. Specifically, label smoothing introduces a certain degree of uncertainty to the label, allocates part of the weight to the non-target category, adjusts the distribution of the target label in the representation learning, makes the features learned by the model smoother, solves the problem of unclear category boundaries or possible changes in data distribution, and thus extracts the core features of the device more robustly.

[0162] In classification tasks, one-hot encoding is usually used as the target label, but this method may cause the model to be overconfident and very certain about the prediction results of the training samples. The prediction probability tends to be extremely 0 or 1, which reduces the generalization ability of the model on new samples. Through label smoothing, the original one-hot label y will be converted into a new label distribution y smooth Among them, the label of the target category is adjusted from 1 to The label of the non-target category is adjusted from 0 to ε is a small positive number used to control the degree of label smoothing (weakening the supervision strength), and K is the number of labels. It is defined as shown in the following formula (9):

[0163]

[0164] Among them, y i,j is the original one-hot label, indicating whether sample i belongs to category j; C represents the total number of categories.

[0165] After the label smoothing is introduced into the LMCL loss function, its corresponding objective function can be adjusted from the above formula (8) to the following formula (10):

[0166]

[0167] in, is the probability of category j predicted by the model, The calculation formula is:

[0168]

[0169] Among them, cos(θ i ,j) represents the cosine similarity between the jth sample and the i-th weight vector; correspondingly, cos(θ k ,i) represents the cosine similarity between the i-th sample and the k-th class weight vector; j = y i Represents the true category y of the current sample i i .

[0170] In this way, considering that the differences between the features corresponding to the RF signals emitted by multiple wireless devices of the same model are small, the feature vector obtained after feature extraction is applied to the multi-task learning framework, with metric learning of the fine-grained structure of categories in the feature space as the main task, and representation learning of capturing the slight differences between the features corresponding to the RF signals emitted by devices of the same model as the auxiliary task. This can enable the RF fingerprint features of the device to have stronger separation capabilities in the high-dimensional feature space, thereby improving the accuracy and robustness of the constructed RF fingerprint recognition model in identifying RF signals.

[0171] The method for constructing a radio frequency fingerprint recognition model provided by the present invention is as follows: first, the radio frequency signal sets emitted by multiple groups of sample devices of the same model are preprocessed, feature extracted and feature shuffled in sequence to obtain a feature vector set; then, the obtained feature vector set is batch normalized, linearly transformed and classified mapped to obtain a first feature representation set and a second feature representation set for training; finally, based on the first feature representation set and the second feature representation set, training is performed by means of a multi-task learning method combining metric learning and representation learning to obtain a radio frequency fingerprint recognition model. In this way, on the one hand, based on feature shuffling, it is possible to perform perturbation processing on the extracted features, thereby increasing the differences between features, and thus improving the generalization ability of the subsequently constructed radio frequency fingerprint recognition model; on the other hand, a multi-task learning framework is constructed by combining metric learning and representation learning, which can realize that the distribution of the feature vectors output by the subsequently obtained radio frequency fingerprint recognition model in the feature space is more compact, and at the same time, the ability to distinguish between class features is enhanced, that is, the slight differences between signals are accurately captured in the model training link, thereby improving the accuracy and robustness of the constructed radio frequency fingerprint recognition model in recognizing signals.

[0172] Example 2

[0173] Correspondingly, an embodiment of the present invention further provides a method for identifying a radio frequency signal, such as Figure 4 As shown, it is a flow chart of a method for identifying a radio frequency signal provided by an embodiment of the present invention. Figure 4 The steps shown are explained below:

[0174] Step 401: Acquire a radio frequency signal to be identified.

[0175] Step 402: input the radio frequency signal to be identified into a radio frequency fingerprint recognition model constructed by any of the above construction methods to identify the device to which the radio frequency signal to be identified belongs.

[0176] In some embodiments of the present invention, the radio frequency signal to be identified may be a baseband transmission signal emitted by any wireless device in an IoT environment. Correspondingly, the number of radio frequency signals to be identified may be one, or two or more. Here, when the number of radio frequency signals to be identified is multiple, they may also be baseband transmission signals emitted by multiple devices of the same model in an IoT environment.

[0177] The radio frequency signal recognition method provided by the present invention recognizes the radio frequency signal to be recognized through a radio frequency fingerprint recognition model that has feature shuffling and is constructed with the help of metric learning and representation learning, and can more accurately and efficiently determine the device to which the radio frequency signal to be recognized belongs; that is, it can achieve: optimizing signal feature extraction, that is, automatically extracting effective features in the radio frequency signal to be recognized through the radio frequency fingerprint recognition model constructed by the deep learning model, thereby enhancing the accuracy and robustness of the recognition of the radio frequency signal to be recognized; improving system adaptability, and being able to dynamically adjust the radio frequency fingerprint recognition model according to different wireless environments and application requirements by means of a flexible neural network architecture to cope with complex and changeable signal interference; improving transmission efficiency: through intelligent signal processing, the transmission rate and reliability of the wireless communication system can be improved to meet the high performance requirements of future communication technologies; promoting interconnection between devices: realizing efficient connection and communication of various wireless devices, and promoting the development of application scenarios such as the Internet of Things and smart homes.

[0178] Based on the above description, for the method for constructing a radio frequency fingerprint recognition model and the method for identifying radio frequency signals provided in the above embodiment, the network framework corresponding to the radio frequency fingerprint recognition model involved is a multi-task terminal radio frequency fingerprint feature extraction framework based on a shuffle network (Radio Frequency Fingerprint IdentificationResNet, RFFRe). Figure 5As shown in FIG. 1 , it is a schematic diagram of the structure of a radio frequency fingerprint recognition model provided by an embodiment of the present invention; wherein, in the link (model training link) of constructing the radio frequency fingerprint recognition model, first, the radio frequency signals 501 emitted by multiple wireless devices can be used as sample signals and input into the preprocessing module 502 for preprocessing, namely, the signal preprocessing and time-frequency analysis mentioned above, so as to obtain the corresponding sample spectrum atlas 503; secondly, 503 is input into the shuffle-based feature extraction network 504 for feature extraction and feature shuffling, so as to obtain a feature vector set; then, the obtained feature vector set is input into the multi-task learning module 505 for training. The architecture corresponding to the link, that is, the embedding head of 5051, is used to realize batch normalization processing of the feature vector set to obtain the first feature representation set, and the first feature representation set is input into the classification head for linear transformation and classification prediction to obtain the second feature representation set; finally, the first feature representation set and the second feature representation set are used as training samples, and the multi-task learning method (fusion of metric learning and representation learning) is used to train the parameters of the RF fingerprint recognition model to be trained (that is, the initial recognition model constructed based on the preprocessing module, the shuffled feature extraction network, the embedding head and other modules) to obtain the trained model, that is, the RF fingerprint recognition model.

[0179] Correspondingly, in the application link of the RF fingerprint recognition model, that is, the testing link (here 501 is taken as the RF signal to be recognized as an example), first, the RF signal to be recognized can be preprocessed through the preprocessing module of the RF fingerprint recognition model to obtain the spectrum diagram to be recognized; secondly, the spectrum diagram to be recognized is input into the shuffle-based feature extraction network of the RF fingerprint recognition model for feature extraction and feature shuffling to obtain the feature vector to be processed; finally, the feature vector to be processed is input into the embedding head module of 5052 for batch normalization to obtain the final feature representation, and the final feature representation is compared with the feature vector in the feature database of the current signal to be tested based on cosine similarity to obtain a similarity score, and based on the similarity score, the recognition result of the RF signal to be tested is determined, that is, the device that sends the RF signal to be recognized.

[0180] For the RF fingerprint recognition model provided by the present invention, the ResNet deep learning model with the traditional fully connected layer removed can be used as the feature extraction network, that is, only the deep network is used to automatically extract high-dimensional features in the original RF signal, and further a shuffling strategy is introduced in the network framework corresponding to the feature extraction network to achieve perturbation processing of the extracted features, increase the differences between features, and thus improve the generalization ability of the subsequently constructed RF fingerprint recognition model. At the same time, in order to further improve the performance of the RF fingerprint recognition model in identifying subtle differences, a multi-task learning framework can be constructed by combining metric learning and representation learning. In this way, the distribution of feature vectors in the feature space is more compact, and the ability to distinguish between class features is enhanced. In this way, new devices can be connected to the RF fingerprint recognition model for signal recognition without retraining, and can achieve fast registration and identity authentication, that is, signal recognition.

[0181] Correspondingly, for the RF fingerprint recognition model provided by the present invention, this solution also conducts a series of extensive experiments to evaluate the reliability of the RF fingerprint recognition model constructed above, the specific details are as follows:

[0182] The method of performing RF fingerprint recognition using the RF fingerprint recognition model provided by the present invention, the method of performing RF fingerprint recognition based on power spectrum combined with one-dimensional convolutional neural network (1D-CNN), and the method of performing RF fingerprint recognition based on channel-independent spectrum graph combined with CNN are compared. All three methods use a data set library, such as: open source data of IEEE DataPort. Table 1 compares the performance of the three methods on the task of RF fingerprint recognition of the same model of equipment. It can be seen that the method of performing RF fingerprint recognition using the RF fingerprint recognition model provided by this solution is significantly better than the other two methods, and the Rank-1 accuracy (referring to the proportion of the first category predicted by the model matching the true label in machine learning and deep learning) is improved by 2.6% and 1.2% respectively.

[0183] Table 1 Comparative experiment

[0184] method Rank-1 accuracy / % Average precision (%) This program 99.4 81.0 A method based on power paving combined with 1D-CNN 96.8 - A method based on channel-independent spectrogram combined with CNN 98.2 -

[0185] Among them, the method of RF fingerprint identification based on power spectrum combined with 1D-CNN mainly relies on the signal strength pattern in the spectrum for identification. Although the power spectrum can capture the energy distribution characteristics of the signal, it cannot fully express subtle hardware differences. And the entire model needs to be retrained when a new device is added, and it is insensitive to subtle hardware differences between devices. At the same time, the feature extraction method based on channel-independent spectrum graphs is innovative, but it mainly focuses on time-frequency features and does not effectively capture the deeper relationship between different RF fingerprint samples. Here, the RFFRe provided by the present invention (that is, the RF fingerprint identification model involved in the present invention) does not need to be retrained frequently. By converting the RF signal into a feature vector and matching it using similarity, it can capture the feature differences between multiple devices (multiple devices of the same model) more finely. At the same time, through multi-task learning, RFFRe not only relies on the training of classification boundaries, but also uses the distance information in the feature space for matching, thereby enhancing the perception of subtle differences between RF signals emitted by multiple devices of the same model.

[0186] In addition, in order to explore the effectiveness of the shuffled network and label smoothing technology, the training corresponding to the RF fingerprint recognition model provided by the present invention is carried out. Here, the basic model, the model trained using the feature extraction network based on the shuffled network, and the model for smoothing the labels in the representation learning process are further subjected to ablation experiments. The model performance comparison results are shown in Table 2. It can be seen from Table 2 that the shuffled network can significantly enhance the sensitivity of the subsequently constructed RF fingerprint recognition model to subtle hardware differences between devices by randomly shuffling the features between different channels during the feature extraction process and changing the order and combination of the features.

[0187] At the same time, it can be concluded from Table 2 that the model faces a variety of feature arrangements during training, so it can learn more generalized feature representations. On the other hand, label smoothing technology can effectively alleviate the problem of overconfidence in specific categories during the construction process (i.e., training process) of the RF fingerprint recognition model by moderately smoothing the category labels. It reduces the dependence on training samples during the training process of the RF fingerprint recognition model and suppresses the overfitting phenomenon, so that the probability distribution of the constructed RF fingerprint recognition model output is more continuous and smooth. This smoothing process not only speeds up the convergence speed of the RF fingerprint recognition model, but also significantly improves the robustness of the RF fingerprint recognition model in identifying subtle differences between devices, and can avoid relying only on certain extreme features and leaning towards a single category.

[0188] Table 2 Model performance comparison results

[0189] Model Rank-1 accuracy / % Average precision (%) Basic Model 98.6 78.9 Base model + shuffle 99.2 80.3 Base model + label smoothing 98.9 80.4 Base model + label smoothing + shuffling 99.4 81.0

[0190] Similarly, in order to explore the effectiveness of the strategy corresponding to multi-task learning (combining metric learning and representation learning) in the preference recognition task corresponding to the same model of equipment, and its generalization ability to unknown similar and different types of equipment. Here, metric learning and representation learning are also used for separate training and joint training at the same time. Referring to the results in Table 3, it can be concluded that the use of metric learning alone can strengthen the structural learning of the feature space of the obtained RF fingerprint recognition model, significantly increase the closeness between samples of the same model and the same equipment, and the discrimination between samples of different equipment. However, due to the lack of direct category judgment supervision signals in metric learning, the improvement of classification performance is limited, and the global recognition ability of the relevant model cannot be fully utilized. In contrast, when representation learning is used alone, the relevant model can focus more on optimizing the correspondence between the output and the true label, thereby effectively improving the accuracy of classification. Especially between different RF signal samples of the same model of equipment, representation learning can capture more subtle differences. However, the lack of attention to the structural information of the feature space by representation learning leads to the weak generalization ability of the relevant model when facing complex or unknown RF signals, and it is impossible to maintain a high degree of discrimination between different devices. The multi-task learning method can take into account both feature discrimination and classification accuracy by combining metric learning with representation learning. Metric learning provides precise structural constraints on the feature space, ensuring tight clustering of samples of the same device and effective distinction between samples of different devices; while representation learning enhances the model's ability to capture subtle differences between categories through direct label supervision. The joint training of the two enables the constructed RF fingerprint recognition model to show stronger robustness and generalization ability in distinguishing RF signal samples of different identities, especially in the face of complex scenarios, it can more accurately identify subtle differences between devices. Experimental results show that the RF fingerprint recognition model constructed using multi-task learning has improved the Rank-1 accuracy by 10.8% and 4.2% respectively, and the average precision by 18.8% and 15.1% respectively, compared with the single-task model constructed using metric learning or representation learning.

[0191] Table 3 Multi-task learning module

[0192] Metric Learning Representation Learning Rank-1 accuracy / % Average precision (%) √ 88.6 62.2 √ 95.2 65.9 √ √ 99.4 81.0

[0193] The performance of the radio frequency fingerprint recognition model provided by the present invention in practical applications can be described as follows:

[0194] The open source data of IEEE DataPort is used as the test data set. The transmission configuration of the LoRa signal in the test data set is the spreading factor SF = 7, the bandwidth B = 125kHz, and the carrier frequency 868MHz. In order to more clearly demonstrate the performance of the RF fingerprint recognition model, the t-Distributed Stochastic Neighbor Embedding (t-SNE) clustering algorithm is used to reduce the dimension of the device RF fingerprint features output by the RF fingerprint recognition model, and the performance is evaluated in combination with the Receiver Operating Characteristic curve (ROC), as shown in Figure 2. Figure 6 As shown; wherein 601 intuitively demonstrates the clustering effect of the RF fingerprint recognition model on the signal samples of the same model of devices in the feature space, and can effectively separate the RF signal features of ten different devices of the same model, whose device ID numbers (Identity document, ID) are: 30-39 respectively. It can be seen from 602 that the RF fingerprint recognition model provided by the present invention shows a higher area under the ROC curve (Area Under Curve, AUC) value through the ROC curve, which can indicate that the RF fingerprint recognition model has excellent performance in distinguishing positive and negative samples. Based on Figure 6 As shown, it not only verifies the clustering ability of the RF fingerprint recognition model provided by the present invention, namely RFFRe, for signal samples of the same device of the same model, but also further proves the high accuracy and robustness of the RF fingerprint recognition model provided by the present invention in practical applications.

[0195] Based on the above description, the present invention uses a deep learning model to perform intelligent analysis on wireless signals. On the one hand, in order to learn more robust and universal features, a feature extraction framework based on a shuffled network is proposed. The framework removes the traditional fully connected layer of ResNet, only uses the deep network model as a feature extractor for the RF signal, and shuffles the extracted features to introduce additional jitter. The framework does not require complex feature engineering. The model can automatically generate highly distinguishable fingerprints from the original signal and directly output the RF signal feature vector of the newly connected device to achieve efficient modeling. On the one hand, considering that the differences between the features of wireless devices of the same model are small, the feature vectors obtained by feature extraction are applied to the multi-task learning framework. The metric learning of the fine-grained structure of the category in the learning feature space is the main task, and the representation learning that captures the slight differences between the devices of the same model is the auxiliary task, so that the RF fingerprint features of the device have stronger separation ability in the high-dimensional feature space. In addition, the label smoothing technology is used to smooth the data labels in the representation learning training process to avoid the overfitting of the model to the training data and the overconfidence of certain categories in the training link, so as to improve the adaptability and robustness of the constructed RF fingerprint recognition model to new scenarios and new samples. In this way, the recognition rate and transmission quality of wireless signals can be effectively improved.

[0196] Embodiment 3:

[0197] The present invention based on the same inventive concept also provides a system for constructing a radio frequency fingerprint recognition model, such as Figure 7 FIG. 1 is a schematic diagram of a system for constructing a radio frequency fingerprint recognition model provided by an embodiment of the present invention, wherein the system 700 for constructing a radio frequency fingerprint recognition model includes:

[0198] The preprocessing module 701 is used to preprocess the radio frequency signals in the radio frequency signal set to obtain a sample spectrum atlas set; wherein the radio frequency signal set includes: a radio frequency signal emitted by each sample device in multiple groups of sample devices of the same model;

[0199] A feature extraction module 702 is used to perform feature extraction and feature shuffling on the sample spectrograms in the sample spectrogram set to obtain a feature vector set;

[0200] An embedding head module 703, configured to perform batch normalization processing on the feature vectors in the feature vector set to obtain a first feature representation set;

[0201] A classification head module 704, configured to perform linear transformation and category mapping on the first feature representation in the first feature representation set to obtain the second feature representation set; wherein each second feature representation in the second feature representation set corresponds to a predicted label;

[0202] The multi-task learning module 705 is used to perform training based on the first feature representation set and the second feature representation set by adopting a multi-task learning method combining metric learning and representation learning to obtain a radio frequency fingerprint recognition model.

[0203] Optionally, the preprocessing module 701 includes: a preprocessing unit, used to preprocess the received RF signal set to obtain a standardized signal set; a time-frequency analysis unit, used to perform time-frequency analysis on the standardized signals in the standardized signal set using short-time Fourier transform to obtain the sample spectrum atlas set independent of the two-dimensional channel.

[0204] Optionally, the feature extraction module 702 includes: a feature extraction layer in an N-layer residual convolutional neural network with a fully connected layer deleted, used to perform feature extraction on the sample spectrum graphs in the sample spectrum graph set to obtain an initial feature set; wherein N is an integer greater than 1; a shuffling unit, used to perform feature shuffling on the initial features in the initial feature set using a shuffling strategy to obtain a shuffled feature set; and an average pooling layer of the N-layer residual convolutional neural network, used to perform average pooling on the shuffled features in the shuffled feature set to obtain the feature vector set.

[0205] Optionally, each initial feature in the initial feature set is a multi-dimensional feature, and the shuffling unit is further used to determine a leading eigenvalue, multiple intermediate eigenvalues, and multiple eigenvalues ​​to be moved that sequentially constitute each initial feature in the initial feature set; the multiple intermediate eigenvalues ​​and the multiple eigenvalues ​​to be moved are moved respectively, and the shuffled features of each initial feature constituted in the order corresponding to the leading eigenvalue, the multiple eigenvalues ​​to be moved, and the multiple intermediate eigenvalues ​​are determined to obtain the shuffled feature set.

[0206] Optionally, the multi-task learning module 705 includes: a loss function acquisition unit, used to respectively acquire a first loss function used to characterize the metric learning and a second loss function used to characterize the representation learning and carrying label smoothing; a loss determination unit, used to use the first loss function to determine the first loss corresponding to the similarity between multiple first feature representations in the first feature representation set; use the second loss function to determine the second loss corresponding to the cosine similarity between the predicted label corresponding to the second feature representation in the second feature representation set and the sample label corresponding to the second feature representation; a training unit, used to use the first loss and the second loss for training to obtain the radio frequency fingerprint recognition model.

[0207] It should be noted that the description of the construction system 700 of the radio frequency fingerprint recognition model is similar to the description of the construction method embodiment of the radio frequency fingerprint recognition model, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the system embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0208] Embodiment 4:

[0209] The present invention based on the same inventive concept also provides a radio frequency signal recognition system, such as Figure 8 FIG. 8 is a schematic diagram of a radio frequency signal recognition system according to an embodiment of the present invention, wherein the radio frequency signal recognition system 800 includes:

[0210] An acquisition module 801 is used to acquire a radio frequency signal to be identified;

[0211] The identification module 802 is used to input the radio frequency signal to be identified into the radio frequency fingerprint identification model constructed by any of the above-mentioned construction systems to identify the device to which the radio frequency signal to be identified belongs.

[0212] It should be noted that the description of the radio frequency signal recognition system 800 is similar to the description of the radio frequency signal recognition method embodiment described above, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the system embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.

[0213] Embodiment 5:

[0214] Based on the same inventive concept, Fig. 9 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor 910, a memory 920, a transceiver component 930, etc. The processor 910, the memory 920 and the transceiver component 930 are connected via a bus 940; the memory 920 can be used to store an execution program, and an exemplary execution program may include instructions; the processor 910 is used to execute the instructions stored in the memory. The memory 920 can also be used to store data, which can be called and / or modified when executing instructions.

[0215] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding function, so as to realize the method for constructing a radio frequency fingerprint recognition model in the above-mentioned embodiment, or the method for recognizing radio frequency signals.

[0216] Embodiment 6:

[0217] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It can be understood that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage. The processor loads and executes one or more instructions stored in the storage medium to implement the method for constructing a radio frequency fingerprint recognition model in the above embodiment, or the method for recognizing radio frequency signals.

[0218] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0219] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0220] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0221] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0222] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for constructing a radio frequency fingerprint recognition model, characterized in that: The method comprises: Preprocessing the radio frequency signals in the radio frequency signal set to obtain a sample spectrum atlas; wherein the radio frequency signal set includes: a radio frequency signal emitted by each sample device in a plurality of groups of sample devices of the same model; Performing feature extraction and feature shuffling on the sample spectrum graphs in the sample spectrum graph set to obtain a feature vector set; Performing batch normalization processing on the feature vectors in the feature vector set to obtain a first feature representation set; Performing linear transformation and category mapping on the first feature representation in the first feature representation set to obtain the second feature representation set; wherein each second feature representation in the second feature representation set corresponds to a predicted label; Based on the first feature representation set and the second feature representation set, a multi-task learning method combining metric learning and representation learning is adopted to perform training to obtain a radio frequency fingerprint recognition model.

2. The method according to claim 1, characterized in that The preprocessing of the radio frequency signals in the radio frequency signal set to obtain a sample spectrum atlas includes: Preprocessing the received radio frequency signals in the radio frequency signal set to obtain a standardized signal set; By using short-time Fourier transform, time-frequency analysis is performed on the standardized signals in the standardized signal set to obtain the sample frequency spectrum set with two-dimensional channels being independent.

3. The method according to claim 1 or 2, characterized in that: The feature extraction and feature shuffling of the sample spectrum graphs in the sample spectrum graph set to obtain a feature vector set includes: Using a feature extraction layer in an N-layer residual convolutional neural network with a fully connected layer deleted, feature extraction is performed on the sample spectrograms in the sample spectrogram set to obtain an initial feature set; wherein N is an integer greater than 1; Using a shuffling strategy, shuffling the initial features in the initial feature set to obtain a shuffled feature set; The average pooling layer of the N-layer residual convolutional neural network is used to average pool the shuffled features in the shuffled feature set to obtain the feature vector set.

4. The method according to claim 3, characterized in that Each initial feature in the initial feature set is a multi-dimensional feature, and the shuffling strategy is used to shuffle the initial features in the initial feature set to obtain a shuffled feature set, including: Determine a first eigenvalue, a plurality of intermediate eigenvalues, and a plurality of eigenvalues ​​to be moved that sequentially constitute each initial feature in the initial feature set; The multiple intermediate eigenvalues ​​and the multiple eigenvalues ​​to be moved are moved respectively, and the shuffled features of each initial feature formed in the order of the first eigenvalue, the multiple eigenvalues ​​to be moved and the multiple intermediate eigenvalues ​​are determined to obtain the shuffled feature set.

5. The method according to claim 1, characterized in that The method of training based on the first feature representation set and the second feature representation set by adopting a multi-task learning method combining metric learning and representation learning to obtain a radio frequency fingerprint recognition model includes: Respectively obtaining a first loss function for characterizing the metric learning and a second loss function for characterizing the representation learning and carrying label smoothing; Using the first loss function, determining the first loss corresponding to the similarity between multiple first feature representations in the first feature representation set; Using the second loss function, determine the second loss corresponding to the cosine similarity between the predicted label corresponding to the second feature representation in the second feature representation set and the sample label corresponding to the second feature representation; The first loss and the second loss are used for training to obtain the radio frequency fingerprint recognition model.

6. A system for constructing a radio frequency fingerprint recognition model, characterized in that: The system comprises: A preprocessing module is used to preprocess the radio frequency signals in the radio frequency signal set to obtain a sample spectrum atlas; wherein the radio frequency signal set includes: a radio frequency signal emitted by each sample device in a plurality of groups of sample devices of the same model; A feature extraction module, used for performing feature extraction and feature shuffling on the sample spectrum graphs in the sample spectrum graph set to obtain a feature vector set; An embedding head module, used for performing batch normalization processing on the feature vectors in the feature vector set to obtain a first feature representation set; A classification head module, configured to perform linear transformation and category mapping on the first feature representation in the first feature representation set to obtain the second feature representation set; wherein each second feature representation in the second feature representation set corresponds to a predicted label; The multi-task learning module is used to perform training based on the first feature representation set and the second feature representation set by adopting a multi-task learning method combining metric learning and representation learning to obtain a radio frequency fingerprint recognition model.

7. The system according to claim 6, characterized in that The preprocessing module comprises: A preprocessing unit, used for preprocessing the received radio frequency signal set to obtain a standardized signal set; The time-frequency analysis unit is used to perform time-frequency analysis on the standardized signals in the standardized signal set by using short-time Fourier transform to obtain the sample spectrum atlas set independent of the two-dimensional channel.

8. The system according to claim 6 or 7, characterized in that: The feature extraction module comprises: The feature extraction layer in the N-layer residual convolutional neural network of the fully connected layer is deleted to extract features from the sample spectrograms in the sample spectrogram set to obtain an initial feature set; wherein N is an integer greater than 1; A shuffling unit, configured to perform feature shuffling on the initial features in the initial feature set using a shuffling strategy to obtain a shuffled feature set; The average pooling layer of the N-layer residual convolutional neural network is used to average pool the shuffled features in the shuffled feature set to obtain the feature vector set.

9. The system according to claim 8, wherein each initial feature in the initial feature set is a multi-dimensional feature. The shuffling unit is further used to determine the first eigenvalue, multiple intermediate eigenvalues ​​and multiple eigenvalues ​​to be moved that sequentially constitute each initial feature in the initial feature set; move the multiple intermediate eigenvalues ​​and the multiple eigenvalues ​​to be moved respectively, determine the shuffled features of each initial feature constructed in the order corresponding to the first eigenvalue, the multiple eigenvalues ​​to be moved and the multiple intermediate eigenvalues, and obtain the shuffled feature set.

10. The system according to claim 6, characterized in that The multi-task learning module includes: A loss function acquisition unit, used to respectively acquire a first loss function for characterizing the metric learning and a second loss function for characterizing the representation learning and carrying label smoothing; a loss determination unit, configured to use the first loss function to determine the first loss corresponding to the similarity between multiple first feature representations in the first feature representation set; and use the second loss function to determine the second loss corresponding to the cosine similarity between the predicted label corresponding to the second feature representation in the second feature representation set and the sample label corresponding to the second feature representation; A training unit is used to use the first loss and the second loss to perform training to obtain the radio frequency fingerprint recognition model.

11. A method for identifying a radio frequency signal, characterized in that: The method comprises: Acquire a radio frequency signal to be identified; The radio frequency signal to be identified is input into the radio frequency fingerprint recognition model constructed by the construction method described in any one of claims 1 to 5 to identify the device to which the radio frequency signal to be identified belongs.

12. A radio frequency signal recognition system, characterized in that: The system comprises: An acquisition module, used for acquiring a radio frequency signal to be identified; An identification module is used to input the radio frequency signal to be identified into the radio frequency fingerprint identification model constructed by the construction system according to any one of claims 6 to 10 to identify the device to which the radio frequency signal to be identified belongs.

13. An electronic device, characterized in that: include: at least one processor and memory; The memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the method for constructing a radio frequency fingerprint recognition model as described in any one of claims 1 to 5, or the method for recognizing a radio frequency signal as described in claim 11 is implemented.

14. A readable storage medium, characterized in that: An execution program is stored thereon, and when the execution program is executed, the method for constructing a radio frequency fingerprint recognition model as described in any one of claims 1 to 5, or the method for recognizing a radio frequency signal as described in claim 11 is implemented.

Images