Deep transfer learning method based on model parameters

Abstract

The invention discloses a deep transfer learning method based on model parameters, and the method comprises the steps: designing a convolutional neural network model based on IQ data features and a deep residual network model based on IQ data features through employing a radiation source public data set; segmenting the radiation source IQ data by using a sliding window, and processing the original radiation source IQ data into input data for constructing a depth model; migration learning and deep learning are combined, and migration of source domain data features to a target domain is achieved; according to the method, the neural network model is constructed, and deep learning and transfer learning are combined, so that the problem that the recognition accuracy of data on a time non-stationary distribution scene is seriously reduced is effectively solved, and the recognition accuracy and timeliness are improved.

Description

Technical Field

[0001] The present invention belongs to the field of reconnaissance technology, and in particular relates to a deep transfer learning method based on model parameters. Background Art

[0002] Radiation source individual identification refers to the detection and analysis of the characteristics of the radiation source signal that can uniquely represent its identity information, and the unique identification of the radiation source through the identification algorithm. This technology can be applied to wireless network security, combating criminals, and providing spectrum detection monitoring management.

[0003] When using statistical machine learning methods to identify radiation sources, it is necessary to extract expert features. This feature extraction technology is related to specific protocols and can only characterize signals from a certain aspect. In addition, since each new wireless standard will generate a new data packet frame structure, channel bandwidth, modulation and coding scheme, the radiation source fingerprint recognition algorithm needs to be redesigned each time, and a new data set needs to be collected for retraining. This has poor adaptability and scalability and is not intelligent enough.

[0004] RF fingerprint recognition based on deep learning has poor robustness over time. Due to random changes in wireless channel conditions, hardware characteristics, etc., these changes will affect the IQ data, and then affect the features learned by the neural network from the IQ data. When using data collected at different time points for cross-training and testing, the wireless channel significantly reduces the accuracy of RF fingerprint recognition by interfering with the hardware. Re-collecting a large amount of IQ data and retraining the entire neural network can restore high accuracy, but this method is time-consuming, wastes precious computing resources, and makes the network unscalable. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a deep transfer learning method based on model parameters, which improves and perfects the existing individual radiation source identification algorithm based on deep learning, effectively solves the problem of serious decline in recognition accuracy under the existing technical conditions for scenarios with non-stationary distribution of data over time, realizes the migration of source domain data features to the target domain, and improves recognition accuracy and timeliness.

[0006] In order to achieve the above-mentioned object, the present invention adopts the following technical scheme: a deep transfer learning method based on model parameters, S1: constructing a radiation source data set;

[0007] S2: Build deep learning network models: CNN1, CNN2, and CNN3 based on the improved convolutional neural network models of AlexNet; use one-dimensional convolution to replace two-dimensional convolution, and build deep residual network models CNN4, CNN5, and CNN6 based on the improved ResNet;

[0008] S3: Dataset preprocessing: Process the original IQ data of the radiation source into input data suitable for the deep model in step S2;

[0009] S4: Design the neural network learning rate so that it can change dynamically to adapt to the model;

[0010] S5: Use the first few layers of CNN to learn low-dimensional features. The last few layers of the CNN network combine the previously extracted features to obtain semantic features. The data from the previous day used for pre-training is regarded as the source domain data, and the data from the second day used for transfer learning is regarded as the target domain data. The weight parameters of the neural network that have been trained in the source domain are transferred to the model with the same structure in the target domain. The network is fine-tuned using the target domain data to achieve transfer learning.

[0011] Furthermore, the radiation source dataset is an open dataset.

[0012] Furthermore, CNN1 in S2 is an 18-layer network, including 5 repeated blocks, wherein the parameters are set as follows: filters are all 128, kernel_size is 7, 5, 5 respectively, pool_size is 2, the number of fully connected layers is set to 3, and the number of neurons is 256, 128, 8 respectively;

[0013] CNN2 is a 6-layer network with two convolutional layers and one MaxPooling. The filters of both convolutional layers are set to 128, the kernel_size in the first convolutional layer is set to 7, the kernel_size in the second convolutional layer is set to 5, and the pool_size is set to 2.

[0014] CNN3 is a 6-layer network. Convolution + MaxPoollng is regarded as a block, which is repeated twice, pool_size is set to 2, and then global average pooling is performed. It is connected to 4 fully connected layers with 256, 256, 128, and 8 neurons respectively.

[0015] The MaxPooling size of the three networks CNN4, CNN5, and CNN6 is set to 3, where convolutionblock and identity block are composed of two one-dimensional convolutional layers, and filters are set to 64, 64, 128, 256, and 512 from stage1 to stage5, respectively. There is only one convolution layer in stage1, and its kernel_size is 7. Cow Block and Identity Block are used in stage2 to stage5, and the kernel_size of these two structures is modified to 1 or 3, and the size of the convolution kernel in the shortcut is 1.

[0016] Furthermore, the specific steps of S3 are: firstly, the I and Q components in the IQ sequence are taken out separately to form a dual-channel time series;

[0017] After using sliding window slicing, you can get n-m+1 fixed-length sequences;

[0018] The sliced ​​fixed-length sequence is used as the input sample of CNN, and the device corresponding to the sequence is used as its label. This ensures that the input size is fixed, avoids the problem of gradient vanishing due to inconsistent input length during training, and enhances the displacement invariance of CNN learning features.

[0019] Furthermore, the S4 adopts the Adam algorithm to adapt the learning rate to the parameters of the network model, and its gradient decay is:

[0020] m t =β1m t-1 +(1-β1)g t

[0021]

[0022] Use m t represents the weighted average of the gradient, v t It indicates the weighted bias of the gradient, and β1 and β2 represent the attenuation factors;

[0023] For m t and v t To correct the deviation:

[0024]

[0025] Finally, we get the parameter update equation:

[0026]

[0027] The beneficial effects of the present invention are as follows: the present invention has perfected and improved the existing deep learning-based individual radiation source identification algorithm, effectively solved the problem of serious decline in recognition accuracy under existing technical conditions for scenarios with non-stationary distribution of data over time, realized the migration of source domain data features to the target domain, and improved recognition accuracy and timeliness; the parts not described in detail in the present invention are existing commonly used technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention will be further described below in conjunction with the accompanying drawings:

[0029] Figure 1 The CNN1 model structure used in the present invention;

[0030] Figure 2 The CNN2 model structure used in the present invention;

[0031] Figure 3 The CNN3 model structure used in the present invention;

[0032] Figure 4 The CNN5 model structure used in the present invention;

[0033] Figure 5 This is an example diagram of preprocessing data of the present invention;

[0034] Figure 6 This is a flow chart of the transfer learning model of the present invention. DETAILED DESCRIPTION

[0035] The present invention is further described in detail below in conjunction with embodiments and specific implementation modes:

[0036] Example 1

[0037] S1 builds radiation source dataset;

[0038] 1.1 The radiation source dataset used in this invention is an open dataset. The open source GNU code is used to establish WiFi transmission between the receiver and the transmitter. BPSK modulation is used, the signal carrier frequency is 2.432GHZ, and the sampling frequency is 20MHz.

[0039] 1.2 The transmitter is about 1 meter away from the receiver. The receiver and the transmitter are fixed. The environment is indoor collection. All transmitters repeatedly send the same data. A total of two days of data are collected, and a single transmitter collects 10 transmission data per day.

[0040] S2 builds a deep learning network model;

[0041] 2.1 Referring to the typical CNN network AlexNet, design the convolutional neural network models CNN1, CNN2, and CNN3 based on the improved AlexNet;

[0042] CNN1 is equivalent to replacing the 5 convolutional layers of AlexNet. It is an 18-layer network with 5 repeated blocks. The parameters are set as follows: filters are all 128, kernel_size is 7, 5, 5 respectively, pool_size is 2, the number of fully connected layers is set to 3, and the number of neurons is 256, 128, and 8 respectively, as shown in the example. Figure 1 As shown;

[0043] CNN2 is a 6-layer network with two convolutional layers and one MaxPooling. The filters of both convolutional layers are set to 128, the kernel_size in the first convolutional layer is set to 7, the kernel_size in the second convolutional layer is set to 5, and the pool_size is set to 2, as shown in the example Figure 2 As shown;

[0044] CNN3 is a 6-layer network. Convolution + MaxPooling is regarded as a block, which is repeated twice, pool_size is set to 2, and then global average pooling is performed. Four fully connected layers are connected, and the number of neurons is 256, 256, 128, and 8 respectively, as shown in the example Figure 3 As shown;

[0045] 2.2 Replace two-dimensional convolution with one-dimensional convolution and design deep residual network models CNN4, CNN5, and CNN6 based on ResNet improvement:

[0046] Set the MaxPooling size of the three networks to 3. Take CNN5 as an example. Both convolutionblock and identity block are composed of two one-dimensional convolutional layers. From stage1 to stage5, filters are set to 64, 64, 128, 256, and 512, respectively. There is only one convolutional layer in stage1, and its kernel_size is 7. Cow Block and Identity Block are used in stage2 to stage5. The kernel_size of these two structures is modified to 1 or 3. The size of the convolution kernel in the shortcut is 1, as shown in the example. Figure 4 shown.

[0047] S3 processes the original IQ data of the radiation source into input data suitable for the deep model designed in step 2;

[0048] The present invention uses a sliding window with a fixed length of m to slice the radiation source IQ sequence with a length of 2n, as shown in the example Figure 5 As shown, the specific steps are as follows:

[0049] 3.1 First, the I and Q components in the IQ sequence are extracted separately to form a dual-channel time series with a dimension of nx2. The figure shows the sequence of the extracted in-phase or orthogonal components;

[0050] 3.2 After using sliding window slicing, we can get n-m+1 fixed-length sequences;

[0051] 3.3 The sliced ​​fixed-length sequence is used as the input sample of CNN, and the device corresponding to the sequence is used as its label. This ensures that the input size is fixed, avoids the problem of gradient vanishing due to inconsistent input length during training, and enhances the displacement invariance of CNN learning features.

[0052] S4 designs the neural network learning rate so that it can change dynamically to adapt to the model;

[0053] The present invention adopts the Adaptive Moment Estimation algorithm to adapt the learning rate to the parameters of the network model, and its gradient decay is:

[0054] m t =β1m t-1 +(1-β1)g t

[0055]

[0056] Use m t represents the weighted average of the gradient, v t It indicates the weighted bias of the gradient, and β1 and β2 represent the attenuation factors;

[0057] For m t and v t To correct the deviation:

[0058]

[0059] Finally, we get the parameter update equation:

[0060]

[0061] S5 builds a fine-tuned model parameter-based deep transfer learning method;

[0062] 5.1 Before testing, try to use the built network to pre-train the data on the first day and save the pre-trained model;

[0063] 5.2 Use the data from the second day for fine-tuning. Use a small amount of data from the second day to retrain the weights of the last few layers of the pre-trained neural network. During the fine-tuning process, the weights of other layers of the neural network remain fixed;

[0064] 5.3 Test the updated neural network using the data from the next day.

[0065] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A deep transfer learning method based on model parameters, characterized in that: S1: Build radiation source dataset; S2: Build deep learning network models: CNN1, CNN2, and CNN3 based on the improved convolutional neural network models of AlexNet; Use one-dimensional convolution to replace two-dimensional convolution, and improve the deep residual network models CNN4, CNN5, and CNN6 based on ResNet; S3: Dataset preprocessing: Process the original IQ data of the radiation source into input data suitable for the deep model in step S2; S4: Design the neural network learning rate so that it can change dynamically to adapt to the model; S5: Use the first few layers of CNN to learn low-dimensional features. The last few layers of the CNN network combine the previously extracted features to obtain semantic features. The data from the previous day used for pre-training is regarded as the source domain data, and the data from the second day used for transfer learning is regarded as the target domain data. The weight parameters of the neural network that have been trained in the source domain are transferred to the model with the same structure in the target domain. The network is fine-tuned using the target domain data to achieve transfer learning.

2. The deep transfer learning method based on model parameters according to claim 1, characterized in that: The radiation source dataset is an open dataset.

3. The deep transfer learning method based on model parameters according to claim 1, characterized in that: CNN1 in S2 is an 18-layer network, which contains 5 repeated blocks, and the parameters are set as follows: filters are all 128, kernel_size is 7, 5, 5 respectively, pool_size is 2, the number of fully connected layers is set to 3, and the number of neurons is 256, 128, 8 respectively; CNN2 is a 6-layer network with two convolutional layers and one MaxPooling. The filters of both convolutional layers are set to 128, the kernel_size in the first convolutional layer is set to 7, the kernel_size in the second convolutional layer is set to 5, and the pool_size is set to 2. CNN3 is a 6-layer network. Convolution + MaxPooling is regarded as a block, which is repeated twice, pool_size is set to 2, and then global average pooling is performed, followed by 4 fully connected layers with 256, 256, 128, and 8 neurons respectively. The MaxPooling size of the three networks CNN4, CNN5, and CNN6 is set to 3, where convolutionblock and identity block are composed of two one-dimensional convolutional layers. The filters are set to 64, 64, 128, 256, and 512 from stage1 to stage5, respectively. There is only one convolution layer in stage1, and its kernel_size is 7. Cow Block and Identity Block are used in stage2 to stage5. The kernel_size of these two structures is modified to 1 or 3, and the size of the convolution kernel in the shortcut is 1.

4. The deep transfer learning method based on model parameters according to claim 1, characterized in that: The specific steps of S3 are: firstly, the I and Q components in the IQ sequence are taken out separately to form a dual-channel time series; After using sliding window slicing, you can get n-m+1 fixed-length sequences; The sliced ​​fixed-length sequence is used as the input sample of CNN, and the device corresponding to the sequence is used as its label. This ensures that the input size is fixed, avoids the problem of gradient vanishing due to inconsistent input length during training, and enhances the displacement invariance of CNN learning features.

5. The deep transfer learning method based on model parameters according to claim 1, characterized in that: The S4 adopts the Adam algorithm to adapt the learning rate to the parameters of the network model, and its gradient decay is: m t =β1m t-1 +(1-β1)g t Use m t represents the weighted average of the gradient, v t It indicates the weighted bias of the gradient, and β1 and β2 represent the attenuation factors; For m t and v t To correct the deviation: Finally, we get the parameter update equation:

Images