A knowledge and data hybrid driving-based radio frequency fingerprinting method

Abstract

The application is a radio frequency fingerprint identification method based on knowledge and data hybrid driving. The method learns the channel characteristics of the input signal by using the SupportNet model, and performs equalization processing on the signal, thereby removing the influence of the Doppler frequency offset on the signal, obtaining a signal irrelevant to the channel. On the other hand, the knowledge graph and the one-dimensional convolution network are used to extract the radio frequency fingerprint features in parallel, the radio frequency fingerprint features extracted by the knowledge graph and the one-dimensional convolution network in parallel are spliced, and then the feature reorganization is realized through the attention module to realize the enhancement of effective features and the weakening of irrelevant features. Finally, the device identification result is obtained through the classifier, and the radio frequency fingerprint identification performance in the high-speed mobile scene is improved. The method uses the SupportNet model to perform equalization processing on the signal, combines the convolutional neural network and the knowledge graph to realize the radio frequency fingerprint identification, and is suitable for the high-speed mobile channel.

Description

Technical Field

[0001] This invention belongs to the field of signal security technology in wireless communication, specifically relating to a radio frequency fingerprinting method based on a hybrid knowledge and data approach. Background Technology

[0002] With the rapid development of IoT technology and the increasing scale and number of IoT devices, network security issues have become particularly prominent. Malicious attacks such as interference, deception, and sniffing pose threats to information security and system stability. To address the security threats faced by wireless communication, traditional encryption mechanisms utilize long keys to ensure secure information transmission. However, this requires the allocation and management of numerous communication keys, consuming excessive computing resources and leading to additional overhead and latency, ultimately reducing system operating efficiency.

[0003] Unlike traditional cryptographic authentication methods, physical layer security authentication offers lower complexity and network overhead. Radio frequency fingerprinting (RFF) technology relies on unique hardware characteristics of devices, often stemming from manufacturing defects. Similar to biometrics, RRF fingerprints are unique and cannot be cloned. RF fingerprint-based authentication leverages these hardware differences between devices, thus better protecting against identity attacks and spoofing.

[0004] Existing radio frequency fingerprinting (RFF) methods can be divided into two branches: traditional machine learning (TML) and deep learning (DL). TML-based RFF recognition learns a device's unique fingerprint as a device feature to identify wireless access devices. However, these TML-based classifiers require manually designed feature extraction algorithms, struggle to capture complex defective features, and lack adaptability to dynamic environments. Signal attenuation and noise interference significantly reduce the robustness of methods like k-nearest neighbor (KNN). In recent years, the development and application of DL-based methods have injected new vitality into RFF recognition research. Neural networks containing fully connected layers and the Softmax activation function can themselves be used as classifiers. Neural networks are beneficial for RFF extraction and further processing to obtain features. The function of a neural network is to automatically extract and classify features. The high-dimensional output of the hidden layers of the trained network is a latent feature representation, and then the abstract features are input into the classification layer for device identification.

[0005] However, the methods described above do not consider the challenges of high-speed mobile scenarios. For high-speed mobile communication systems, such as vehicle-to-everything (V2X) systems, the channel typically changes rapidly due to the high speed of the vehicle. Furthermore, when the signal has a large bandwidth, it is more susceptible to the effects of multipath environments. Therefore, current RFF extraction methods used in low-speed mobile communication systems cannot be directly applied to high-speed scenarios because they do not consider the impact of multipath and time-varying channels. Commonly used least squares (LS) channel estimation and frequency domain equalization techniques can remove channel information and recover the original signal. However, these methods are easily affected by noise and other adverse effects, and cannot accurately estimate and reconstruct the signal in high-speed mobile scenarios. Moreover, DL-based methods only change the network structure for RF fingerprinting, without changing their data-driven nature. Data-driven means that deep neural networks automatically learn RF fingerprint features from data, but this cannot guarantee optimal feature combinations and efficient computation. Furthermore, noise and channel uncertainties can lead to model instability and overfitting. Summary of the Invention

[0006] Technical Problem: To address the above-mentioned problems, this invention proposes a knowledge- and data-driven radio frequency fingerprinting method. This method utilizes the SupportNet model to perform signal equalization processing and combines convolutional neural networks and knowledge graphs to achieve radio frequency fingerprinting, making it suitable for high-speed mobile channels.

[0007] Technical Solution: The knowledge- and data-driven RF fingerprinting method of this invention utilizes the SupportNet model to learn the channel characteristics of the input signal and performs equalization processing on the signal to remove the influence of Doppler frequency offset on the signal, thus obtaining a channel-independent signal. On the other hand, it uses a knowledge graph and a one-dimensional convolutional network to extract RF fingerprint features in parallel. The RF fingerprint features extracted by the knowledge graph and the one-dimensional convolutional network are concatenated, and then an attention module is used to reorganize the features to enhance effective features and weaken irrelevant features. Finally, the device is classified by a classifier to obtain the device identification result, thereby improving the performance of RF fingerprinting in high-speed mobile scenarios.

[0008] The method specifically includes the following steps:

[0009] Step 1: Data Acquisition. Data is acquired using co-directional / orthogonal signals collected in a high-speed mobile environment. Where x is the data, N x It is the amount of data.

[0010] Step 2: Data preprocessing. Normalize the dataset X to adjust the dynamic range of the signal and reduce the impact of noise on key fingerprint features.

[0011] Step 3: Channel equalization. A SupportNet equalization model is built based on a one-dimensional convolutional network (1D-CNN). Training data is extracted from dataset X using a dataset loader to train the SupportNet model.

[0012] Step 4: Feature extraction. Construct a convolutional neural network that combines knowledge graph and 1D-CNN to extract features for RFID fingerprint recognition. Extract features F from the trained 1D-CNN. Data ,

[0013] Step 5: Feature combination, concatenating two column vectors F KG and F Data Get F c ;

[0014] Step 6: Feature Classification, F a The recognition result is obtained after processing through a fully connected layer.

[0015] in,

[0016] The third step, channel equalization, specifically includes the following steps:

[0017] Step 31. Construct a SupportNet equalization model based on a one-dimensional convolutional network (1D-CNN) to remove channel-related information from the signal while preserving RF fingerprint features. This model consists of two parts: a channel estimation component (Supporter) and a channel equalization component (CoreNet). Both Supporter and CoreNet have a 1D-CNN structure. The input to Supporter is data x from the dataset X after preprocessing in step 1, and the output is the estimated channel α. H CoreNet input is α H Given data x, CoreNet uses the estimated channel α H The fading-free signal is recovered from the data x affected by channel fading, and the output equalized signal constitutes a set. The signal has had its channel characteristics removed, making the input data distribution of the RF fingerprint recognition model more consistent in training and testing in real environments, thereby reducing the difficulty of RF fingerprint recognition.

[0018] Step 32. Use the dataset loader to extract training data from dataset X to train the SupportNet model.

[0019] Step 32 includes the following process.

[0020] Step 321. Initialize the parameters θ of the SupportNet equilibrium model using the Kaiming uniform distribution initialization method. SRandomly select values; input the dataset X into SupportNet to begin training;

[0021] Step 322. Calculate the output of the Supporter to obtain the channel estimation result.

[0022] α H =Supporter(x) (3)

[0023] Step 323. Calculate the α output by the Supporter module. H Perform windowing to remove α H The noise from the final output Y makes it difficult to output Y. predicted It can retain the information of the radio frequency fingerprint, after windowing. for

[0024]

[0025] Where w(n) is the window function at time n, n = 1, 2, ..., N, and N is the α value. H (n) represents the length, w(n) is expressed as

[0026]

[0027] Where, n l and n r It is α H The effective duration point of (n);

[0028] Step 324. Calculate the output of CoreNet to obtain the equalized signal.

[0029] y = CoreNet(x, α) H (4)

[0030] Step 325. Calculate the model loss function, that is, calculate the mean square error of the equalized signal and the fading-free signal, expressed as:

[0031]

[0032] Where s i y is the fading-free value of the i-th sample. i y is the value of the i-th sample after being balanced by SupportNet, and y i ∈Y, the mean squared error loss function is used to evaluate the equalization performance of the SupportNet model;

[0033] Step 326. Optimize the network parameters using backpropagation with the loss function L:

[0034] Step 327. Loop through (2.1) to (2.6) until the SupportNet model converges;

[0035] The fourth step, feature extraction, includes the following process:

[0036] Step 41. Equalize the dataset X using the SupportNet model to obtain the signal X containing channel-independent features. I From X I Extract expert features and construct a knowledge graph; for X I Perform the following operations on all the data in sequence;

[0037] Step 411. Calculate the time-domain characteristics, including calculating the mean, variance, standard deviation, maximum value, fourth-order cumulant, and sixth-order cumulant for the real and imaginary parts of the signal, respectively;

[0038] Step 412. Calculate the spectral kurtosis characteristics, including calculating the kurtosis mean, kurtosis standard deviation, and skewness for the real and imaginary parts of the signal, respectively. Kurtosis is defined as...

[0039]

[0040] μ is the population mean, σ is the standard deviation, i = 1, 2, ..., k, x Ii Indicates sample x I The i-th point, and x I ∈X I x Ii The length is k, and the skewness is defined as...

[0041]

[0042] Step 413. Calculate the frequency domain characteristics, including calculating the frequency variance, frequency standard deviation, barycentric frequency, mean square frequency, and root mean square frequency for both the real and imaginary parts of the signal. The barycentric frequency is...

[0043]

[0044] Where P(m) is the power spectrum, f m It is the frequency value at the m-th frequency point; the mean square frequency and the root mean square frequency are...

[0045]

[0046] Step 414. For X I The extracted expert features are combined into a vector X. I ′, and perform normalization representation.

[0047]

[0048] Where min is the minimum value of the expert feature, and max is the maximum value of the expert feature;

[0049] Step 42. Construct a convolutional neural network that combines knowledge graph and 1D-CNN to extract features for RFID fingerprint recognition, using a balanced dataset X. I Extract training data for training, and use the cross-entropy loss function to optimize the network parameters through backpropagation.

[0050] Step 43. Extract features F from the trained 1D-CNN Data .

[0051] The fifth step, feature combination, involves concatenating two column vectors F. KG and F Data Get F c ;

[0052]

[0053] F c It is a hybrid feature of knowledge and data, which helps to alleviate the overfitting of RF fingerprinting models to specific channel data and improve the diversity of features, then F c After being mapped to the [-1,1] interval by the attention layer, the attention weights are obtained.

[0054] a = tanh(W) a F c +b a (13)

[0055] Among them, W a and b a These are the network weights and biases obtained after training the attention layer, and the features F. c Then, element-wise multiplication is performed with weight 'a' to enhance important features and suppress irrelevant features. The features after the combined effect of the attention module are:

[0056] F a =a⊙F c (14)

[0057] Where ⊙ represents element-wise multiplication.

[0058] The sixth step, feature classification, F a The recognition result obtained after processing by the fully connected layer is:

[0059] r = Dense(F a (15)

[0060] Where r represents the category label.

[0061] Beneficial Effects: The knowledge- and data-driven RF fingerprinting method proposed in this invention has the following advantages: Compared with the channel-independent spectrogram (CIS) method and the least square-frequency domain equalization (LS-FDE) method, this invention uses the SupportNet equalization model to remove channel features, achieving better equalization performance in high-speed mobile scenarios. Furthermore, this invention extracts features in parallel using 1D-CNN and knowledge graphs, employing an attention mechanism to focus on the RF fingerprint features of the signal itself with high weights, while ignoring channel-related features with low weights, thus avoiding overfitting of the network to the channels and noise in the real environment. Attached Figure Description

[0062] Figure 1 For a knowledge- and data-driven RFID fingerprint recognition system,

[0063] Figure 2 The SupportNet model structure is as follows.

[0064] Figure 3 Designed for 1D-CNN networks,

[0065] Figure 4 This is a diagram of the convolutional neural network structure for radio frequency fingerprint recognition.

[0066] Figure 5 This is the structure of a 1D-CNN. Detailed Implementation

[0067] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0068] The present invention provides a knowledge- and data-driven radio frequency fingerprint recognition system, such as... Figure 1 As shown, the system employs a SupportNet model to learn the channel characteristics of the input signal and performs equalization processing to remove the influence of Doppler frequency offset, resulting in a channel-independent signal. On the other hand, it utilizes a knowledge graph and a one-dimensional convolutional network to extract RF fingerprint features in parallel. These RF fingerprint features are then concatenated, and an attention module is used to reorganize the features, enhancing effective features and weakening irrelevant ones. Finally, a classifier classifies the features to obtain the device identification result, improving the performance of RF fingerprint recognition in high-speed mobile scenarios. The overall system flow is as follows:

[0069] Step 1: Data collection.

[0070] The dataset uses co-directional / orthogonal signals collected in a high-speed mobile environment. Where N x It refers to the quantity of data.

[0071] Step 2: Data preprocessing.

[0072] Normalizing X adjusts the dynamic range of the signal, reducing the impact of noise on key fingerprint features.

[0073] Step 3: Channel equalization. This step includes the following process:

[0074] (1) A SupportNet equalization model is constructed based on 1D-CNN to remove channel-related information from the signal while preserving RF fingerprint features. This model consists of two parts: a channel estimation component (Supporter) and a channel equalization component (CoreNet). Both Supporter and CoreNet have a 1D-CNN structure. The input to Supporter is data x from the dataset X after the first step of data preprocessing, and the output is the estimated channel α. H CoreNet input is α H Given input data x, CoreNet can utilize the estimated channel α H The fading-free signal is recovered from the data x affected by channel fading, and the output equalized signal constitutes a set. The signal has had its channel features removed, resulting in a more consistent input data distribution for the RF fingerprint recognition model during training and real-world testing, thus reducing the difficulty of RF fingerprint recognition. This example designs a SupportNet balanced model composed of two 1D-CNN sub-networks. The network structure is as follows... Figure 2 As shown, the Supporter consists of fully connected layers, a 1D-CNN, and a Softmax layer. CoreNet consists of two fully connected layers and a 1D-CNN module. The 1D-CNN subnetwork structure is as follows: Figure 3 As shown, it includes 4 convolutional layers, 2 pooling layers, and 1 fully connected layer.

[0075] (2) Use the dataset loader to extract training data from X to train the SupportNet model. This step includes the following process.

[0076] (a) Initializing the parameters θ of the SupportNet equilibrium model using the Kaiming uniform distribution method S Random values ​​are selected. Input the dataset X into SupportNet to begin training.

[0077] (b) Calculate the output of the Supporter to obtain the channel estimation result.

[0078] αH =Supporter(x) (5)

[0079] (c) α output by the Supporter module H Perform windowing to remove α H The noise from the final output Y makes it difficult to output Y. predicted It can retain the information of the radio frequency fingerprint. After windowing... for

[0080]

[0081] Where w(n) is the window function at time n, n = 1, 2, ..., N, and N is the α value. H (n) represents the length. w(n) is expressed as...

[0082]

[0083] Where, n l and n r It is α H The effective duration point of (n).

[0084] (d) Calculate the output of CoreNet to obtain the equalized signal.

[0085] y = CoreNet(x, α) H (4)

[0086] (e) Calculate the model loss function, i.e., calculate the mean square error of the equalized signal and the fading-free signal, expressed as:

[0087]

[0088] Where s i y is the fading-free value of the i-th sample. i y is the value of the i-th sample after being balanced by SupportNet, and y i ∈Y. The mean squared error loss function is used to evaluate the equalization performance of the SupportNet model.

[0089] (f) Optimize the network parameters using backpropagation with the loss function L.

[0090] (g) Loop through (b) to (f) until the SupportNet model converges.

[0091] Step 4: Feature Extraction. This step includes the following process.

[0092] (1) Use the SupportNet model to equalize the dataset X to obtain a signal X containing channel-independent features. I From X IExtract expert features and construct a knowledge graph. For X I Perform the following operations on all the data in sequence.

[0093] (a) Calculate time-domain characteristics, including the mean, variance, standard deviation, maximum value, fourth-order cumulant and sixth-order cumulant for the real part and imaginary part of the signal, respectively.

[0094] (b) Calculate the spectral kurtosis characteristics, including calculating the kurtosis mean, kurtosis standard deviation, and skewness for both the real and imaginary parts of the signal. Kurtosis is defined as...

[0095]

[0096] μ is the population mean, σ is the standard deviation, i = 1, 2, ..., k, x Ii Indicates sample x I The i-th point, and x I ∈X I x Ii The length is k. Skewness is defined as...

[0097]

[0098] (c) Calculate the frequency domain characteristics, including calculating the frequency variance, frequency standard deviation, centroid frequency, mean square frequency, and root mean square frequency for both the real and imaginary parts of the signal. The centroid frequency is...

[0099]

[0100] Where P(m) is the power spectrum, f m This is the frequency value at the m-th frequency point. The mean square frequency and root mean square frequency are...

[0101]

[0102] (d) From X I The extracted expert features are combined into a vector X. I ′, and perform normalization representation.

[0103]

[0104] Where min is the minimum value of the expert feature and max is the maximum value of the expert feature.

[0105] (2) Convolutional neural network for RFID fingerprint recognition, which combines knowledge graph and 1D-CNN to extract features. The network structure designed in this example is as follows: Figure 4 As shown. From the balanced dataset X I Extract training data for training, and use the cross-entropy loss function to optimize the network parameters through backpropagation.

[0106] (3) Extract features F from the trained 1D-CNN Data This example designs a 1D-CNN model. The network structure is as follows: Figure 5 As shown, it includes 3 convolutional layers and 2 pooling layers, with the convolutional and pooling layers used for feature extraction and dimensionality reduction, respectively.

[0107] Step 5: Feature Combination. Concatenate two column vectors F. KG and F Data Get F c

[0108]

[0109] F c It is a hybrid feature of knowledge and data, which helps to alleviate the overfitting of RFID models to specific channel data and improve the diversity of features. Then F c After being mapped to the [-1,1] interval by the attention layer, the attention weight a is obtained.

[0110] a = tanh(W) a F c +b a (13)

[0111] Among them, W a and b a These are the network weights and biases obtained after training the attention layer. Feature F c Then, element-wise multiplication is performed with weight 'a' to enhance important features and suppress irrelevant features. The features after the combined effect of the attention module are:

[0112] F a =a⊙F c (14)

[0113] Where ⊙ represents element-wise multiplication.

[0114] Step 6: Feature Classification. a The recognition result is obtained after processing through a fully connected layer.

[0115] r = Dense(F a (15)

[0116] Where r represents the category label.

[0117] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A radio frequency fingerprint recognition method based on a knowledge- and data-driven hybrid approach, characterized in that, This method utilizes the SupportNet model to learn the channel characteristics of the input signal and performs equalization processing on the signal to remove the influence of Doppler frequency offset on the signal, thus obtaining a channel-independent signal. On the other hand, it uses a knowledge graph and a one-dimensional convolutional network to extract radio frequency fingerprint features in parallel. The radio frequency fingerprint features extracted by the knowledge graph and the one-dimensional convolutional network in parallel are concatenated, and then an attention module is used to achieve feature recombination to enhance effective features and weaken irrelevant features. Finally, the device is classified by a classifier to obtain the device recognition result, thereby improving the performance of radio frequency fingerprint recognition in high-speed mobile scenarios. The method specifically includes the following steps: Step 1: Data Acquisition. Data is acquired using co-directional / orthogonal signals collected in a high-speed mobile environment. ,in It's data. It's the amount of data. Step 2: Data preprocessing, processing the dataset. Normalization is performed to adjust the dynamic range of the signal, thereby reducing the impact of noise on key fingerprint features. Step 3: Channel equalization. A SupportNet equalization model is built based on a one-dimensional convolutional network (1D-CNN). A dataset loader is used to load data from the dataset... Training data is extracted from the dataset to train the SupportNet model. Step 4: Feature extraction. Construct a convolutional neural network that combines knowledge graph and 1D-CNN to extract features for RFID fingerprint recognition. Extract features from the trained 1D-CNN. Step 5: Feature combination, concatenating the two column vectors. and get ,in These are the expert features used to construct knowledge graphs; Step 6: Feature Classification The recognition result is obtained after processing through the fully connected layer; These are the features resulting from the combined action of the attention module; The third step, channel equalization, specifically includes the following steps: Step 31. Construct a SupportNet equalization model based on a one-dimensional convolutional network 1D-CNN to remove channel-related information from the signal while preserving radio frequency fingerprint features; The model consists of two parts: a channel estimation component called Supporter and a channel equalization component called CoreNet. Both Supporter and CoreNet have a 1D-CNN structure. The input to Supporter is the dataset after the first step of data preprocessing. Data in Output estimated channel CoreNet input is and data CoreNet utilizes the estimated channel and data affected by channel fading The fading-free signal is recovered, and the output equalized signal constitutes a set. The signal has had its channel characteristics removed, making the input data distribution of the RF fingerprint recognition model more consistent in training and testing in real environments, thereby reducing the difficulty of RF fingerprint recognition. Step 32. Use the dataset loader to retrieve data from the dataset. Training data is extracted from it to train the SupportNet model.

2. The radio frequency fingerprint recognition method based on knowledge and data hybrid driving according to claim 1, characterized in that, Step 32 includes the following process. Step 321. Initialize the parameters of the SupportNet equilibrium model using the Kaiming uniform distribution initialization method. Randomly select values; [This refers to] the dataset. Input into SupportNet to begin training; Step 322. Calculate the output of the Supporter to obtain the channel estimation result. (1) ； Step 323. Analyze the output of the Supporter module. Add a window to remove The noise from the final output makes the final output... It can retain the information of the radio frequency fingerprint, after windowing. for (2) ； in yes The window function at time t, , yes length, Represented as (3) ； in, and yes The effective duration point; Step 324. Calculate the output of CoreNet to obtain the equalized signal. (4) ； Step 325. Calculate the model loss function, that is, calculate the mean square error of the equalized signal and the fading-free signal, expressed as: (5) ； in It is the first The no-fading value of each sample, It is the first The values ​​of each sample after being balanced by SupportNet, and The mean squared error loss function is used to evaluate the equalization performance of the SupportNet model. Step 326. Use the loss function Optimize network parameters through backpropagation; Step 327. Repeat steps 321 to 326 until the SupportNet model converges.

3. The radio frequency fingerprint recognition method based on knowledge and data hybrid driving according to claim 2, characterized in that, The fourth step, feature extraction, includes the following process. Step 41. Balance the dataset using the SupportNet model. Obtain a signal containing channel-independent features ,from Extract expert features and construct a knowledge graph; for Perform the following operations on all the data in sequence; Step 411. Calculate the time-domain characteristics, including calculating the mean, variance, standard deviation, maximum value, fourth-order cumulant, and sixth-order cumulant for the real and imaginary parts of the signal, respectively; Step 412. Calculate the spectral kurtosis characteristics, including calculating the kurtosis mean, kurtosis standard deviation, and skewness for both the real and imaginary parts of the signal. Kurtosis is defined as... (6) ； The population mean Standard deviation, , Indicates sample The a point, and , The length is Skewness is defined as (7) ； Step 413. Calculate the frequency domain characteristics, including calculating the frequency variance, frequency standard deviation, barycentric frequency, mean square frequency, and root mean square frequency for both the real and imaginary parts of the signal. The barycentric frequency is... (8) ； in It is the power spectrum. It is the first The frequency values ​​at each frequency point; the mean square frequency and the root mean square frequency are... (9) ； (10) ； Step 414. From The extracted expert features are combined into a vector. And perform normalization representation. (11) ； in It is the minimum value of the expert feature. It is the maximum value of the expert feature; Step 42. Construct a convolutional neural network that combines knowledge graph and 1D-CNN to extract features for RFID fingerprint recognition, using a balanced dataset. Extract training data for training, and use the cross-entropy loss function to optimize the network parameters through backpropagation. Step 43. Extract features from the trained 1D-CNN .

4. The radio frequency fingerprint recognition method based on knowledge and data hybrid driving according to claim 3, characterized in that, The fifth step, feature combination, involves concatenating two column vectors. and get ; (12) ； It is a hybrid feature of knowledge and data, which helps to alleviate the overfitting of RFID fingerprinting models to specific channel data and improve the diversity of features. After being mapped to the [-1,1] interval by the attention layer, the attention weights are obtained. (13) ； in, and These are the network weights and biases obtained after training the attention layer, and the features. Again with The weights are multiplied element-wise to enhance important features and suppress irrelevant features. The combined features after the attention module are: (14) ； in This indicates element-wise multiplication.

5. The radio frequency fingerprint recognition method based on knowledge and data hybrid driving according to claim 4, characterized in that, The sixth step, feature classification, The recognition result obtained after processing by the fully connected layer is: (15) ； in Indicates category label.

Images