Lightweight and high-expansibility unmanned aerial vehicle model intelligent identification method

Abstract

The invention relates to a lightweight and high-expansibility unmanned aerial vehicle model intelligent identification method. The method comprises the following steps: acquiring information of an unmanned aerial vehicle to be identified; inputting the to-be-identified unmanned aerial vehicle information into an unmanned aerial vehicle model intelligent identification model to obtain an unmanned aerial vehicle model; wherein the unmanned aerial vehicle model intelligent identification model is constructed based on a MoblieNetv3 network, and is obtained through data set training; the data set comprises real unmanned aerial vehicle signals and background signals without unmanned aerial vehicles. The method is simple, the optimization speed is high, and the unknown unmanned aerial vehicle appearing in the environment can be effectively recognized.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) model detection, and in particular to a lightweight and highly scalable intelligent identification method for UAV models. Background Art

[0002] While drones are experiencing explosive growth, the phenomenon of uncooperative drones flying illegally often occurs, posing a serious threat to public safety and personal privacy. In order to prevent drones from flying illegally and causing significant losses to society, establishing a complete drone detection system has become a research hotspot.

[0003] Previously, there were two main methods for detecting drone models. The first detection method aims to identify drone objects in images or videos and determine their orientation. Its core idea is to learn and extract the features of drones in images through deep learning. These algorithms first collect large-scale data sets to form a database, and perform classification training based on the database, so that the model can more accurately distinguish drones from other objects or backgrounds, that is, find the location of the detection object to obtain a suggestion box. The drones in the suggestion box are further classified. In recent years, with the improvement of computing power and the expansion of data sets, researchers have continuously developed more complex and accurate neural network models. The accuracy of this two-stage drone detection method has been continuously improved, but there are still the following two problems:

[0004] Complex background: Drones often appear at low altitudes, and there may be objects around them, such as the sky, buildings, and trees, that are similar to drones or that may easily cause misjudgment. These factors may interfere with the target recognition process, especially when the drone blends into the background, making it more difficult to identify.

[0005] Small size of drones: Drones come in many different models, and most of the drones in widespread use today are of small size, which limits their visibility and identification capabilities in images.

[0006] The second detection method mainly targets the remote control signals and image transmission signals of drones. Its core idea is to combine radio detection technology with deep learning technology. Compared with traditional radio detection technology, deep learning technology adopts an end-to-end approach to automatically learn and extract the features of input data, and then complete the task of classifying drone signals. By classifying drone signals through deep learning technology, on the one hand, it can quickly process and identify signals in a real-time environment and improve the response speed of the drone detection system; on the other hand, the deep learning model can expand its capabilities by increasing data and computing resources. With the investment of more training data and more powerful computing hardware, the performance of the model can be further improved. However, detecting drone signals by radio also has certain limitations and disadvantages:

[0007] Large number of model parameters: The model parameters are trained in advance in an end-to-end manner to identify drone models. When new models need to be added, the entire network needs to be retrained, which will inevitably generate huge training overhead. When the number of drone models increases to a certain level, the required network scale is also quite large, so the scalability of this method is insufficient.

[0008] High deployment conditions: The high-precision drone model recognition model structure is relatively complex and has certain requirements on the computing power of the device. In real-world scenarios, such as mobile devices and embedded devices with limited computing power, algorithm deployment is very difficult. Summary of the invention

[0009] The purpose of this invention is to propose a lightweight and highly scalable intelligent identification method for UAV models. The algorithm can cope with more complex real-life scenarios and can effectively identify unknown UAVs appearing in the environment.

[0010] To achieve the above object, the present invention provides the following solutions:

[0011] A lightweight and highly scalable intelligent identification method for UAV models, comprising:

[0012] Collect information about drones to be identified;

[0013] The information of the drone to be identified is input into the drone model intelligent identification model to obtain the drone model; wherein the drone model intelligent identification model is constructed based on the MoblieNetv3 network and is obtained by training with a data set; the data set includes: real drone signals and background signals without drones.

[0014] Optionally, collecting information of the drone to be identified includes:

[0015] Learn the information of the background environment, identify the real-time background and eliminate the preset background signal;

[0016] The extracted potential UAV weak signals are processed in combination with the signal reprocessing method to achieve signal sorting and finally output the information of the UAV to be identified.

[0017] Optionally, constructing the data set includes:

[0018] Preprocess the I / Q dual-channel data of different types of drones;

[0019] The preprocessed data is divided into training set and test set.

[0020] Optionally, preprocessing the I / Q dual-channel data of different models of drones includes:

[0021] Extract I data and Q data from I / Q dual-channel data of different models of drones;

[0022] Based on the I data and the Q data, construct a three-dimensional matrix of a preset depth;

[0023] The three-dimensional matrix is ​​normalized.

[0024] Optionally, the MoblieNetv3 network includes: an expansion layer, a depth-separable convolution layer and a linear projection layer;

[0025] The expansion layer is used to expand the number of input channels using a lightweight nonlinear function;

[0026] The depthwise separable convolution layer is used to perform feature extraction using depthwise separable convolution;

[0027] The linear projection layer is used to reduce the number of output channels.

[0028] Optionally, using depthwise separable convolution for feature extraction includes:

[0029] Each input channel is first spatially convolved, and then a 1x1 convolution is applied to combine the channels.

[0030] Optionally, training a MoblieNetv3 network using a dataset includes:

[0031] Based on the constructed training set and test set, the model parameters are adjusted and optimized using the stochastic gradient descent optimizer.

[0032] The beneficial effects of the present invention are:

[0033] In the present invention, the information of the drone to be identified is collected, that is, the potential drone signal is detected. In this step, the model combines environmental characteristics, modulation characteristics, spectrum characteristics and other information to make judgments and detect "abnormal signals" different from background signals. Each part operates independently, which is convenient for flexible configuration and expansion according to needs in practical applications;

[0034] The drone model recognition model used in the present invention has a simple structure and a fast optimization speed, and can effectively identify unknown drones appearing in the environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 A schematic diagram of a method for intelligently identifying a drone according to an embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of the overall flow of the model of an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the drone data processing process according to an embodiment of the present invention;

[0039] Figure 4 It is a schematic diagram of the overall structural frame of the model of an embodiment of the present invention;

[0040] Figure 5 A schematic diagram of recognition accuracy and reasoning time after model optimization in an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of the visualization result of model classification according to an embodiment of the present invention;

[0042] Figure 7 A schematic diagram of an Nvidia development module according to an embodiment of the present invention;

[0043] Figure 8 Schematic diagram of the intelligent identification result of drones based on real data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] This embodiment proposes a lightweight and highly scalable intelligent identification method for drone models, including:

[0047] Collect information about drones to be identified;

[0048] The information of the drone to be identified is input into the drone model intelligent identification model to obtain the drone model; wherein, the drone model intelligent identification model is built based on the MoblieNetv3 network and is obtained by training with a data set; the data set includes: real drone data.

[0049] Specifically, in this embodiment, Figure 1As shown in the figure, the first part of the method is an autonomous perception signal detection model, which is used to solve the problem of signal detection in complex environments. First, by learning the information of the background environment, the real-time background is identified and some background signals are eliminated. Then, the extracted potential UAV weak signals are processed by combining signal reprocessing technology to achieve signal sorting and finally output potential UAV signals.

[0050] Drone signals are usually in an environment full of various radio signals, such as TV broadcasts, Wi-Fi signals, Bluetooth signals, etc. The signal strength of these signals may be much higher than that of drone signals, and they can easily drown out or interfere with the effective signals of drones. Therefore, before identifying drone signals, it is necessary to determine the background environment and effectively remove the background noise while effectively retaining the drone signals in the environment. Secondly, drone signals are often weak, especially at long distances or when blocked by obstacles, the signal attenuation is more obvious. Therefore, it is necessary to use signal enhancement technology to improve the detectability of weak signals.

[0051] The real-time background learning model is one of the core parts of the drone identification algorithm. Its purpose is to identify and eliminate background noise signals in the environment, thereby highlighting drone signals. This step collects real-time environmental signals, which are input into the background learning model for preliminary judgment based on signal amplitude, power spectrum density, modulation mode, spectrum characteristics, etc. The training data of the model are some environments in daily life, such as schools, office buildings, parks, etc. For new input data, the model will output data with low similarity, that is, eliminate background signals generated in daily life. These abnormal data are then processed by signal reprocessing technology to finally obtain potential drone signals.

[0052] Before reprocessing the signal, the received radio signal must first be preprocessed. The purpose of preprocessing is to standardize the signal and remove possible significant noise components to lay the foundation for subsequent reprocessing steps. It mainly uses simple filters such as low-pass filters or mean filters to remove obvious high-frequency noise or impulse noise. Then the signal is amplitude standardized so that the amplitude range of the signal is suitable for subsequent processing steps to avoid affecting the processing effect due to differences in signal strength. After preprocessing, wavelet transform and adaptive filtering techniques are used to amplify and highlight weak drone signals to ensure that they are more obvious in the noise. Finally, short-time Fourier transform and nonlinear filtering are used to further extract signal features.

[0053] The second part of the method is a drone signal classification network, which is used to identify the drone model. Based on the information of the drone to be identified output by the first part, the model performs multi-dimensional signal analysis through environmental features, modulation features, spectrum features, and latent features, and finally outputs the recognition result by combining environmental feedback and database feedback. The model can effectively perform feature recognition through the remote control signal and image transmission signal of the drone, and finally identify the model of the drone.

[0054] Furthermore, constructing the dataset includes:

[0055] Preprocess the I / Q dual-channel data of different types of drones;

[0056] The preprocessed data is divided into training set and test set.

[0057] Furthermore, the preprocessing of I / Q dual-channel data of different types of drones includes:

[0058] Extract I data and Q data from I / Q dual-channel data of different models of drones;

[0059] Based on I data and Q data, a three-dimensional matrix of preset depth is constructed;

[0060] Normalize a three-dimensional matrix.

[0061] Specifically, in this embodiment, the preparation and construction process of the model is mainly divided into four parts. First, a data set is constructed using real drone data, and then a model framework of the recognition algorithm is constructed. After the parameters are optimized through the training set data, it is deployed on a smaller GPU development module to recognize radio signals in a real environment. The overall process is as follows: Figure 2 shown.

[0062] Build the dataset:

[0063] In order to construct high-quality training and testing data sets for training the drone intelligent recognition neural network, this embodiment will process and prepare I / Q dual-channel data of different models of drones (including DJI's Phantom series and Mavic series, and Daotong's drone signal data). The specific steps of data set construction are as follows:

[0064] 1. Data preprocessing:

[0065] 1.1. Data Separation:

[0066] Extract the I (In-phase) and Q (Quadrature) parts of the drone from the original data. Each data point contains two channels, corresponding to I and Q data respectively.

[0067] 1.2. Matrix construction and standardization:

[0068] The I and Q parts are constructed into matrices of size 2464x2464 respectively. In this way, each sample data will form a three-dimensional matrix with a depth of 2 and a size of (C, H, W) = (2, 2464, 2464). The data in the constructed three-dimensional matrix is ​​standardized to ensure that the data is on the same scale. This helps to improve the training efficiency and model performance of the neural network. The data range after standardization is [-1, 1]. The specific method is to divide all data by the data with the largest absolute value in the current matrix. The drone data processing process is as follows Figure 3 shown.

[0069]

[0070] 1.3. Data Storage:

[0071] Save the normalized matrix as a pkl file. The pkl file is Python's pickle serialization format, which is suitable for saving complex data structures.

[0072] According to the data of different types of drones, these pkl files are classified and stored in corresponding folders. The folder names use the drone types as labels, such as: Avata, Mavic, Daotong, etc.

[0073] 2. Dataset Division

[0074] 2.1 Training set and test set division:

[0075] All processed data sets are divided into training set and test set in the ratio of 8:2: the pkl files of each type of drone are randomly sorted, and the data are divided into training set and test set in the ratio of 80% and 20%.

[0076] 2.2 Dataset Storage:

[0077] Save the divided training set and test set into two different folders for subsequent model training and evaluation.

[0078] The data set in this embodiment includes the following types of signals: 1. Background signals without drones. 2. DJI's Phantom series drone signals. 3. DJI's Mavic series drone signals. 4. Daotong drone signals.

[0079] Build the model and optimize parameters:

[0080] First, under the premise of ensuring the recognition accuracy of the model, we chose a model with a small number of parameters and a fast training speed - MobileNetv3 (large). With its unique bottleneck structure, the model successfully maintains the expressiveness of the model while reducing the computational complexity and number of parameters.

[0081] Specifically, the bneck structure of MobileNetV3 usually consists of the following key parts:

[0082] 1.Expansion (expansion layer):

[0083] This layer usually uses a lightweight nonlinear function (such as ReLU) to expand the number of input channels. For example, the number of input channels is expanded to a larger value to enable more efficient feature extraction in subsequent convolutional layers.

[0084] The expansion layer is a key component in MobileNetV3. It increases the nonlinear expression ability of the network by performing dimensionality increase through 1x1 convolution. In the expansion layer, the input is first expanded through 1x1 convolution to increase the number of channels, and then nonlinear activation functions such as ReLU or h-swish may be applied. The number of output channels (expansionsize) of the expansion layer is several times the number of input channels, and this multiple is optimized through NAS technology.

[0085] 2.Depthwise Convolution (depth separable convolution):

[0086] After the expansion layer, depthwise separable convolution is usually used for feature extraction. Depthwise separable convolution divides the convolution operation into two steps: first, spatial convolution is performed on each input channel, and then 1x1 convolution is applied to combine the channels. This structure can significantly reduce the amount of calculation and the number of parameters.

[0087] The depthwise separable convolution layer is one of the key components in MobileNetV3. It reduces the amount of computation and model parameters by decomposing the standard convolution operation into two smaller operations: depthwise convolution (DW) and pointwise convolution (PW). Compared with traditional convolution, each convolution layer has only one DW convolution kernel, and its number of channels is consistent with the number of channels of the input feature map. Therefore, each DW operation only obtains one type of feature map, and the original features cannot be increased or decreased in dimension. To solve this problem, MobileNet introduced pointwise convolution (PW), which actually uses a 1×1×M×N convolution kernel to perform a traditional convolution operation on the feature map obtained by DW convolution (M is the number of channels of the PW convolution kernel, and N is the number of PW convolution kernels). The number of channels of the features finally output by the depthwise separable convolution is the number of PW convolution kernels.

[0088] 3.Linear Projection:

[0089] After performing depth-wise separable convolution, the number of output channels may be reduced to a smaller value through linear projection. This helps control the complexity of the model and enhance its efficiency on devices with limited computing resources.

[0090] The linear projection layer is another innovation in MobileNetV3, which is used in the inverted residual structure to adjust the feature dimension before and after the depth-wise separable convolution. In the inverted residual structure, the linear projection layer is usually included after the expansion layer and before the depth-wise separable convolution to reduce the number of channels of the feature map to a smaller "bottleneck" dimension. This design helps reduce the amount of computation while maintaining the expressiveness of the network.

[0091] In the inverted residual structure of MobileNetV3, the depthwise separable convolution layer and the linear projection layer are combined through a shortcut connection. The input is first increased in dimension through the expansion layer, then feature extracted through the depthwise separable convolution, and finally reduced in dimension through the linear projection layer. If the number of channels of the input and output is the same, the input can be added directly to the output through a shortcut connection to achieve a residual connection, which helps the flow of information and reduces the problem of gradient disappearance.

[0092] The overall framework of the final model and some network data such as Figure 4As shown in the figure, Input represents the format of the network input data of each layer, Operator represents the content of the layer, Exp size represents the size of the dimension after the layer is convolved, out represents the number of channels output by the layer, SE represents whether the layer uses the SE module, NL represents the activation function used, and s represents the strides of the convolution operation of the layer. After the model framework is built, the model parameters are adjusted and optimized using the stochastic gradient descent optimizer based on the constructed training and test data sets. The recognition accuracy and inference speed of the final model are shown in the figure. Figure 5 As shown, the model classification visualization results are as follows Figure 6 shown.

[0093] Physical test

[0094] Deploy the optimized model on Invidia's GPU module, such as Figure 7 As shown, an external software radio device is connected to receive radio signals in the environment through an antenna and input them into the GPU module, so as to realize intelligent recognition of drone models under the GPU development module.

[0095] Algorithm File

[0096] The main file of the algorithm consists of two parts. The first part is the script file for controlling the software radio device. Its main function is to control the external antenna to receive the signal of the corresponding drone signal frequency band and transmit it to the deployed GPU module. The second part is the main code of the drone model identification algorithm, which mainly includes the following parts:

[0097] 1. The optimized MobileNetV3 network model file;

[0098] 2.Signal preprocessing code file;

[0099] 3. Retrain the overall network and optimize the code file of parameters;

[0100] 4. Code file that controls the start and stop of the recognition algorithm.

[0101] The process of identifying the drone model is as follows:

[0102] Run the software radio script file to receive radio signals in the environment through the antenna of the software radio device.

[0103] Run the main code file of the recognition algorithm. The recognition model will pre-process the received radio signal, input the processed signal into the neural network model for classification, and finally output the classification result.

[0104] When an unknown drone model appears in the model, the corresponding drone signal is saved and the entire network is retrained to reoptimize the parameters to achieve the recognition of new drone models. Figure 8 shown.

[0105] Lightweight: The mainstream drone model recognition algorithm model is relatively complex, with a large number of parameters and high deployment difficulty. The algorithm model used in this patent has a small number of network parameters and a small overall computational workload. The small and fast characteristics allow the algorithm to be deployed on many devices, such as common GPU modules or some embedded devices on the market. The low deployment conditions allow this drone model recognition algorithm to cope with more complex real-world scenarios.

[0106] High scalability: Mainstream drone model recognition algorithms have poor recognition effects on new drones, while the recognition model built in this patent has a simple structure and fast optimization speed, and can effectively identify unknown drones that appear in the environment.

[0107] Lightweight: MobileNetv3 is a lightweight neural network designed for mobile and embedded devices. Its efficient deep separable convolution and bottleneck structure significantly reduce the amount of computation, enabling it to significantly improve the inference speed while ensuring classification accuracy. In the drone signal recognition network, MobileNetv3 can quickly locate the start time, carrier frequency and bandwidth information of the signal, and achieve efficient background noise filtering and signal extraction.

[0108] High scalability: By dividing the signal feature extraction process into multiple modules such as environmental features, modulation features, spectrum features and latent features, each module runs independently, which is convenient for flexible configuration and expansion according to needs in actual applications. The MobileNetv3 model has few parameters and low computational complexity, which is convenient for rapid adaptation and expansion in different environments. In addition, combined with the library feedback mechanism, the system can regularly update and optimize the recognition model to ensure that when new drone signal types and features are introduced, the recognition network can quickly adapt and expand, ensuring efficient recognition performance.

[0109] The above embodiments are only descriptions of the preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the protection scope determined by the claims of the present invention.

Claims

1. A lightweight and highly scalable intelligent identification method for UAV models, characterized in that: include: Collect information about drones to be identified; The information of the drone to be identified is input into the drone model intelligent identification model to obtain the drone model; wherein the drone model intelligent identification model is constructed based on the MoblieNetv3 network and is obtained by training with a data set; the data set includes: real drone signals and background signals without drones.

2. The lightweight and highly scalable intelligent identification method for UAV models according to claim 1 is characterized in that: The information collected about the drone to be identified includes: Learn the information of the background environment, identify the real-time background and eliminate the preset background signal; The extracted potential UAV weak signals are processed in combination with the signal reprocessing method to achieve signal sorting and finally output the information of the UAV to be identified.

3. The lightweight and highly scalable intelligent identification method for UAV models according to claim 1 is characterized in that: Constructing the dataset includes: Preprocess the I / Q dual-channel data of different types of drones; The preprocessed data is divided into training set and test set.

4. The lightweight and highly scalable intelligent identification method for UAV models according to claim 3 is characterized in that: Preprocessing of I / Q dual-channel data of different types of drones includes: Extract I data and Q data from I / Q dual-channel data of different models of drones; Based on the I data and the Q data, construct a three-dimensional matrix of a preset depth; The three-dimensional matrix is ​​normalized.

5. The lightweight and highly scalable intelligent identification method for UAV models according to claim 1 is characterized in that: The MoblieNetv3 network includes: an expansion layer, a depth-separable convolution layer and a linear projection layer; The expansion layer is used to expand the number of input channels using a lightweight nonlinear function; The depthwise separable convolution layer is used to perform feature extraction using depthwise separable convolution; The linear projection layer is used to reduce the number of output channels.

6. The lightweight and highly scalable intelligent identification method for UAV models according to claim 5 is characterized in that: Using depthwise separable convolution for feature extraction includes: Each input channel is first spatially convolved, and then a 1x1 convolution is applied to combine the channels.

7. The lightweight and highly scalable UAV model identification method according to claim 3 is characterized in that: Using the dataset to train the MoblieNetv3 network includes: Based on the constructed training set and test set, the model parameters are adjusted and optimized using the stochastic gradient descent optimizer.

Images