Light-weight low-altitude aircraft detection method based on deep learning
Through the improved RTMdet target detection framework and multi-scale feature fusion module, combined with deep learning and lightweight technology, the problems of model complexity and insufficient computing power in low-altitude aircraft detection are solved, and efficient and real-time low-altitude aircraft detection is achieved.
Patent Information
- Application Number
- CN202510476408.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-16
- Publication Date
- 2025-09-12
AI Technical Summary
Existing low-altitude aircraft illegal flight detection solutions have complex model architectures, high training costs, difficulty balancing the detection accuracy of objects of different scales, and insufficient computing power in low-altitude environments, making real-time monitoring difficult.
The improved RTMdet target detection framework is adopted, combined with cardization and group convolution strategies to optimize feature extraction. A multi-scale feature fusion module is added to the model, and deep learning is used for lightweight detection. The model is deployed on the Jetson Xavier NX platform to accelerate the model.
It achieves efficient and real-time low-altitude aircraft detection in low-altitude environments, reduces the computational cost of training and adjusting models, ensures the balance of detection accuracy for objects of different scales, and has good real-time performance and low-cost advantages.
Smart Images
Figure CN120635748A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of unmanned aerial vehicles (UAVs) and aircraft, and in particular to a lightweight low-altitude aircraft detection method based on deep learning. Background Art
[0002] In recent years, with the continuous maturity and widespread application of low-altitude aircraft technology, the low-altitude economy has experienced rapid growth. According to statistics from the Civil Aviation Administration of China, as of June 2024, the number of registered civilian drones in my country has exceeded 1.875 million. While this booming low-altitude economy presents opportunities, it also poses new challenges to low-altitude flight safety. To ensure a safe and clean low-altitude flight environment, monitoring for illegal low-altitude aircraft is particularly important.
[0003] Existing solutions for detecting illegal low-altitude aircraft can leverage deep learning for low-altitude aircraft detection. Traditional object detection frameworks are complex to build and require significant time and effort to adjust training parameters. They are also inflexible when handling multi-scale objects, making it difficult to balance detection accuracy across scales. Summary of the Invention
[0004] In order to solve the above technical problems, the present invention provides a lightweight low-altitude aircraft detection method based on deep learning, comprising the following steps:
[0005] S1. Collect visual images of low-altitude aircraft and establish a low-altitude aircraft image dataset;
[0006] S2, filter and crop the data images, and use annotation tools to perform feature annotation on the data set;
[0007] S3. Convert the dataset into COCO format and divide it into training set and validation set;
[0008] S4. Build a low-altitude aircraft target detection model based on deep learning. The target detection model is improved based on the RTMdet target detection framework. The model's feature extraction is optimized using cardization and grouped convolution strategies. A multi-scale feature fusion module is added to the target detection model.
[0009] S5. Set training hyperparameters, train the target detection model, and evaluate the target detection model to obtain the optimal target detection model;
[0010] S6. Input the image of the low-altitude aircraft to be detected into the optimal target detection model to obtain the detection result.
[0011] The technical solution further defined in the present invention is:
[0012] Furthermore, in step S1, visual images of low-altitude aircraft at different pitch angles are collected to establish a low-altitude aircraft image dataset.
[0013] As described above, in a lightweight low-altitude aircraft detection method based on deep learning, in step S2, the low-altitude aircraft image dataset is uniformly numbered and cropped, and the resolution of the frozen data image is 1024*1024.
[0014] As described above, in a lightweight low-altitude aircraft detection method based on deep learning, in step S3, the training set and the validation set are randomly divided in a ratio of 8:2; the labeled file is converted to the COCO dataset, which contains the model training set image file, the validation set image file, the training set json file, and the validation set json file.
[0015] As described above, a lightweight low-altitude aircraft detection method based on deep learning is described. In step S4, the low-altitude aircraft target detection model based on deep learning includes a backbone part, a neck part, and a head part. The input of the model is a low-altitude aircraft visual image, and the output is a low-altitude aircraft image with detection results and annotation information. The cardization and group convolution strategies are used to extract features in the backbone part of the target detection model. The group convolution groups different feature maps of the input layer, and then uses different convolution kernels to convolve each group.
[0016] As described above, in a lightweight low-altitude aircraft detection method based on deep learning, in step S4, the group convolution divides the input channel and the output channel into multiple groups, and convolution calculation is performed on each group. The convolution operation formula for each group is as follows:
[0017] y g =W g *x g +b g , g=1,2,...,G
[0018] Among them, W g is the convolution kernel of the g-th group, x g is the input feature map of group g, y g is the output feature map of group g, b g is the bias of group g;
[0019] The convolution results of each layer are weighted and summed to obtain the final output feature map:
[0020] y=concat(y1, y2, ..., y G )
[0021] Among them, y1, y2, ..., y GRepresents the feature results of each group of convolution.
[0022] As described above, in a lightweight low-altitude aircraft detection method based on deep learning, in step S4, during group convolution, the parameter C is introduced to change the group convolution strategy. C represents the number of convolution operations per group, which is greater than the number of channels of the standard 1*1 convolution and 3*3 convolution. The convolution output size of each group is:
[0023]
[0024] Among them, C out is the number of channels of the final output feature map, and G is the number of groups.
[0025] As described above, in a method for detecting lightweight low-altitude aircraft based on deep learning, in step S4, a multi-scale feature fusion module is added to the neck part of the target detection model to observe feature sampling at different scales. The multi-scale feature fusion module includes a bottom-up feature extraction module, a top-down feature fusion module, and a horizontal linking module. The processing process of the multi-scale feature fusion module is as follows:
[0026] C1=Conv 1×1 (P i )
[0027] Among them, P i It is a bottom-up feature map, and C1 represents the feature map after 1*1 convolution;
[0028] In the top-down feature fusion module, bilinear interpolation is first used to sample high-level features so that their size is closer to the underlying feature map:
[0029] P′ l =Upsample(P l+1 )
[0030] Among them, P l+1 It is a high-level feature map, and after sampling, P′ is obtained l ;
[0031] Then the sampled high-level feature map P′ l and the underlying feature map C l The feature maps after 1*1 convolution are fused, and the fusion method is the splicing operation:
[0032] F l =Concat(P l ′,C l )
[0033] The final output multi-scale feature map F l .
[0034] As described above, in a lightweight low-altitude aircraft detection method based on deep learning, in step S5, when training the target detection model, the number of iterations b = 300, the training batch size n = 4, the learning rate is adjusted with the model training, the initial learning rate is 0.004, and the minimum learning rate is 0.0125.
[0035] As described above, in a lightweight low-altitude aircraft detection method based on deep learning, in step S5, the trained model is optimized and accelerated using the ONNX framework, and the target detection model is deployed on the Jetson Xavier NX platform.
[0036] The beneficial effects of the present invention are:
[0037] (1) In this paper, a new lightweight low-altitude aircraft detection method is proposed based on the premise of improving the traditional RTMdet target detection framework, thereby reducing the computational cost of training and adjusting the model and ensuring the balance of detection accuracy for objects of different scales;
[0038] (2) Compared with the existing low-altitude aircraft target detection method, the present invention has the detection accuracy and inference speed of the target detection algorithm, and is also suitable for real-time monitoring of low-altitude aircraft in a low-altitude environment, solving the problem of low computing power of the illegal flight detection algorithm in a low-altitude environment;
[0039] (3) In the present invention, based on deep learning, while overcoming the shortcomings of traditional manual detection such as long time consumption and low detection accuracy, it also has the advantages of low cost, easy implementation and good real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 It is a schematic diagram of the overall process of the present invention;
[0041] Figure 2 Schematic diagram of the structure of the improved low-altitude aircraft target detection model based on deep learning in an embodiment of the present invention;
[0042] Figure 3 Schematic diagram of the cardization and group convolution concept in an embodiment of the present invention;
[0043] Figure 4 2 is a schematic diagram of the structure of a multi-scale feature fusion module in an embodiment of the present invention;
[0044] Figure 5 Schematic diagrams of the detection results of low-altitude aircraft at different locations during model verification in an embodiment of the present invention, wherein (a) is a schematic diagram of the detection results of a low-altitude aircraft located in Hongsha Bay, Wuxi City during model verification; (b) is a schematic diagram of the detection results of a low-altitude aircraft located in Tuanjie Town, Harbin City during model verification. DETAILED DESCRIPTION
[0045] This embodiment provides a method for detecting lightweight low-altitude aircraft based on deep learning, such as Figure 1 As shown, the following steps are included:
[0046] S1. Use drone aerial photography technology to collect visual images of low-altitude aircraft and illegal flying in low-altitude environments, and establish a low-altitude aircraft image dataset. Due to the wide variety of low-altitude aircraft and their strong maneuverability, it is necessary to collect visual images of low-altitude aircraft at different pitch angles.
[0047] S2. Uniformly number and crop the low-altitude aircraft image dataset. To improve the accuracy of the low-altitude aircraft target detection model and reduce the space cost required for training, the data image resolution is fixed at 1024*1024. Feature annotation of the dataset is performed using annotation tools.
[0048] S3. Randomly divide the training set and validation set in a ratio of 8:2; convert the labeled file to the COCO dataset. The COCO dataset contains the model training set image file, validation set image file, training set json file, and validation set json file.
[0049] S4. Build a low-altitude aircraft target detection model based on deep learning. In order to improve the detection performance of the model, we choose to improve the traditional RTMdet target detection framework when building the target detection model, use cardization and group convolution strategies to optimize the model's feature extraction, and add a multi-scale feature fusion module FPN to the target detection model.
[0050] like Figure 2 As shown in the figure, the low-altitude aircraft target detection model based on deep learning includes the Backbone part, the Neck part and the Head part. The input of the model is the low-altitude aircraft visual image, and the output is the low-altitude aircraft image with detection results and annotation information.
[0051] The idea of cardization and group convolution is selected to perform feature extraction in the backbone part of the target detection model, such as Figure 3 As shown in the figure, grouped convolution groups different feature maps of the input layer, and then uses different convolution kernels to perform convolution on each group; grouped convolution divides the input channels and output channels into multiple groups, and performs convolution calculation on each group. The convolution operation formula for each group is as follows:
[0052] y g =W g *x g +b g , g=1,2,...,G
[0053] Among them, Wg is the convolution kernel of the g-th group, x g is the input feature map of group g, y g is the output feature map of group g, b g is the bias of group g.
[0054] The convolution results of each layer are weighted and summed to obtain the final output feature map:
[0055] y=concat(y1, y2, ..., y G )
[0056] Among them, y1, y2, ..., y G Represents the feature results of each group of convolution.
[0057] At the same time, when performing group convolution, the parameter C is introduced to change the group convolution strategy. C represents the number of convolution operations in each group, which is usually larger than the number of channels in standard 1*1 convolution and 3*3 convolution. The convolution output size of each group is:
[0058]
[0059] Among them, C out is the number of channels of the final output feature map, and G is the number of groups.
[0060] When building a target detection network model, a multi-scale feature fusion module is added to the neck part of the target detection model to observe feature sampling at different scales, helping the model to better handle targets of different scales and improve detection accuracy; Figure 4 As shown in Figure 3, the multi-scale feature fusion module consists of three parts, including bottom-up feature extraction, top-down feature fusion, and lateral linking.
[0061] The entire processing process of the multi-scale feature fusion module is as follows:
[0062] C1=Conv 1×1 (P i )
[0063] Among them, P i It is a bottom-up feature map, and C1 represents the feature map after 1*1 convolution.
[0064] The top-down feature fusion process first needs to use bilinear interpolation to sample the high-level features so that their size is closer to the underlying feature map:
[0065] P′ l =Upsample(P l+1 )
[0066] Among them, P l+1It is a high-level feature map, and after sampling, P′ is obtained l .
[0067] Then the sampled high-level feature map P′ l and the underlying feature map C l The feature maps after 1*1 convolution are fused, and the fusion method is the splicing operation:
[0068] F l =Concat(P l ′,C l )
[0069] The final output multi-scale feature map F l .
[0070] S5. Set the training hyperparameters, train the target detection model, and evaluate the target detection model to obtain the optimal target detection model. When training the target detection model, set the number of iterations b = 300, the training batch size n = 4, and the learning rate to adjust with the model training. The initial learning rate is 0.004, the minimum learning rate is 0.0125, and the low-altitude aircraft dataset is input.
[0071] After obtaining the trained weight model, the model is verified and evaluated, such as Figure 5 As shown in the figure, low-altitude aircraft data images were collected at two different locations, Figure 5 Figure (a) is located in Hongsha Bay, Wuxi City, and shows the detection results of the test flight image of a small civilian drone; Figure 5 Figure (b) is located in Tuanjie Town, Harbin City, and shows the low-altitude logistics drone image detection results.
[0072] Use the ONNX framework to optimize and accelerate the trained model, convert the PyTorch training model into an ONNX model, and convert the floating-point values in the model into the lower-precision int8 data type. Deploy the target detection model to the Jetson Xavier NX platform to improve model processing speed.
[0073] S6. Input the image of the low-altitude aircraft to be detected into the optimal target detection model to obtain the detection result.
[0074] Compared with the existing low-altitude aircraft target detection method, the method of this embodiment not only has the detection accuracy and inference speed of the target detection algorithm, but is also suitable for real-time monitoring of low-altitude aircraft in a low-altitude environment, solving the problem of low computing power of the illegal flight detection algorithm in a low-altitude environment; based on deep learning, the present invention overcomes the shortcomings of traditional manual detection such as long time consumption and low detection accuracy, and also has the advantages of low cost, easy implementation and good real-time performance.
[0075] In addition to the above embodiments, the present invention may also have other implementations. Any technical solution formed by equivalent replacement or equivalent transformation falls within the scope of protection required by the present invention.
Claims
1. A lightweight low-altitude aircraft detection method based on deep learning, characterized by: The following steps are involved: S1. Collect visual images of low-altitude aircraft and establish a low-altitude aircraft image dataset; S2, filter and crop the data images, and use annotation tools to perform feature annotation on the data set; S3. Convert the dataset into COCO format and divide it into training set and validation set; S4. Build a low-altitude aircraft target detection model based on deep learning. The target detection model is improved based on the RTMdet target detection framework. The model's feature extraction is optimized using cardization and grouped convolution strategies. A multi-scale feature fusion module is added to the target detection model. S5. Set training hyperparameters, train the target detection model, and evaluate the target detection model to obtain the optimal target detection model; S6. Input the image of the low-altitude aircraft to be detected into the optimal target detection model to obtain the detection result.
2. The method for detecting lightweight low-altitude aircraft based on deep learning according to claim 1, characterized in that: In step S1, visual images of low-altitude aircraft at different pitch angles are collected to establish a low-altitude aircraft image dataset.
3. The method for detecting lightweight low-altitude aircraft based on deep learning according to claim 1, characterized in that: In step S2, the low-altitude aircraft image dataset is uniformly numbered and cropped, and the resolution of the frozen data image is 1024*1024.
4. The method for detecting lightweight low-altitude aircraft based on deep learning according to claim 1, characterized in that: In step S3, the training set and the validation set are randomly divided in a ratio of 8:2; the labeled file is converted to the COCO dataset, which includes a model training set image file, a validation set image file, a training set json file, and a validation set json file.
5. The method for detecting lightweight low-altitude aircraft based on deep learning according to claim 1, characterized in that: In step S4, the low-altitude aircraft target detection model based on deep learning includes a backbone part, a neck part, and a head part. The input of the model is a low-altitude aircraft visual image, and the output is a low-altitude aircraft image with detection results and annotation information. Cardization and group convolution strategies are used to extract features in the backbone part of the target detection model. Group convolution groups different feature maps of the input layer, and then uses different convolution kernels to convolve each group.
6. The method for detecting lightweight low-altitude aircraft based on deep learning according to claim 5, characterized in that: In step S4, the group convolution divides the input channels and output channels into multiple groups, and performs convolution calculation on each group. The convolution operation formula for each group is as follows: y g =W g *x g +b g ,g=1,2,...,G Among them, W g is the convolution kernel of the g-th group, x g is the input feature map of group g, y g is the output feature map of group g, b g is the bias of group g; The convolution results of each layer are weighted and summed to obtain the final output feature map: y=concat(y1,y2,...,y G ) Among them, y1, y2, ..., y G Represents the feature results of each group of convolution.
7. The method for detecting lightweight low-altitude aircraft based on deep learning according to claim 6, characterized in that: In step S4, during group convolution, parameter C is introduced to change the group convolution strategy. C represents the number of convolution operations per group, which is greater than the number of channels of standard 1*1 convolution and 3*3 convolution. The convolution output size of each group is: Among them, C out is the number of channels of the final output feature map, and G is the number of groups.
8. The method for detecting lightweight low-altitude aircraft based on deep learning according to claim 5, characterized in that: In step S4, a multi-scale feature fusion module is added to the neck part of the target detection model to observe feature sampling at different scales. The multi-scale feature fusion module includes a bottom-up feature extraction module, a top-down feature fusion module, and a horizontal link module. The processing process of the multi-scale feature fusion module is as follows: C1=Conv 1×1 (P i ) Among them, P i It is a bottom-up feature map, and C1 represents the feature map after 1*1 convolution; In the top-down feature fusion module, bilinear interpolation is first used to sample high-level features so that their size is closer to the underlying feature map: P′ l =Upsample(P l+1 ) Among them, P l+1 It is a high-level feature map, and after sampling, P′ is obtained l ; Then the sampled high-level feature map P′ l and the underlying feature map C l The feature maps after 1*1 convolution are fused, and the fusion method is the splicing operation: F l =Concat(P′ l ,C l ) The final output multi-scale feature map F l .
9. The method for detecting lightweight low-altitude aircraft based on deep learning according to claim 1, characterized in that: In step S5, when training the target detection model, the number of iterations b=300, the training batch size n=4, the learning rate is adjusted with the model training, the initial learning rate is 0.004, and the minimum learning rate is 0.0125.
10. The method for detecting lightweight low-altitude aircraft based on deep learning according to claim 1, characterized in that: In step S5, the trained model is optimized and accelerated using the ONNX framework, and the target detection model is deployed on the Jetson Xavier NX platform.