A real-time target detection method and system for a UAV platform

Abstract

The application relates to a real-time target detection method and system for a UAV platform. The method comprises the following steps: acquiring a UAV image dataset; pre-processing and manually labeling the UAV image dataset to obtain a UAV dataset; establishing an improved YOLOv3-spp network model; training and verifying the improved YOLOv3-spp network model by using the UAV dataset to obtain a UAV fire detection model; and inputting a UAV image test dataset into the UAV fire detection model to obtain a test result and an evaluation result of the performance of the UAV fire detection model. The application has stronger feature extraction capability and anti-interference capability, and has better real-time performance, and can well complete a UAV aerial image target detection task.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and target detection technology, and in particular to a real-time target detection method and system for unmanned aerial vehicle (UAV) platforms. Background Technology

[0002] The rapid development of drones has led to their widespread application (military, transportation, security, emergency response, etc.). Drone aerial image target detection technology, as one of the key technologies for drone applications, can broaden the application areas of drones and enhance their environmental perception capabilities. With the development of deep learning, target detection has become easier, and more and more scholars are beginning to use deep learning for target detection research. Two-stage target detection algorithms, such as RCNN, Faster-RCNN, Libra RCNN, Cascade RCNN, and Grid RCNN; single-stage target detection algorithms, such as the YOLO series, RetinaNet, SSD, and CenterNet; and Transformer-based target detection algorithms, such as VIT, SwinTransformer, PVTv2, DETR, and Deformer DETR, have all achieved ideal results in target detection in conventional scenarios. However, due to the characteristics of drone aerial images—such as the variety of targets, small size, complex backgrounds, and uneven distribution—existing technologies lack effective feature information, and the aforementioned algorithm models cannot be well applied to such scenarios. These problems make drone aerial image target detection a meaningful and challenging research topic. Summary of the Invention

[0003] The purpose of this invention is to provide a real-time target detection method and system for unmanned aerial vehicle (UAV) platforms, which can solve the problem that there are many small-scale target instances and a lack of effective feature information in UAV images in the prior art.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] A real-time target detection method for unmanned aerial vehicle (UAV) platforms includes:

[0006] Obtain drone image dataset;

[0007] The UAV image dataset is preprocessed and manually labeled to obtain the UAV dataset;

[0008] Establish an improved YOLOv3-spp network model;

[0009] The improved YOLOv3-spp network model was trained and validated using the aforementioned UAV dataset to obtain a UAV fire detection model.

[0010] The drone image test dataset is input into the drone fire detection model to obtain the test results and evaluation results of the drone fire detection model's performance.

[0011] Optionally, the preprocessing and manual annotation of the UAV image dataset to obtain the UAV dataset specifically includes:

[0012] The UAV image dataset is expanded by image preprocessing methods such as image cropping, rotation, stitching, noise addition, and copy-paste to obtain the expanded UAV image dataset.

[0013] The expanded UAV image dataset was manually annotated using the Labelme annotation software to obtain the UAV dataset.

[0014] Optionally, establishing the improved YOLOv3-spp network model specifically includes:

[0015] Based on the baseline model yolov3-spp, the feature map after 8x downsampling is upsampled and fused with the feature map after 4x downsampling in the backbone network to form a small target detection layer. At the same time, it forms a new multi-scale detection architecture with the original 8x and 16x downsampled feature maps.

[0016] The improved YOLOv3-spp network model introduces the adaptive feature fusion method ASFF. Combined with the new multi-scale detection architecture, the adaptive feature fusion method ASFF-Small is designed. By assigning different weights to feature maps of different scales, the effective feature maps are strengthened, and feature fusion is better achieved.

[0017] The improved YOLOv3-spp network model employs a new loss function SIoU and the Bottleneck module improved using depthwise separable convolution.

[0018] The new loss function SIoU includes the angle cost function, distance cost function, shape cost function, and IoU cost function;

[0019] The depthwise separable convolution improved Bottleneck module includes: two 1×1 convolutions, one 3×3 convolution, a residual structure, and a feature fusion operation. The 1×1 convolutions are mainly used for dimensionality reduction and expansion, the 3×3 convolutions are used for feature extraction, and the residual structure and feature fusion operation are used to fuse the input feature map and the output of the improved Bottleneck module.

[0020] Optionally, the step of training and validating the improved YOLOv3-spp network model using the UAV dataset to obtain the UAV fire detection model specifically includes:

[0021] The improved YOLOv3-spp network model was trained and validated using the aforementioned UAV dataset under set experimental conditions and training parameters to obtain a UAV fire detection model.

[0022] The experimental conditions were set as follows: PyTorch framework, PyTorch version 1.7.0, programming language Python 3.8.10, hardware configuration Intel i7-9700, 32G memory, GPU GeForce RTX3080 with 16G video memory, and operating environment Ubuntu 20.04.

[0023] The training parameters were set as follows: BitchSize was set to 16, and epochs were set to 150. The initial learning rate was 0.001, the momentum factor was 0.937, and the SGD optimizer was selected; mAP was used. 50 mAP 75 Metrics such as Params, GFLOPs, and FPS are used as evaluation metrics for model performance.

[0024] Optionally, the real-time target detection method for unmanned aerial vehicle (UAV) platforms further includes:

[0025] The trained, tested, and validated drone fire detection model was deployed to a drone platform. The specific hardware for deployment was a Jetson Nano B01 development board, which was compiled based on the PyTorch framework in an Ubuntu environment.

[0026] A real-time target detection system for unmanned aerial vehicle (UAV) platforms includes:

[0027] The UAV image dataset acquisition module is used to acquire UAV image datasets.

[0028] The UAV dataset determination module is used to preprocess and manually annotate the UAV image dataset to obtain the UAV dataset;

[0029] The network model building module is used to build the improved YOLOv3-spp network model.

[0030] The drone fire detection model determination module is used to train and validate the improved YOLOv3-spp network model using the drone dataset to obtain the drone fire detection model.

[0031] The model performance test evaluation result determination module is used to input the UAV image test dataset into the UAV fire detection model to obtain the test results and evaluation results of the UAV fire detection model performance.

[0032] Optionally, the UAV dataset determination module specifically includes:

[0033] The dataset expansion unit is used to expand the UAV image dataset by using image preprocessing methods such as image cropping, rotation, stitching, noise addition, and copy-paste to obtain the expanded UAV image dataset.

[0034] The manual annotation unit is used to manually annotate the expanded UAV image dataset using the annotation software Labelme to obtain the UAV dataset.

[0035] Optionally, the network model building module specifically includes:

[0036] The network model building unit is used to upsample the feature map after 8x downsampling based on the baseline model yolov3-spp and fuse it with the feature map after 4x downsampling in the backbone network to form a small target detection layer. At the same time, it forms a new multi-scale detection architecture with the original 8x and 16x downsampling feature maps.

[0037] The improved YOLOv3-spp network model introduces the adaptive feature fusion method ASFF. Combined with the new multi-scale detection architecture, the adaptive feature fusion method ASFF-Small is designed. By assigning different weights to feature maps of different scales, the effective feature maps are strengthened, and feature fusion is better achieved.

[0038] The improved YOLOv3-spp network model employs a new loss function SIoU and the Bottleneck module improved using depthwise separable convolution.

[0039] The new loss function SIoU includes the angle cost function, distance cost function, shape cost function, and IoU cost function;

[0040] The depthwise separable convolution improved Bottleneck module includes: two 1×1 convolutions, one 3×3 convolution, a residual structure, and a feature fusion operation. The 1×1 convolutions are mainly used for dimensionality reduction and expansion, the 3×3 convolutions are used for feature extraction, and the residual structure and feature fusion operation are used to fuse the input feature map and the output of the improved Bottleneck module.

[0041] Optionally, the UAV fire detection model determination module specifically includes:

[0042] The drone fire detection model determination unit is used to train and validate the improved YOLOv3-spp network model using the drone dataset under set experimental conditions and set training parameters, so as to obtain the drone fire detection model.

[0043] The experimental conditions were set as follows: PyTorch framework, PyTorch version 1.7.0, programming language Python 3.8.10, hardware configuration Intel i7-9700, 32G memory, GPU GeForce RTX 3080 with 16G video memory, and operating environment Ubuntu 20.04.

[0044] The training parameters were set as follows: BitchSize was set to 16, and epochs were set to 150. The initial learning rate was 0.001, the momentum factor was 0.937, and the SGD optimizer was selected; mAP was used. 50 mAP 75 Metrics such as Params, GFLOPs, and FPS are used as evaluation metrics for model performance.

[0045] Optionally, the real-time target detection system for unmanned aerial vehicle (UAV) platforms further includes:

[0046] The drone platform deployment module is used to deploy the trained, tested and verified drone fire detection model to the drone platform. The specific deployment hardware is the Jetson Nano B01 development board, which is compiled based on the PyTorch framework in the Ubuntu environment.

[0047] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0048] This invention provides a real-time target detection method for unmanned aerial vehicle (UAV) platforms. The method includes: acquiring a UAV image dataset; preprocessing and manually labeling the UAV image dataset to obtain a UAV dataset; establishing an improved YOLOv3-spp network model; training and validating the improved YOLOv3-spp network model using the UAV dataset to obtain a UAV fire detection model; and inputting a UAV image test dataset into the UAV fire detection model to obtain test results and evaluation results of the UAV fire detection model's performance. The method disclosed in this invention has stronger feature extraction and anti-interference capabilities, good real-time performance, and can be applied to mobile devices, effectively completing UAV aerial image target detection tasks. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart of the real-time target detection method for unmanned aerial vehicle (UAV) platforms according to the present invention;

[0051] Figure 2 This is a structural diagram of the ASFF-Small of the present invention;

[0052] Figure 3 This is a schematic diagram of the improved Bottleneck module of the present invention;

[0053] Figure 4 This is a schematic diagram illustrating the SIoU function calculation principle of the present invention;

[0054] Figure 5 These are diagrams of the ordinary convolution and depthwise separable convolution structures of this invention;

[0055] Figure 6 This is a schematic diagram of some detection results of the improved YOLOv3-SPP on the test set according to the present invention;

[0056] Figure 7 This is a structural diagram of the real-time target detection system for unmanned aerial vehicle (UAV) platforms according to the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] The purpose of this invention is to provide a real-time target detection method and system for unmanned aerial vehicle (UAV) platforms, which can solve the problem that there are many small-scale target instances and a lack of effective feature information in UAV images in the prior art.

[0059] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Figure 1 This is a flowchart of the real-time target detection method for unmanned aerial vehicle (UAV) platforms according to the present invention. Figure 1 As shown, a real-time target detection method for unmanned aerial vehicle (UAV) platforms includes:

[0061] Step 101: Obtain the drone image dataset.

[0062] Step 102: Preprocess and manually label the UAV image dataset to obtain the UAV dataset, specifically including:

[0063] The UAV image dataset is expanded by image preprocessing methods such as image cropping, rotation, stitching, noise addition, and copy-paste, resulting in an expanded UAV image dataset.

[0064] The expanded UAV image dataset was manually annotated using the Labelme annotation software to obtain the UAV dataset.

[0065] Step 103: Establish the improved YOLOv3-spp network model, specifically including:

[0066] Based on the baseline model yolov3-spp, the feature map after 8x downsampling is upsampled and fused with the feature map after 4x downsampling in the backbone network to form a small target detection layer. At the same time, it forms a new multi-scale detection architecture with the original 8x and 16x downsampled feature maps.

[0067] YOLO-SPP does not consider the differences between feature maps of different scales. Shallow features contain more location information, texture, and edge features, while deep features contain more semantic information. The impact of feature maps at different scales on object detection varies. In drone aerial images, small targets are common, making shallow feature maps containing location information, texture, and edge features play a more important role. To maximize the effectiveness of feature maps at different scales, the adaptive feature fusion method ASFF is introduced. Combined with improvements to the object detection scale, the adaptive feature fusion method ASFF-Small is designed. By assigning different weights to feature maps of different scales, effective feature maps are strengthened, resulting in better feature fusion. Figure 2 As shown. Figure 2 This is a structural diagram of the ASFF-Small of the present invention.

[0068] set up To adjust the feature vector at position (i, j) on the feature map from level n to level l, the feature fusion at level l is performed through the following relationship:

[0069]

[0070] In the formula, and And define The following relationship must be satisfied:

[0071]

[0072] In the formula, Using weights respectively The Softmax function is defined as the control parameters. These weights are obtained by 1×1 convolution and can therefore be learned and adaptively adjusted through standard backpropagation.

[0073] The improved YOLOv3-spp network model introduces the adaptive feature fusion method ASFF. Combined with a new multi-scale detection architecture, the adaptive feature fusion method ASFF-Small is designed. By assigning different weights to feature maps of different scales, the effective feature maps are strengthened, and feature fusion is better achieved.

[0074] The improved YOLOv3-spp network model employs a new loss function SIoU and a depthwise separable convolution improved Bottleneck module.

[0075] The new loss function SIoU includes an angle cost function, a distance cost function, a shape cost function, and an IoU cost function. SIoU takes into account the vector angle between the desired regressions and redefines the penalty metric, effectively improving training speed and inference accuracy. Figure 4 This is a schematic diagram illustrating the SIoU function calculation principle of the present invention.

[0076] (1) Angle cost function

[0077] The purpose of adding the angular cost function λ is to minimize the number of distance-related variables. During training, the predicted box moves closer to the ground truth box along the X-axis or Y-axis, and then continues moving closer along the relevant axis. First minimize α, otherwise minimize β.

[0078]

[0079] In the formula, λ is the angle cost function, and c h Let σ be the minimum bounding rectangle height of the ground truth bounding box and the predicted bounding box, and let σ be the Euclidean distance between the center points of the ground truth bounding box and the predicted bounding box. and These are the center coordinates of the ground truth bounding box and the predicted bounding box, respectively.

[0080] (2) Distance cost function

[0081] As α approaches 0, the contribution of the distance cost function decreases significantly; conversely, the closer α is to zero, the greater the contribution. The larger Δ is, the greater its contribution to the distance cost function; γ increases with the increase of angle.

[0082]

[0083] (3) Shape cost function

[0084]

[0085] In the formula, θ∈[2,6], h, w, h gt and w gt These represent the height and width of the predicted bounding box and the ground truth bounding box, respectively.

[0086] (4) IoU cost function

[0087]

[0088] The SIoU loss function is:

[0089]

[0090] The depthwise separable convolution improved Bottleneck module includes: two 1×1 convolutions, one 3×3 convolution, a residual structure, and a feature fusion operation. The 1×1 convolutions are mainly used for dimensionality reduction and expansion, the 3×3 convolutions are used for feature extraction, and the residual structure and feature fusion operation are used to fuse the input feature map and the output of the improved Bottleneck module.

[0091] The Bottleneck module in the original model is improved using depthwise separable convolutions. The standard 3×3 convolutions in the original Bottleneck module are replaced with depthwise separable convolutions. A schematic diagram of the improved Bottleneck module is shown below. Figure 3 As shown.

[0092] Depthwise separable convolution (DWConv) is used to reduce computational cost and model complexity. Its principle is as follows:

[0093] Depthwise separable convolution consists of two stages: depthwise convolution and pointwise convolution. In the channel-wise convolution stage, each channel in the input feature map is convolved only with its corresponding single-channel kernel. The input channels are independent of each other, eliminating feature fusion between channels and preserving the number of channels in the output feature map. Pointwise convolution uses one-dimensional convolution to integrate the output feature maps from all channels, thus changing the number of channels in the output feature map.

[0094] Figure 5 These are diagrams illustrating the ordinary convolution and depthwise separable convolution structures of this invention. Figure 5 As shown, for standard convolution, the input image is D. F ×D F ×M, and N elements of size D K ×D K A convolution operation is performed using a convolution kernel of size ×C, and the output is D. F ×D F ×N.

[0095] The computational cost of standard convolution is:

[0096] D K ·D K ·M·N·D F ·D F

[0097] The above formula shows that the computational cost of standard convolution is proportional to the input channel, output channel, convolution kernel size, and feature map size.

[0098] Depthwise separable convolution consists of two stages: depthwise convolution and point-to-point convolution. The computational cost of the depthwise convolution stage is:

[0099] D K ·D K ·M·D F ·D F

[0100] The computational cost of point-to-point convolution is:

[0101] M·N·D F ·D F

[0102] The computational cost of depthwise separable convolution is:

[0103] D K ·D K ·M·D F ·D F +M·N·D F ·D F

[0104] The ratio of the computational cost of depthwise separable convolution to that of standard convolution is:

[0105]

[0106] As shown in the above formula, depthwise separable convolution has fewer parameter calculations than standard convolution, making the network more lightweight and improving model training speed and real-time performance.

[0107] Step 104: Train and validate the improved YOLOv3-spp network model using the aforementioned UAV dataset to obtain the UAV fire detection model, specifically including:

[0108] The improved YOLOv3-spp network model was trained and validated using the aforementioned UAV dataset under set experimental conditions and training parameters to obtain a UAV fire detection model.

[0109] The experimental conditions were set as follows: PyTorch framework, PyTorch version 1.7.0, programming language Python 3.8.10, hardware configuration Intel i7-9700, 32G memory, GPU GeForce RTX 3080 with 16G video memory, and operating environment Ubuntu 20.04.

[0110] The training parameters were set as follows: BitchSize was set to 16, and epochs were set to 150. The initial learning rate was 0.001, the momentum factor was 0.937, and the SGD optimizer was selected; mAP was used. 50 mAP 75 Metrics such as Params, GFLOPs, and FPS are used as evaluation metrics for model performance.

[0111] Step 105: Input the UAV image test dataset into the UAV fire detection model to obtain the test results and evaluation results of the UAV fire detection model performance.

[0112] The real-time target detection method for unmanned aerial vehicle (UAV) platforms of the present invention further includes:

[0113] The trained, tested, and validated drone fire detection model was deployed to a drone platform. The specific hardware used for deployment was a Jetson Nano B01 development board, compiled using the PyTorch framework within an Ubuntu environment.

[0114] To verify the effectiveness of the introduced loss function SIoU, adaptive multi-feature fusion ASFF-Small, and improved DWBottleneck module in UAV image target detection tasks, this invention conducted a series of ablation experiments on the VisDrone 2021 test set, using YOLOv3-SPP as the baseline algorithm, with mAP... 50 mAP 75 Params, GFLOPs, and FPS were used as evaluation indicators, and the ablation experiment results are shown in Table 1.

[0115] Table 1 Ablation Experiments on the VisDrone Test Set

[0116]

[0117] Note: FPS 640 The 640 indicates that the input image resolution is 640×640.

[0118] The impact of K-means clustering on the baseline model:

[0119] K-means clustering recalculates the anchor boxes that match the dataset based on the characteristics of the dataset, making the anchor boxes more consistent with the characteristics of dense small target images of drones. The anchor box regression is more accurate, and the mAP is improved by 40.7% compared to the baseline model. It has no impact on the amount of computation and the number of floating-point operations, and has a significant effect on improving the model performance.

[0120] The impact of ASFF-Small on the baseline model:

[0121] After the improved ASFF was added to the backbone network of the baseline algorithm, the addition of three feature fusion layers, each utilizing the output features from the previous feature scales, introduced a certain computational overhead, increasing the number of algorithm parameters by 5.7M (1M = 10^35). 6 Meanwhile, the number of floating-point operations increased by 47.5G (1G = 10^6 G). 9 The FPS decreased by 12.6%. However, the mAP increased to 46.2%, reflecting the effectiveness of the multi-scale feature fusion module in adaptively allocating weights to different levels of features, allowing them to play a greater role. Table 2 also shows that the improved ASFF not only enhances the overall performance of the model but also significantly improves the detection performance of small targets. For example, the detection performance of Bicycle, Tricycle, and Awn-tri categories all showed exponential increases, demonstrating ASFF-Small's excellent ability to detect small targets.

[0122] Table 2. Detailed data from the ablation experiments on the test set.

[0123]

[0124]

[0125] The impact of SIoU on the baseline model:

[0126] Introducing the SIoU loss function keeps the number of algorithm parameters and floating-point operations almost unchanged, slightly reduces real-time performance, but also slightly improves mAP and makes the model converge faster. This is because the SIoU loss function takes into account the vector angle between the required regressions, making the regression more accurate and faster.

[0127] The impact of DWBottleneck on the baseline model:

[0128] Improving the Bottleneck module using depthwise separable convolution resulted in a reduction of 38.7M (61%) in algorithm parameters and 108.7G (69.9%) in floating-point computation, while improving FPS by 33.2. This demonstrates that the DW Bottleneck module can reduce model complexity, make the model lightweight, and improve its portability. However, depthwise separable convolution acquires fewer channels compared to ordinary convolution, resulting in fewer dimensionality features in the feature map and thus less effective information. Furthermore, the channels are independent of each other, and features between channels are not fused. Therefore, while making the model lightweight, it reduces the model's feature extraction capability and lowers model performance, with mAP decreasing by 6.1%.

[0129] In summary, the real-time target detection method for UAV platforms proposed in this invention reduces the number of parameters and floating-point operations by 33.2M (53%) and 72.4G (46.6%), respectively, compared to the baseline model. The mAP is improved by 11.3% to 42.5%, and the FPS reaches 66.7. This invention achieves better detection performance while reducing computational overhead and can better guide UAV image target detection tasks.

[0130] Figure 6 This is a schematic diagram of some detection results of the improved YOLOv3-SPP on the test set according to the present invention.

[0131] To verify the detection performance of the improved algorithm in real-world drone scenarios, this invention uses representative and challenging images from the VioDrone 2021 test set for testing, evaluating the algorithm's detection performance, and providing visual analysis.

[0132] This invention selects daytime, nighttime, and occluded images as comparative examples. In daytime scenes, there are many small targets, and the baseline model detects distant houses as trucks, resulting in false detections. The improved algorithm reduces false detections, indicating that the improved YOLOv3-SPP algorithm can effectively reduce the false detection rate. In nighttime image detection, the baseline algorithm misses some vehicles. The improved algorithm has stronger information acquisition capabilities and exhibits stronger anti-interference capabilities, effectively improving the false detection and missed detection situation at night. Experimental results show that the small target detection head designed in this invention can improve the model's small target detection capability, consistent with the above theoretical analysis results, verifying the effectiveness of this invention.

[0133] This invention also provides a real-time target detection system for unmanned aerial vehicle (UAV) platforms. Figure 7 This is a structural diagram of the real-time target detection system for unmanned aerial vehicle (UAV) platforms according to the present invention. Figure 7 As shown, a real-time target detection system for unmanned aerial vehicle (UAV) platforms includes:

[0134] The UAV image dataset acquisition module 201 is used to acquire UAV image datasets;

[0135] The UAV dataset determination module 202 is used to preprocess and manually annotate the UAV image dataset to obtain the UAV dataset;

[0136] Network model building module 203 is used to build the improved YOLOv3-spp network model;

[0137] The drone fire detection model determination module 204 is used to train and validate the improved YOLOv3-spp network model using the drone dataset to obtain the drone fire detection model.

[0138] The model performance test evaluation result determination module 205 is used to input the UAV image test dataset into the UAV fire detection model to obtain the test results and evaluation results of the UAV fire detection model performance.

[0139] The UAV dataset determination module 202 specifically includes:

[0140] The dataset expansion unit is used to expand the UAV image dataset by using image preprocessing methods such as image cropping, rotation, stitching, noise addition, and copy-paste to obtain the expanded UAV image dataset.

[0141] The manual annotation unit is used to manually annotate the expanded UAV image dataset using the annotation software Labelme to obtain the UAV dataset.

[0142] The network model building module 203 specifically includes:

[0143] The network model building unit is used to upsample the feature map after 8x downsampling based on the baseline model yolov3-spp and fuse it with the feature map after 4x downsampling in the backbone network to form a small target detection layer. At the same time, it forms a new multi-scale detection architecture with the original 8x and 16x downsampling feature maps.

[0144] The improved YOLOv3-spp network model introduces the adaptive feature fusion method ASFF. Combined with the new multi-scale detection architecture, the adaptive feature fusion method ASFF-Small is designed. By assigning different weights to feature maps of different scales, the effective feature maps are strengthened, and feature fusion is better achieved.

[0145] The improved YOLOv3-spp network model employs a new loss function SIoU and the Bottleneck module improved using depthwise separable convolution.

[0146] The new loss function SIoU includes the angle cost function, distance cost function, shape cost function, and IoU cost function;

[0147] The depthwise separable convolution improved Bottleneck module includes: two 1×1 convolutions, one 3×3 convolution, a residual structure, and a feature fusion operation. The 1×1 convolutions are mainly used for dimensionality reduction and expansion, the 3×3 convolutions are used for feature extraction, and the residual structure and feature fusion operation are used to fuse the input feature map and the output of the improved Bottleneck module.

[0148] The UAV fire detection model determination module 204 specifically includes:

[0149] The drone fire detection model determination unit is used to train and validate the improved YOLOv3-spp network model using the drone dataset under set experimental conditions and set training parameters, so as to obtain the drone fire detection model.

[0150] The experimental conditions were set as follows: PyTorch framework, PyTorch version 1.7.0, programming language Python 3.8.10, hardware configuration Intel i7-9700, 32G memory, GPU GeForce RTX 3080 with 16G video memory, and operating environment Ubuntu 20.04.

[0151] The training parameters were set as follows: BitchSize was set to 16, and epochs were set to 150. The initial learning rate was 0.001, the momentum factor was 0.937, and the SGD optimizer was selected; mAP was used. 50 mAP 75 Metrics such as Params, GFLOPs, and FPS are used as evaluation metrics for model performance.

[0152] The real-time target detection system for unmanned aerial vehicle (UAV) platforms of the present invention further includes:

[0153] The drone platform deployment module is used to deploy the trained, tested and verified drone fire detection model to the drone platform. The specific deployment hardware is the Jetson Nano B01 development board, which is compiled based on the PyTorch framework in the Ubuntu environment.

[0154] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0155] Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this invention. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A real-time target detection method for unmanned aerial vehicle (UAV) platforms, characterized in that, include: Obtain drone image dataset; The UAV image dataset is preprocessed and manually labeled to obtain the UAV dataset; Establish an improved YOLOv3-spp network model; The improved YOLOv3-spp network model was trained and validated using the aforementioned UAV dataset to obtain a UAV fire detection model. The drone image test dataset is input into the drone fire detection model to obtain the test results and evaluation results of the drone fire detection model's performance. The establishment of the improved YOLOv3-spp network model specifically includes: Based on the baseline model yolov3-spp, the feature map after 8x downsampling is upsampled and fused with the feature map after 4x downsampling in the backbone network to form a small target detection layer. At the same time, it forms a new multi-scale detection architecture with the original 8x and 16x downsampled feature maps. The improved YOLOv3-spp network model introduces the adaptive feature fusion method ASFF. Combined with the new multi-scale detection architecture, the adaptive feature fusion method ASFF-Small is designed. By assigning different weights to feature maps of different scales, the effective feature maps are strengthened, and feature fusion is better achieved. The improved YOLOv3-spp network model employs a new loss function SIoU and the Bottleneck module improved using depthwise separable convolution. The new loss function SIoU includes the angle cost function, distance cost function, shape cost function, and IoU cost function; The depthwise separable convolution improved Bottleneck module includes: two 1×1 convolutions, one 3×3 convolution, a residual structure, and a feature fusion operation. The 1×1 convolutions are mainly used for dimensionality reduction and expansion, the 3×3 convolutions are used for feature extraction, and the residual structure and feature fusion operation are used to fuse the input feature map and the output of the improved Bottleneck module.

2. The real-time target detection method for unmanned aerial vehicle (UAV) platforms according to claim 1, characterized in that, The process of preprocessing and manually annotating the UAV image dataset to obtain the UAV dataset specifically includes: The UAV image dataset is expanded by image preprocessing methods such as image cropping, rotation, stitching, noise addition, and copy-paste to obtain the expanded UAV image dataset. The expanded UAV image dataset was manually annotated using the Labelme annotation software to obtain the UAV dataset.

3. The real-time target detection method for unmanned aerial vehicle (UAV) platforms according to claim 1, characterized in that, The step of training and validating the improved YOLOv3-spp network model using the UAV dataset to obtain the UAV fire detection model specifically includes: The improved YOLOv3-spp network model was trained and validated using the aforementioned UAV dataset under set experimental conditions and training parameters to obtain a UAV fire detection model. The experimental conditions were set as follows: PyTorch framework, PyTorch version 1.7.0, programming language Python 3.8.10, hardware configuration Intel i7-9700, 32G memory, GPU GeForce RTX 3080 with 16G video memory, and operating environment Ubuntu 20.

04. The training parameters were set as follows: BitchSize was set to 16, epochs to 150; initial learning rate was 0.001, momentum factor was 0.937, and SGD was selected as the optimizer; mAP was used. 50 mAP 75 Metrics such as Params, GFLOPs, and FPS are used as evaluation metrics for model performance.

4. The real-time target detection method for unmanned aerial vehicle (UAV) platforms according to claim 1, characterized in that, Also includes: The trained, tested, and validated drone fire detection model was deployed to a drone platform. The specific hardware for deployment was a Jetson Nano B01 development board, which was compiled based on the PyTorch framework in an Ubuntu environment.

5. A real-time target detection system for unmanned aerial vehicle (UAV) platforms, characterized in that, include: The UAV image dataset acquisition module is used to acquire UAV image datasets. The UAV dataset determination module is used to preprocess and manually annotate the UAV image dataset to obtain the UAV dataset; The network model building module is used to build the improved YOLOv3-spp network model. The drone fire detection model determination module is used to train and validate the improved YOLOv3-spp network model using the drone dataset to obtain the drone fire detection model. The model performance test evaluation result determination module is used to input the UAV image test dataset into the UAV fire detection model to obtain the test results and evaluation results of the UAV fire detection model performance; The network model building module specifically includes: The network model building unit is used to upsample the feature map after 8x downsampling based on the baseline model yolov3-spp and fuse it with the feature map after 4x downsampling in the backbone network to form a small target detection layer. At the same time, it forms a new multi-scale detection architecture with the original 8x and 16x downsampling feature maps. The improved YOLOv3-spp network model introduces the adaptive feature fusion method ASFF. Combined with the new multi-scale detection architecture, the adaptive feature fusion method ASFF-Small is designed. By assigning different weights to feature maps of different scales, the effective feature maps are strengthened, and feature fusion is better achieved. The improved YOLOv3-spp network model employs a new loss function SIoU and the Bottleneck module improved using depthwise separable convolution. The new loss function SIoU includes the angle cost function, distance cost function, shape cost function, and IoU cost function; The depthwise separable convolution improved Bottleneck module includes: two 1×1 convolutions, one 3×3 convolution, a residual structure, and a feature fusion operation. The 1×1 convolutions are mainly used for dimensionality reduction and expansion, the 3×3 convolutions are used for feature extraction, and the residual structure and feature fusion operation are used to fuse the input feature map and the output of the improved Bottleneck module.

6. The real-time target detection system for unmanned aerial vehicle (UAV) platforms according to claim 5, characterized in that, The UAV dataset determination module specifically includes: The dataset expansion unit is used to expand the UAV image dataset by using image preprocessing methods such as image cropping, rotation, stitching, noise addition, and copy-paste to obtain the expanded UAV image dataset. The manual annotation unit is used to manually annotate the expanded UAV image dataset using the annotation software Labelme to obtain the UAV dataset.

7. The real-time target detection system for unmanned aerial vehicle (UAV) platforms according to claim 5, characterized in that, The drone fire detection model determination module specifically includes: The drone fire detection model determination unit is used to train and validate the improved YOLOv3-spp network model using the drone dataset under set experimental conditions and set training parameters, so as to obtain the drone fire detection model. The experimental conditions were set as follows: PyTorch framework, PyTorch version 1.7.0, programming language Python 3.8.10, hardware configuration Intel i7-9700, 32G memory, GPU GeForce RTX 3080 with 16G video memory, and operating environment Ubuntu 20.

04. The training parameters were set as follows: BitchSize was set to 16, epochs to 150; initial learning rate was 0.001, momentum factor was 0.937, and SGD was selected as the optimizer; mAP was used. 50 mAP 75 Metrics such as Params, GFLOPs, and FPS are used as evaluation metrics for model performance.

8. The real-time target detection system for unmanned aerial vehicle (UAV) platforms according to claim 5, characterized in that, Also includes: The drone platform deployment module is used to deploy the trained, tested and verified drone fire detection model to the drone platform. The specific deployment hardware is the Jetson Nano B01 development board, which is compiled based on the PyTorch framework in the Ubuntu environment.

Images