Target identification model and method for infrared thermal imaging of high-altitude unmanned aerial vehicle based on YOLOv8

Abstract

According to the YOLOv8-based high-altitude unmanned aerial vehicle infrared thermal imaging target identification model and method, a C2f module in a neck network is replaced by C2f (C2fDSConv) fused with distribution shift convolution, and a detection head at the tail end is replaced by a diverse Branch Block (DBB) detection head, so that the feature extraction of the model is improved, the generalization ability is enhanced, and the defects that the precision identification is not high and the identification accuracy is high can be improved. And leak detection is serious.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to an image recognition method using an unmanned aerial vehicle infrared image and a YOLOv8 network. Background Art

[0002] Infrared images from high-altitude drones play an important role in the civilian sector. In environmental monitoring, they can promptly detect potential forest fire hazards and monitor volcanic activity; in the agricultural sector, they can monitor crop growth to detect pests and diseases and guide irrigation management; during power inspections, they can locate power equipment fault points to ensure safe operation; during search and rescue, in natural disasters or missing persons incidents, they can quickly determine the location of trapped or missing persons even at night or in bad weather.

[0003] The development of deep learning has brought about tremendous changes in the infrared thermal imaging recognition of drones. In the past, the infrared thermal imaging recognition of drones faced many challenges, such as the accurate classification and positioning of targets in complex environments. With the rise of deep learning technology, algorithms such as convolutional neural networks (CNN) have been widely used in the processing of infrared thermal imaging data of drones. Deep learning algorithms can automatically learn the features in infrared thermal imaging and no longer rely on manual feature extraction. Moreover, deep learning can improve the accuracy and efficiency of infrared thermal imaging recognition of drones. Even under complex conditions such as low light and bad weather, it can effectively process infrared thermal imaging data, enhance the application capabilities of drones in environmental monitoring, search and rescue and other fields, and open up broad prospects for the development of infrared thermal imaging recognition of drones.

[0004] Thanks to the efforts of many researchers, high-altitude UAV infrared thermal imaging target recognition based on deep learning has achieved high robustness and accuracy, but its accuracy is still insufficient and there are problems of missed detection, so there is a lot of room for improvement. With the continuous expansion of deep learning, the target recognition of high-altitude UAV infrared thermal imaging will have a higher level of recognition and detection. Summary of the invention

[0005] In view of the problem of insufficient recognition accuracy of traditional high-altitude UAV infrared thermal imaging, the present invention provides a target recognition model and recognition method for high-altitude UAV infrared thermal imaging based on YOLOv8, replaces the C2f module in the neck network with C2f_DSConv, and replaces the terminal detection head with a DBB detection head to improve the problems of low recognition accuracy and serious missed detection.

[0006] The technical solution of the present invention is:

[0007] A target recognition model for infrared thermal imaging of high-altitude unmanned aerial vehicles based on YOLOv8 includes a YOLOv8 network model. The network model includes a Backbone network for extracting features from an input image, a Neck network for enhancing multi-scale feature fusion capabilities, and a Head network for generating a final prediction output. The C2f of the Neck network is replaced by C2f_DSConv, a DBB detection head is added to the Head network, and the output end of C2f_DSConv is connected to the input end of the DBB detection head.

[0008] Preferably, C2f_DSConv uses depthwise separable convolution and produces two feature map outputs, with an input tensor of shape (batch_size, in_channels, height, width). The depthwise separable convolution is: first apply a 3x3 convolution kernel to each input channel separately to generate feature maps with the same number of in_channels, and then use a 1x1 convolution kernel to merge the feature maps and adjust them to the required number of out_channels; the dual feature output paths are: the first output path is the result of the activation function processing, and the second output path is the result directly obtained from the intermediate step, or obtained through a different processing path, such as another convolutional layer or pooling layer.

[0009] Preferably, the detection head of the Head network is replaced with a DBB detection head, and the C2f_DSConv modules of the Neck network are respectively connected to a DBB detection head for dynamic balancing processing. After processing, the feature map is further processed through multiple convolutional layers Conv, and finally the bounding box BBox and category Cls are predicted through the Conv2d layer.

[0010] The target recognition method of high-altitude UAV infrared thermal imaging based on YOLOv8 includes the following steps:

[0011] S1, obtain the data set and divide the data set;

[0012] S2, build a target recognition model for high-altitude UAV infrared thermal imaging based on YOLOv8,

[0013] S21, enhanced multi-scale feature fusion stage, the neck network replaces C2f with C2f_DSConv to reduce the amount of calculation and the number of parameters;

[0014] S22, prediction stage, a DBB detection head is added, and the DBB detection head is connected to C2f_DSConv of the neck network;

[0015] S3 sets the training parameters, trains and evaluates the model of S2, obtains the trained model, inputs the UAV image into the trained model, and outputs the detection results.

[0016] Preferably, in S21, in the fused C2f_DSConv, some convolution layers are replaced by distribution shift convolution layers in the original convolution operation stage. When data flows in the C2f_DSConv module, the distribution shift convolution is adjusted between different components according to the data distribution. In the multi-level feature fusion process, the feature maps of different levels are adjusted according to their respective distributions and then fused, thereby ensuring that the features are always effectively extracted and optimized according to the data distribution.

[0017] Preferably, in S22, the DBB detection head includes multiple parallel branches, each branch has different convolution operations or network structure characteristics, some branches use large convolution kernels to obtain a larger receptive field to capture the global semantic information of the target, and other branches use small convolution kernels to focus more on the extraction of local detail features; the parallel branches process the input feature maps at the same time, and then fuse the different features extracted by each branch. This fusion integrates multiple feature representations, so that the DBB detection head can adapt to different data distributions and enhance the expression ability of the model, thereby improving the accuracy, adaptability and generalization ability of target detection.

[0018] As a preference, precision, recall and average precision (mAP) are used as evaluation indicators, and precision is expressed as:

[0019] Recall is expressed as

[0020] Among them, TP represents true positive examples, FP represents false positive examples, and FN represents positive samples predicted as negative by the model;

[0021] The average precision mAP is expressed as: Among them, AP refers to the average precision of instance segmentation, expressed as

[0022] Compared with the existing technology, the present invention has the following advantages:

[0023] After the C2f module in YOLOv8 is replaced by C2f_DSConv, the present invention enhances the change of data distribution, and the adaptability of the model to the change of data distribution is significantly improved; in addition, it is more accurate in feature extraction, which helps to better capture the subtle features of the target, thereby more effectively extracting the features of small targets and improving the detection rate of small targets; at the same time, in a complex background, the feature distribution of the target may be interfered by the background, and the fused module can adapt to this complex feature distribution, extract the target features more accurately, and reduce false detection and missed detection; because the fused module can adapt to different data distributions, the model will not overly rely on a specific data distribution for feature extraction, which enables the model to perform well in different data sets and different task scenarios, and improves the generalization ability.

[0024] The terminal detection head is replaced with a DBB detection head, which adopts a multi-branch structure and can obtain multiple features. The DBB detection head enhances the expressiveness of the entire model; it adapts to different data distributions, making it more adaptable to different data distributions, improving generalization capabilities, and can work well with different backbone networks. It can further perform diversified feature extraction and processing based on the features provided by the backbone network. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a method step diagram of an embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of a model structure of an embodiment of the present invention;

[0027] Figure 3 is a program-based model structure diagram of an embodiment of the present invention;

[0028] Figure 4 It is a schematic diagram of a better test result before and after the comparison of the present invention;

[0029] Figure 5 It is a schematic diagram of a better test result before and after the comparison of the present invention.

[0030] Specific implementation method

[0031] The following is a further detailed description of the target recognition method of high-altitude drone infrared thermal imaging based on YOLOv8 with specific examples. These examples are only used for comparison and explanation purposes and are not limited to these examples.

[0032] Example 1

[0033] The YOLOv8 network model includes the Backbone network that extracts features from the input image, the Neck network that enhances the multi-scale feature fusion capability, and the Head network that generates the final prediction output. Figures 1 to 3 As shown,

[0034] The main function of the backbone network is to extract features from the input image. Figure 3 The backbone network in contains multiple convolutional layers (Conv) and residual modules (C2f), as well as a spatial pyramid pooling (SPPF) module.

[0035] in,

[0036] Convolutional layer (Conv): used to extract basic features of the image.

[0037] Residual module (C2f): contains multiple convolutional layers to deepen the network while maintaining the resolution of features.

[0038] Spatial Pyramid Pooling (SPPF): used to extract features of different scales and enhance the model's ability to detect objects of different sizes.

[0039] The input image is passed through multiple convolutional layers and residual modules (C2f) for feature extraction, and finally the multi-scale features are extracted through the SPPF module.

[0040] The main function of the neck network (Neck) is to fuse feature maps of different scales to enhance the model's ability to detect multi-scale targets. Figure 3 The neck network in contains multiple convolutional layers (Conv) and C2f_DSConv.

[0041] in,

[0042] C2f_DSConv: Residual module combined with depthwise separable convolution (DSConv) for efficient feature map processing.

[0043] Concat: Concatenate feature maps of different scales to fuse multi-scale information.

[0044] Upsample: Upsampling operation is used to enlarge the low-resolution feature map to the size of the high-resolution feature map.

[0045] Conv: Convolutional layer, used to further process and fuse feature maps.

[0046] The feature map output by the backbone network is processed by C2f_DSConv, and the processed feature map is spliced ​​by the Concat module (the C2f and SPPF of the backbone network are connected to the Concat module of the neck network). The low-resolution feature map is enlarged to the size of the high-resolution feature map through the Upsample module, and finally the feature fusion is performed again through C2f_DSConv and Concat modules.

[0047] The main function of the head network (Head) is to predict bounding boxes and categories from the fused feature maps. Figure 3 The head network in contains multiple DBB detection heads, multiple convolutional layers, and multiple Conv2d layers.

[0048] in,

[0049] DBB: is a dynamic balancing module that optimizes the information interaction between feature maps of different scales. It ensures that features of all scales can be used fairly and effectively, thereby improving the detection performance of the model

[0050] Conv: Convolutional layer, used to further process feature maps.

[0051] Conv2d: Convolutional layer for predicting bounding box (BBox) and class (Cls).

[0052] The output of C2f_DSConv of the neck network is connected to the input of DBB. The feature map output by DBB is processed by two branches: one branch is used for bounding box prediction and the other branch is used for category prediction. The Conv layer further processes the feature map output by DBB to enhance the expressiveness of the features. The Conv2d layer makes the final prediction on the feature map processed by convolution: one branch predicts the bounding box coordinates (BBox) through the Conv2d layer, and the other branch predicts the category (Cls) probability through the Conv2d layer.

[0053] Example 2

[0054] This embodiment discloses a target recognition method for high-altitude UAV infrared thermal imaging based on YOLOv8, including the following steps:

[0055] S1, obtain the data set and divide the data set.

[0056] In this embodiment, the HIT-UAV (High-altitude Infrared Thermal dataset for Unmanned Aerial Vehicle-based object detection) dataset is used, which is a dataset designed for high-altitude infrared thermal imaging target detection of drones. It contains 2866 infrared thermal imaging images, which are taken by drones in different scenes, covering a wide range of object types, flight altitude data, camera viewing angle data, and light intensity. It is divided into training set, test set, and validation set in a ratio of 7:2:1.

[0057] S2, build a target recognition model for high-altitude UAV infrared thermal imaging based on YOLOv8, including

[0058] S21, enhanced multi-scale feature fusion stage, the neck network replaces C2f with C2f_DSConv to reduce the amount of calculation and the number of parameters.

[0059] The C2f_DSConv module is originally responsible for processing input features to extract features that are beneficial to target detection, and includes multiple components working together. The distribution shift convolution can sense the distribution characteristics of the input data and adjust the convolution kernel parameters based on this. In the fused C2f_DSConv, some convolution layers are replaced by distribution shift convolution layers in the original convolution operation stage. When data flows in the C2f_DSConv module, the distribution shift convolution is adjusted between different components according to the data distribution. In the multi-level feature fusion process, the feature maps of different levels are adjusted according to their respective distributions and then fused, thereby ensuring that the features are always effectively extracted and optimized according to the data distribution.

[0060] S22, a DBB detection head is added at the output of the YOLOv8 algorithm.

[0061] The principle of DBB detection head is based on multi-branch structure design. It contains multiple parallel branches, each branch has different convolution operations or network structure characteristics. Some branches use large convolution kernels to obtain a larger receptive field to capture the global semantic information of the target, while other branches use small convolution kernels to focus more on the extraction of local detail features. These branches process the input feature map at the same time, and then fuse the different features extracted by each branch. This fusion combines multiple feature representations, allowing the DBB detection head to adapt to different data distributions and enhance the expression ability of the model, thereby improving the accuracy, adaptability and generalization of target detection.

[0062] S3 sets the training parameters, trains and evaluates the model of S2, obtains the trained model, inputs the UAV image into the trained model, and outputs the detection results.

[0063] In this embodiment, the training batch is set to 16, the number of epochs is set to 200, and the input size defaults to 640×512.

[0064] Precision, recall and average precision (mAP) are used as evaluation indicators. In the experiment, the IoU threshold is set to 0.5 to calculate the average precision, i.e. mAP@0.5. The specific calculation formula is as follows:

[0065] Precision P refers to the proportion of samples predicted by the model to be positive that are actually positive. The calculation formula is:

[0066]

[0067] The recall rate R refers to the proportion of samples that are actually positive and are correctly predicted as positive by the model. The calculation formula is:

[0068]

[0069] TP stands for true positive, which refers to the number of targets correctly detected by the model; FP stands for false positive, which refers to the number of background or other non-target objects that the model mistakenly detects as targets; TN stands for true negative, which is the sample correctly predicted as negative by the model; FN refers to the positive sample predicted as negative by the model. AP refers to the average precision of instance segmentation, which is calculated as:

[0070]

[0071] mAP refers to the mean of the accuracy of multiple categories, and the calculation formula is:

[0072]

[0073] The test data is the test set divided according to S1. The test results are shown in Table 1. Figure 4 and Figure 5 .

[0074] Table 1 Test results

[0075] Method Precision(%) Recall(%) mAP@0.5(%) YOLOv8 82.5 73.5 79.1 Improved YOLOv8 90.8 72.5 80.8

[0076] Table 1 Test results:

[0077] It can be seen that in the YOLOv8 network, the neck network C2f part is replaced with C2f_DSConv; and the terminal detection head is replaced with the DBB detection head. The trained model has higher Precision and mAP@0.5 on the test set than the original network model, and the Recall is slightly reduced. The missed detection is also greatly improved compared with the original network. It can be seen that overall, the method proposed in this patent improves the performance of the model to a certain extent.

[0078] Finally, it should be noted that the above examples are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. The target recognition model of high-altitude UAV infrared thermal imaging based on YOLOv8 is characterized by: The invention comprises a YOLOv8 network model, wherein the network model comprises a Backbone network for extracting features from an input image, a Neck network for enhancing multi-scale feature fusion capability, and a Head network for generating a final prediction output. The C2f of the Neck network is replaced by C2f_DSConv, a DBB detection head is added to the Head network, and the output end of C2f_DSConv is connected to the input end of the DBB detection head.

2. The target recognition model according to claim 1, characterized in that: C2f_DSConv uses depth-wise separable convolution and produces two feature map outputs. The input tensor is of shape (batch_size, in_channels, height, width). The depth-wise separable convolution is: first apply a 3x3 convolution kernel to each input channel separately to generate feature maps with the same number of in_channels, and then use a 1x1 convolution kernel to merge the feature maps and adjust them to the required number of out_channels; The dual-feature output path is: the first output path is the result after the activation function is processed, and the second output path is the result directly obtained from the intermediate step, or obtained through different processing paths.

3. The target recognition model according to claim 2, characterized in that: The detection head of the Head network is replaced with a DBB detection head, and the C2f_DSConv modules of the Neck network are respectively connected to a DBB detection head for dynamic balancing processing. After processing, the feature map is further processed through multiple convolutional layers Conv, and finally the bounding box BBox and category Cls are predicted through the Conv2d layer.

4. The target recognition method of high-altitude UAV infrared thermal imaging based on YOLOv8 is characterized by: The following steps are involved: S1, obtain the data set and divide the data set; S2, build a target recognition model for high-altitude UAV infrared thermal imaging based on YOLOv8, S21, enhanced multi-scale feature fusion stage, the neck network replaces C2f with C2f_DSConv to reduce the amount of calculation and the number of parameters; S22, prediction stage, a DBB detection head is added, and the DBB detection head is connected to C2f_DSConv of the neck network; S3 sets the training parameters, trains and evaluates the model of S2, obtains the trained model, inputs the UAV image into the trained model, and outputs the detection results.

5. The target recognition method according to claim 4, characterized in that: In S21, in the fused C2f_DSConv, some convolution layers are replaced by distribution shift convolution layers in the original convolution operation stage. When data flows in the C2f_DSConv module, the distribution shift convolution is adjusted between different components according to the data distribution. In the multi-level feature fusion process, the feature maps of different levels are adjusted according to their respective distributions and then fused, thereby ensuring that the features are always effectively extracted and optimized according to the data distribution.

6. The target recognition method according to claim 4 or 5, characterized in that: In S22, the DBB detection head contains multiple parallel branches, each of which has different convolution operations or network structure characteristics. Some branches use large convolution kernels to obtain a larger receptive field to capture the global semantic information of the target, while other branches use small convolution kernels to focus more on the extraction of local detail features. The parallel branches process the input feature maps at the same time, and then fuse the different features extracted by each branch.

7. The target recognition method according to claim 4, characterized in that: Taking precision, recall and average precision mAP as evaluation indicators, precision is expressed as: Recall is expressed as Among them, TP represents true positive examples, FP represents false positive examples, and FN represents positive samples predicted as negative by the model; the average precision mAP is expressed as: Among them, AP refers to the average precision of instance segmentation, expressed as

Images