A UAV aerial target detection method based on PCRS-YOLO network

By using the PCRS-YOLO network method in the detection of aerial targets of drones, the four-scale feature fusion structure, CGAFusion module and Rep-CPCA attention mechanism are used to solve the problem of false detection and missed detection in small target detection and complex environments. Improve detection accuracy and model efficiency, and achieve fast and accurate detection on terminal devices.

CN118799766BActive Publication Date: 2024-11-15BEIJING HANSHENG CHUANGYUE TECHNOLOGY CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411283076.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-11-15
Estimated Expiration
2044-09-13

AI Technical Summary

Technical Problem

The existing drone aerial target detection algorithm has problems such as false detection, missed detection and low detection rates in terms of small target detection, complex environment and high computing complexity. Especially in dark environments, target features are difficult to identify, resulting in a decrease in detection accuracy. .

Method used

Using the PCRS-YOLO network-based method, the model's ability to detect small targets by introducing four-scale feature fusion structure, CGAFusion feature fusion module and Rep-CPCA attention mechanism into the neck network, combined with the shared partial parameter detection head, enhance the model's ability to detect small objects and Noise resistance, reducing computational complexity.

Benefits of technology

It significantly improves the accuracy of small object detection, enhances the object detection capability in complex environments, reduces errors, realizes the lightweight and rapid deployment of the model, and ensures efficient object detection on terminal devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118799766B_ABST
    Figure CN118799766B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for detecting aerial target of unmanned aerial vehicle based on PCRS-YOLO network, comprising the following steps: step 1, obtaining a public unmanned aerial vehicle image data set and preprocessing; step 2, configuring a model training environment; step 3, constructing a PCRS-YOLO network model; step 4, loading the constructed PCRS-YOLO network model into a configured training environment, and then using a pre-divided training set and a verification set to train and verify the network model; step 5, using an image to be detected as input, and performing unmanned aerial target detection through the trained and verified PCRS-YOLO network model. This method is applicable to the small target detection algorithm of unmanned aerial vehicle platform, and solves the problems of false detection, missed detection, low detection rate, etc. faced by small target detection in complex environments, and has very important practical significance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and to technologies such as deep learning, image processing, and target detection and recognition, and specifically refers to a method for detecting targets in drone aerial photography based on a PCRS-YOLO network. Background Art

[0002] Traditional aerial photography, which originated in the 1850s, is a mature method in the field of remote sensing images. It uses professional photographic equipment carried by aircraft to obtain surface images. Although traditional aerial photography can cover a large area, it also has disadvantages such as high cost, poor flexibility, and long response time. These factors limit the practicality of traditional aerial photography in specific application scenarios. With the rapid development of drone technology, drone aerial photography has the advantages of low cost, good flexibility, and fast response speed compared to traditional aerial photography. It can more efficiently obtain large-scale, high-resolution high-altitude overhead images. Target detection on high-altitude overhead images taken by drones can help people quickly capture the target features contained in the picture. Therefore, drone aerial target detection has been widely used in environmental monitoring, precision agriculture, and rescue missions.

[0003] In recent years, the rapid development of deep learning technology has shown remarkable results in many fields. This trend has prompted researchers at home and abroad to begin to apply this cutting-edge technology to the field of target detection, aiming to achieve more accurate and efficient detection. Existing target detection algorithms, such as YOLO, Faster R-CNN, RetinaNet, SSD and other algorithms, can achieve high detection performance on medium and large targets, but face multiple challenges in the field of small target detection in drone aerial photography, such as small target size, dark environment, target occlusion and computational complexity of deep learning models. In the field of drone aerial photography target detection, small targets occupy fewer pixels in the image, and their subtle features are easily lost in the complex background after continuous convolution and pooling operations of the deep network. This loss of feature information has an adverse effect on the positioning and classification of small targets in the image. Target occlusion reduces the effective visual information used to identify the target, increases the difficulty of classification and positioning, and thus leads to a decrease in the accuracy of the target detection algorithm. In a dark environment, the image details are unclear and the noise increases due to insufficient light. These factors reduce the visibility of the target and make it difficult to accurately identify the target features. The deep learning model has a large number of parameters due to its complex neural network structure, which requires a huge amount of computing during model training and inference, resulting in slower detection speed, which to some extent reduces the real-time performance of the model in practical applications. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention proposes a UAV aerial target detection method based on the PCRS-YOLO network, which is suitable for the small target detection algorithm of the UAV platform. It can solve the problems of false detection, missed detection, low detection rate and so on faced by small target detection in complex environments, and has very important practical significance.

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] A method for detecting drone aerial photography targets based on PCRS-YOLO network includes the following steps:

[0007] Step 1: Obtain a public drone aerial image dataset and preprocess it;

[0008] Step 2: Configure the model training environment;

[0009] Step 3, construct a PCRS-YOLO network model, wherein the PCRS-YOLO network model is based on the YOLOv8 model, the neck network in the YOLOv8 model adopts a four-scale feature fusion structure, the CGAFusion feature fusion module is introduced into the neck network, the Rep-CPCA attention mechanism module is inserted into the neck network, and the detection head in the detection head network is replaced with a detection head with shared partial parameters;

[0010] Step 4: Load the constructed PCRS-YOLO network model into the configured training environment, initialize the parameter file of the network model, adjust the input image size to 640×640, set the number of training rounds to 200, the batch-size to 4, the category of the detection object to 10, and then use the pre-divided training set and verification set to train and verify the network model;

[0011] Step 5: Take the image to be detected as input and perform drone aerial target detection through the trained and verified PCRS-YOLO network model.

[0012] Preferably, the preprocessing method is: formatting the acquired data set to obtain a data set that meets the requirements of the PCRS-YOLO network model, and then dividing the data set that meets the requirements into a training set and a validation set.

[0013] Preferably, in step 2, the configured training environment is: cuda11.7, deep learning framework pytorch1.13.1, Intel core i9-13900ks CPU, 128G memory, GPU is NVIDIA GeForce RTX4090, and video memory is 24G.

[0014] Preferably, the four-scale feature fusion structure is based on the original YOLOv8 model. First, the 80×80 feature map output by upsampling Neck in the YOLOv8 model is output as a 160×160 feature map after one upsampling operation, and then it is spliced ​​with the shallow feature map that is 4 times downsampled in Backbone in the YOLOv8 network to generate a new feature map P2 rich in small target information. Finally, P2 and the four layers of feature maps P3, P4 and P5 generated by the YOLOv8 model are detected.

[0015] In the above technical solution, Neck adopts a four-scale feature fusion structure. The P2 layer feature map generated by this structure effectively addresses the problem of small target information loss caused by the output of only three feature detection maps P3, P4, and P5 after the input image has undergone multiple convolution and downsampling operations of the traditional model.

[0016] Preferably, the neck network introduces a CGAFusion feature fusion module after each C2f module based on the four-scale feature fusion structure.

[0017] Preferably, the method for realizing feature fusion through the CGAFusion feature fusion module is:

[0018] The deep features output by the previous layer C2f of the CGAFusion feature fusion module in Neck and the shallow features with the same size of the C2f output features in Backbone are used as the input of the CGAFusion feature fusion module, and the input shallow features and deep features are added. Then, the added feature X is passed to the CGA module to calculate its adaptive weight and perform adaptive fusion.

[0019] Preferably, the adaptive fusion formula is expressed as:

[0020]

[0021] in, represents a normal convolution with a kernel size of 1×1, represents the shallow feature map, represents the deep feature map, Represents the weight calculated by the CGA module, Represents the new feature map obtained by the CGAFusion feature fusion module.

[0022] In the above technical solution, the CGAFusion feature fusion module is introduced in the Neck of the model. This module uses the content-guided attention mechanism (CGA) to adaptively weighted fuse the shallow semantic information and deep semantic information in the network model, and finally outputs a feature map with richer feature information.

[0023] Preferably, the Rep-CPCA attention mechanism module includes CPCA and RepConv, and a Rep-CPCA attention mechanism module is introduced after each of the P2, P3 and P4 feature maps generated by the four-scale feature fusion structure. The CPCA uses the channel attention mechanism to perform average pooling and maximum pooling operations on the feature map passed into the Rep-CPCA attention mechanism module. The aggregated spatial information is sent to a shared multi-layer perceptron. After being processed by the shared multi-layer perceptron, a channel attention map is generated. The CPCA calculates the channel prior by element-by-element multiplication of the feature map passed into the Rep-CPCA attention mechanism module and the features of the channel attention map. The channel prior is sent to the deep convolution module and converted into a spatial attention map. On this basis, the spatial attention map is channel-mixed with 1×1Conv. Finally, the optimized feature map is output by element-by-element multiplication of the result of the channel mixing with the channel prior. Then, the feature map passed into the Rep-CPCA attention mechanism module is further extracted through the residual structure of RepConv and the obtained feature map is added to the feature map output by CPCA to supplement the lost original information. In the above technical solution, the Rep-CPCA attention mechanism module is inserted into the Neck of the model. Rep-CPCA introduces the RepConv residual structure based on the channel prior convolutional attention (CPCA). This module can not only use CPCA to effectively extract channel information and spatial information in the feature map, but also supplement the original image information lost in the convolution process through the RepConv residual structure.

[0024] Preferably, the shared partial parameter detection head simplifies the two parallel and identical 3×3 convolution structures in the original detection head into a shared convolution structure, and then divides it into two paths to perform a 1×1 convolution to extract information, and finally calculates Box.loss and Cls.loss respectively, and then replaces the shared convolution in the shared partial parameter detection head with the residual heavy parameter partial convolution.

[0025] In the above technical solution, the head is replaced with a new shared partial parameter detection head (SPdetect). SPdetect first simplifies the structure of two identical ordinary convolutions in the original detection head into a shared convolution structure. Then the shared convolution is replaced with an innovative residual re-parameter partial convolution (Res-Rep-Conv). This module effectively reduces the number of parameters and calculations of the model.

[0026] Preferably, the residual re-parameterized partial convolution comprises PConv, RepConv and a residual structure connected in sequence.

[0027] Preferably, in step 3, the training method is: dividing the preprocessed data set into a training set and a validation set, and setting the number of model training rounds to 200 rounds, with 4 pictures input for one training, observing the training log in real time through Wandb during the training process, and saving the training results after the training is completed.

[0028] The present invention has the following characteristics and beneficial effects:

[0029] (1) In order to enhance the model’s ability to detect small targets, the present invention adopts a four-scale feature fusion structure in Neck. The P2 layer feature map generated by this structure has more feature information of small targets, thereby significantly improving the detection accuracy of the network model for small-size targets.

[0030] (2) In order to cope with the challenges brought by dark environments and target occlusion, the present invention introduces a CGAFusion feature fusion module in Neck, which transmits the shallow semantic information in Backbone and the deep semantic information in Neck to the content-guided attention mechanism (CGA) for adaptive weight fusion, which improves the network's ability to capture image details and enhances noise suppression performance. Therefore, under conditions of insufficient light or target occlusion, CGAFusion effectively reduces detection errors and improves target detection accuracy.

[0031] (3) In order to better capture local features in the image, the present invention adds a re-parameterized channel prior convolutional attention mechanism (Rep-CPCA) to Neck. Rep-CPCA is a comprehensive structure of RepConv and CPCA, which can effectively extract feature information and multi-scale spatial information of each channel. Therefore, this innovation can more effectively focus on important areas and enhance the feature recognition ability of the model.

[0032] (4) In order to reduce the number of model parameters and computational complexity, the present invention replaces the original detection head (Detect) with a new shared partial parameter detection head (SPdetect) in the detection head (Head). This innovation reduces the computational complexity of the model while maintaining the stability of the detection accuracy, achieving lightweight models and ensuring that they can be quickly and accurately deployed on terminal devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0034] Figure 1 It is a diagram of the overall model structure of an embodiment of the present invention.

[0035] Figure 2 for Figure 1 CGAFusion feature fusion structure diagram.

[0036] Figure 3 for Figure 1 Diagram of the Rep-CPCA structure.

[0037] Figure 4 for Figure 1 Figure 2. SPdetect structure diagram.

[0038] Figure 5 This is the original model detection diagram.

[0039] Figure 6 This is a correct detection diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0040] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0041] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings.

[0042] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0043] The present invention provides a method for detecting aerial targets of unmanned aerial vehicles based on a PCRS-YOLO network, wherein P represents a four-scale feature fusion structure, C represents a CGAFusion feature fusion module, R represents a Rep-CPCA attention mechanism, and S represents a shared partial parameter detection head (SPdetect). The main contents of the present invention are: (1) In order to overcome the challenge of less RGB information of small-scale targets, the present invention adopts a four-scale feature fusion structure in the neck network (Neck) of the model. This structure not only ensures efficient detection of large-scale targets, but also significantly improves the detection accuracy of the network for small-size targets, thereby achieving accurate detection of small targets. (2) In order to overcome the influence of dark environments and target occlusion, the present invention introduces a CGAFusion feature fusion module in Neck, which improves the network's ability to represent detailed information and anti-noise performance, and reduces the occurrence of false detection and missed detection events in dark environments and target occlusion to a certain extent. (3) In order to better capture local features in the image, the present invention adds a re-parameterized channel prior convolution attention mechanism (Rep-CPCA) in Neck. This improvement can help the network better capture important features and improve the ability to represent features. (4) In order to reduce the computational complexity of the deep learning model and make it easier to deploy on the terminal, the present invention replaces the original detection head (Detect) with a new shared partial parameter detection head (SPdetect) in the detection head (Head). SPdetect not only greatly reduces the computational complexity of the model, but also ensures that the target detection accuracy is only slightly affected, achieving a balance between performance and efficiency.

[0044] The specific steps of the present invention are as follows:

[0045] Step 1. Data acquisition:

[0046] In this embodiment, the VisDrone2019 dataset released by the Machine Learning and Data Mining Laboratory of Tianjin University is used. The dataset contains 10,209 pictures, covering different scenes, weather and lighting conditions in 14 different cities in China. The pictures in the dataset have been carefully divided, of which 6,471 pictures were selected as training sets, 548 as validation sets, and 1,610 as test sets, which together constitute the data basis of this experiment. The VisDrone2019 dataset includes ten categories including pedestrians, cars, vans, trucks and motorcycles, and contains a total of 2.6 million label information. The complexity and diversity of this dataset provides an ideal testing platform for evaluating and optimizing target detection algorithms.

[0047] Step 2. YOLOv8 model dataset configuration

[0048] After completing the preprocessing of the dataset, we then build the dataset.yaml configuration file corresponding to the dataset, and write the paths of the training set and validation set as well as the number of categories of the VisDrone2019 dataset in the dataset.yaml file. Then, we modify the default.yaml file of the YOLOv8 source code according to the experimental requirements, adjust the input image size to 640×640, set the number of training rounds to 200, the batch-size to 4, and the number of categories of the detected objects to 10.

[0049] Step 3. Improve the original YOLOv8 model structure and combine it with the attached Figure 1 Description: The following improvements are mainly made:

[0050] (1) After the input image has been convolved and downsampled multiple times by the YOLOv8 model, three output feature maps are obtained. A large amount of small target information is lost in the feature maps. Therefore, passing the three feature maps output by the YOLOv8 model to the detection head for target detection will cause the problem of misdetection and missed detection of small targets. In order to solve the above problem, the present invention proposes a four-scale feature fusion structure in Neck, as shown in the attached figure. Figure 1 As shown in the figure. The structure first upsamples the 80×80 (H×W, the same below) feature map output by the YOLOv8 model to 160×160, and then concatenates it with the shallow feature map that has been downsampled 4 times in Backbone to generate a new feature map P2 rich in small target information. Finally, the four layers of feature maps P2, P3, P4 and P5 are detected. Although the parameters and calculation amount of the model are increased, the detection accuracy of small targets is significantly improved, while maintaining the effective recognition of large-scale targets.

[0051] (2) The present invention introduces the CGAFusion feature fusion module into Neck, as shown in the attached figure. Figure 2 As shown in FIG, based on the four-scale feature fusion structure, a CGAFusion feature fusion module is introduced after each C2f module in the neck network. The deep features output by the previous C2f of the CGAFusion feature fusion module in the neck and the shallow features with the same output features of C2f in the backbone are used as the input of the CGAFusion feature fusion module, and the input shallow features and deep features are added (Add). Then, the added feature X is passed to the CGA module to calculate its adaptive weight, as shown in FIG. Figure 2 The specific process of the CGA module is shown in equations (1)-(4):

[0052] r (1)

[0053] (2)

[0054] (3)

[0055] (4)

[0056] In formula (1) represents a normal convolution with a kernel size of 7×7, Indicates the channel splicing operation, Represents the input features Do global average pooling across spatial dimensions, Represents the input features Do global max pooling across spatial dimensions, Represents the weight of spatial information.

[0057] In formula (2) represents a normal convolution with a kernel size of 1×1, represents the ReLU activation function, Represents the input features Do global average pooling across channel dimensions, Represents the channel information weight.

[0058] In formula (3) and Represent the spatial information weight and channel information weight respectively, Represents the spatial and channel fusion weights.

[0059] In formula (4) represents the sigmoid operation, represents a grouped convolution with a kernel size of 7×7, Represents a channel shuffle operation, Represents a channel concatenation operation.

[0060] Finally, as attached Figure 2 As shown, the weights calculated by the CGA module are used to adaptively fuse shallow features and deep features. The fusion process is shown in formula (5):

[0061] (5)

[0062] In formula (5) represents a normal convolution with a kernel size of 1×1, represents the shallow feature map, represents the deep feature map, Represents the weight calculated by the CGA module, Represents the new feature map obtained by the CGAFusion feature fusion module

[0063] The CGAFusion feature fusion module generates feature maps rich in more detailed information through its advanced fusion technology. This module not only enhances the network's ability to recognize subtle features in images, but also improves the stability and reliability of the model in complex environments.

[0064] (3) The present invention adds a Rep-CPCA attention mechanism to Neck. Figure 3 As shown, the Rep-CPCA attention mechanism module includes CPCA and RepConv, and a Rep-CPCA attention mechanism module is introduced after each of the P2, P3 and P4 feature maps generated by the four-scale feature fusion structure. The CPCA uses the channel attention mechanism to perform average pooling and maximum pooling operations on the feature map passed into the Rep-CPCA attention mechanism module. The aggregated spatial information is sent to a shared multi-layer perceptron. After being processed by the shared multi-layer perceptron, a channel attention map is generated. The CPCA calculates the channel prior by element-by-element multiplication of the feature map passed into the Rep-CPCA attention mechanism module and the features of the channel attention map. The channel prior is sent to the deep convolution module and converted into a spatial attention map. On this basis, the spatial attention map is channel-mixed with 1×1Conv. Finally, the optimized feature map is output by element-by-element multiplication of the result of channel mixing with the channel prior. Then, the feature map passed into the Rep-CPCA attention mechanism module is further extracted through the residual structure of RepConv and the feature map obtained is added to the feature map output by CPCA to supplement the lost original information.

[0065] Specifically, Rep-CPCA consists of two modules, CPCA and RepConv, and combines the characteristics of the two. The overall structure of channel prior convolutional attention (CPCA) is a sequential structure of channel attention and spatial attention. First, CPCA uses the channel attention mechanism to perform operations such as average pooling and maximum pooling on the feature map to effectively aggregate spatial information from the feature map. Then, this aggregated spatial information is fed into a shared multi-layer perceptron (MLP), and after processing, a channel attention map is generated, which maps the importance of each channel. Then CPCA calculates the channel prior by element-by-element multiplication of the features of the original input feature map and the channel attention map, which significantly enhances the expressiveness of the features. Subsequently, the channel prior is fed into the depth-wise convolution module, where it is converted into a spatial attention map, further refining the key areas in the feature map. On this basis, the convolution module performs channel mixing based on the spatial attention map, which optimizes the integration of features in the channel dimension. Finally, by multiplying the result of channel mixing by the channel prior element by element, CPCA extracts a more refined feature output, providing a rich feature representation for the model. Reparameterized convolution (RepConv) is an efficient convolutional neural network component. Its core principle is to reparameterize the parameters of the branch to the main branch in the inference stage by sharing the convolution kernel to improve the efficiency of the model. Therefore, the residual structure of RepConv retains the most original information of the image while reducing the number of parameters. The introduction of the Rep-CPCA attention mechanism enables the model to focus on the key areas in the image, improving the feature representation ability and the overall performance of the model.

[0066] (4) The present invention replaces the original detection head (Detect) with a new shared partial parameter detection head (SPdetect) in the detection head (Head). The original detection head has two branches, each of which first extracts information through two 3×3 convolutions and one 1×1 convolution, and finally calculates Box.loss and Cls.loss respectively. Figure 4 As shown in the figure, SPdetect simplifies the two parallel and identical 3×3 convolution structures in the original detection head into a shared convolution structure, and then divides the two paths into a 1×1 convolution to extract information, and finally calculates Box.loss and Cls.loss respectively. Then the number of convolutions in each detection head is reduced from 4 to 2, which greatly reduces the number of model parameters. Then the shared convolution in the detection head is replaced by the residual re-parameterized partial convolution (Res-Rep-PConv), as shown in the figure. Figure 4As shown in the figure, the convolution is improved from PConv, RepConv and residual structure. The basic principle of partial convolution (PConv) is to use the redundancy of feature maps, apply conventional convolution only on part of the input channel for spatial feature extraction, keep the remaining channel parts unchanged, and splice the feature maps processed by partial convolution with the remaining feature maps that have not been convolved to obtain the output feature map. Therefore, PConv greatly reduces the number of floating points (FLOPs) by calculating on fewer channels, but the target detection accuracy may be greatly affected. For example, if the partial rate is set to 1 / 4, the calculation amount of PConv is only 1 / 16 of that of conventional convolution, but the accuracy may drop by about 2%. Therefore, the present invention introduces RepConv to replace the conventional convolution in PConv. As shown in the figure, RepConv uses structural reparameterization technology to make the multi-branch structure equivalent to a single-branch structure in the inference stage, avoiding the problem of a large number of redundant features and gradient disappearance in the conventional convolution single-branch structure, and ensuring that the detection accuracy of the model is not reduced while reducing the amount of calculation. In addition, both PConv and RepConv perform convolution on only some channels, and no convolution operation is performed on the remaining channels for feature extraction. Therefore, the residual structure is introduced to perform 1×1 convolution on the original image to obtain the feature map of the remaining channels, and then it is concatenated with the feature map processed by some convolution to obtain the feature map with rich information.

[0067] Step 4. Train with the modified model

[0068] The VisDrone2019 dataset was preprocessed and divided into a training set of 6471 images and a validation set of 548 images. The number of model training rounds was set to 200, and the number of images input for each training round was 4. The modified model was then used for training. During the training process, the training log was observed in real time through Wandb, and the training results were saved after the training was completed.

[0069] Step 5. Test with the modified model

[0070] The image to be detected is taken as input, and the trained and verified PCRS-YOLO network model is used to detect drone aerial targets.

[0071] Specifically, in the Backbone network, the image to be detected is first taken as input, and a 320*320*64*W feature map is obtained through Conv0; the feature map output by Conv0 is input to Conv1 to obtain a 160*160*128*W feature map; the feature map output by Conv1 is input to C2f2 to obtain a 160*160*128*W feature map through feature fusion; the feature map output by C2f2 is input to Conv3 to obtain a 80*80*256*W feature map; the feature map output by Conv3 is input to C2f4 to obtain a 80*80*256*W feature map through feature fusion. The feature map of 0*256*W is input into Conv5; the feature map output by C2f4 is input into Conv5 to obtain a feature map of 40*40*512*W; the feature map output by Conv5 is input into C2f6 to obtain a feature map of 40*40*512*W through feature fusion; the feature map output by C2f6 is input into Conv7 to obtain a feature map of 20*20*1024*W; the feature map output by Conv7 is input into C2f8 to obtain a feature map of 20*20*1024*W through feature fusion; the feature map output by C2f8 is input into SPPF9 to obtain a feature map of 20*20*1024*W.

[0072] In the neck network (Neck), the feature map output by SPPF is upsampled by Upsample10 to obtain a feature map of 40*40*1024*W; the feature map output by Upsample10 and C2f6 is concatenated by Concat11 to obtain a feature map of 40*40*1536*W; the feature map output by Concat11 is input to C2f12 to obtain a feature map of 40*40*512*W through feature fusion; the feature map output by C2f12 and C2f6 is input to the CGAFusion13 feature fusion module to obtain a feature map of 40*40*512*W; the CGAF is upsampled by Upsample14 The feature map output by usion13 is upsampled to obtain a feature map of 80*80*512*W; the feature map output by Upsample14 and C2f4 is concatenated through Concat15 to obtain a feature map of 80*80*768*W; the feature map output by Concat15 is input to C2f16 to obtain a feature map of 80*80*256*W through feature fusion; the feature map output by C2f16 and C2f4 is input to the CGAFusion17 feature fusion module for feature fusion to obtain a feature map of 80*80*256*W; the feature map output by CGAFusion17 is upsampled through Upsample18 to obtain a feature map of 160*1 60*256*W feature map; The feature maps output by Upsample18 and C2f2 are concatenated through Concat19 to obtain a 160*160*384*W feature map; The feature map output by Concat19 is input into C2f20 to obtain a 160*160*128*W feature map (i.e., feature map P2) through feature fusion; The feature maps output by C2f20 and C2f2 are input into the CGAFusion21 feature fusion module to obtain a 160*160*128*W feature map; The feature map output by CGAFusion21 is input into Rep-CPCA22 to obtain a 160*160*128*W feature map; The feature map output by Rep-CPCA22 is input to Conv23 to obtain a feature map of 80*80*128*W; the feature map output by Conv23 and the feature map output by CGAFusion17 are spliced ​​through Concat24 to obtain a feature map of 80*80*384*W; the feature map output by Concat24 is input to C2f25 to obtain a feature map P3 of 80*80*256*W through feature fusion; the feature map output by C2f25 is input to Rep-CPCA26 to obtain a feature map of 80*80*256*W; the feature map output by Rep-CPCA26 is input to Conv27 to obtain a feature map of 40*40*256*W;The feature map output by Conv27 and the feature map output by CGAFusion13 are concatenated through Concat29 to obtain a 40*40*768*W feature map; the feature map output by Concat29 is input to C2f29 to obtain a 40*40*512*W feature map P4 through feature fusion; the feature map output by C2f29 is input to Rep-CPCA30 to obtain a 40*40*512*W feature map; the feature map output by Rep-CPCA30 is input to Conv31 to obtain a 20*20*512*W feature map; the feature map output by Conv31 and the feature map output by SPPF9 are concatenated through Concat32 to obtain a 20*20*1538*W feature map; the feature map output by Concat29 is input to C2f33 to obtain a feature map P5 through feature fusion. ;

[0073] In the Head network, the feature map output by Rep-CPCA22, the feature map output by Rep-CPCA26, the feature map output by Rep-CPCA30, and the feature map P5 output by C2f33 are respectively input into four shared partial parameter detection heads (SPdetect).

[0074] Furthermore, in order to prove the effectiveness of the technical solution of the present invention, the effects achieved by the present invention are described in detail below in combination with the ablation experiment results and the attached figure of the image detection effect. In order to test the effect of each module on the model improvement, the present invention conducted the following ablation experiment, and the experimental results are shown in Table 1:

[0075] Table 1 Detection results after using different improvement strategies

[0076]

[0077] Note: √ indicates that the module is used

[0078] The ablation experiment results show that after model B uses a four-layer feature fusion structure to generate the P2 feature map, the mAP of model B is higher than that of the baseline model A (YOLOv8n). 50 The improvement was 2.9%, but the floating point FLOPs also increased by 4G, nearly 50%. Then, based on model B, the CGAFusion feature fusion module was introduced for improvement. Compared with model B, model C had a higher mAP. 50 It increased by 0.6%, but the floating point number only increased by 0.5G. Then, based on model C, the Rep-CPCA attention mechanism was introduced to improve it. The mAP of model D was compared with the baseline model. 50The improvement was 4.4%, but the floating point number increased by 6.6G, nearly 80%. Therefore, the present invention introduces the SPhead detection head based on model D and proposes the final improved model. Compared with the baseline model, the improved model has a mAP of 50 Improved by 3.3%, mAP 50-95 It increased by 2.4%, the number of parameters decreased by 0.09M, but the floating point number only increased by 1.5G.

[0079] In order to demonstrate the effect of the invention, Figure 5 and Figure 6 Description: Compare the detection image of the improved model with the detection image of the original model. Figure 5 is the original model detection graph, Figure 6 To improve the model detection diagram. Figure 5 As shown in the figure, in the case of small target detection, the original model misdetected the van as a car and the shed as a van. Figure 6 To improve the model without false detection, it can accurately detect each target comparison Figure 5 and Figure 6 ,It can be seen that the improved model has better ,detection performance than the original model in small target detection, dark ,environment, target occlusion and other backgrounds.

[0080] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A UAV aerial target detection method based on PCRS-YOLO network, characterized in that: The steps include: Step 1: Obtain a public drone aerial image dataset and preprocess it; Step 2: Configure the model training environment; Step 3, construct a PCRS-YOLO network model, wherein the PCRS-YOLO network model is based on the YOLOv8 model, the neck network in the YOLOv8 model adopts a four-scale feature fusion structure, the CGAFusion feature fusion module is introduced into the neck network, the Rep-CPCA attention mechanism module is inserted into the neck network, and the detection head in the detection head network is replaced with a detection head with shared partial parameters; The four-scale feature fusion structure is based on the original YOLOv8 model. First, the 80×80 feature map output by Neck in the YOLOv8 model is upsampled once to output a 160×160 feature map, and then it is spliced ​​with the shallow feature map that is 4 times downsampled in the Backbone in the YOLOv8 network to generate a new feature map P2 rich in small target information. Finally, P2 and the feature maps P3, P4 and P5 generated by the YOLOv8 model are detected. The neck network is based on the four-scale feature fusion structure, and a CGAFusion feature fusion module is introduced after each C2f module; The method for implementing feature fusion through the CGAFusion feature fusion module is: The deep features output by the previous layer C2f of the CGAFusion feature fusion module in Neck and the shallow features with the same size as the output features of C2f in Backbone are used as the input of the CGAFusion feature fusion module, and the input shallow features and deep features are added. Then, the added feature map X is passed to the CGA module to calculate its adaptive weight and perform adaptive fusion. The adaptive fusion formula is expressed as: F fuse =C 1×1 (F low *W+F high *(1-W)+F low +F high ) Among them, C 1×1 represents a normal convolution with a kernel size of 1×1, F low represents the shallow feature map, F high represents the deep feature map, represents the weight calculated by the CGA module, Represents the new feature map obtained by the CGAFusion feature fusion module; The Rep-CPCA attention mechanism module includes CPCA and RepConv, and a Rep-CPCA attention mechanism module is introduced after each of the P2, P3 and P4 feature maps generated by the four-scale feature fusion structure. The CPCA uses the channel attention mechanism to perform average pooling and maximum pooling operations on the feature map passed into the Rep-CPCA attention mechanism module to obtain aggregated spatial information. The aggregated spatial information is sent to a shared multi-layer perceptron. After being processed by the shared multi-layer perceptron, a channel attention map is generated. The CPCA calculates the channel prior by element-by-element multiplication of the feature map passed into the Rep-CPCA attention mechanism module and the features of the channel attention map. The channel prior is sent to the deep convolution module and converted into a spatial attention map. On this basis, the spatial attention map is channel-mixed with 1×1Conv. Finally, the optimized feature map is output by element-by-element multiplication of the channel mixing result and the channel prior. Then, the feature of the original feature map is further extracted through the residual structure of RepConv and the feature map obtained is added to the feature map output by CPCA to supplement the lost original information. Step 4: Load the constructed PCRS-YOLO network model into the configured training environment, modify the parameter file of the network model, and use the pre-divided training set and verification set to train and verify the network model; Step 5: Take the image to be detected as input and perform drone aerial target detection through the trained and verified PCRS-YOLO network model.

2. According to claim 1, a method for detecting drone aerial photography targets based on PCRS-YOLO network, characterized in that: The preprocessing method is: formatting the acquired data set to obtain a data set that meets the requirements of the PCRS-YOLO network model, and then dividing the data set that meets the requirements into a training set and a validation set.

3. The method for detecting drone aerial photography targets based on PCRS-YOLO network according to claim 1, characterized in that: In step 2, the configured training environment is: cuda11.7, deep learning framework pytorch1.13.1, Intelcore i9-13900ks CPU, 128G memory, GPU is NVIDIA GeForce RTX 4090, and the video memory is 24G.

4. The method for detecting drone aerial photography targets based on PCRS-YOLO network according to claim 1, characterized in that: The shared partial parameter detection head simplifies the two parallel and identical 3×3 convolution structures in the original detection head into a shared convolution structure, and then divides it into two paths to perform a 1×1 convolution to extract information, and finally calculates Box.loss and Cls.loss respectively, and then replaces the shared convolution in the shared partial parameter detection head with the residual heavy parameter partial convolution.

5. The method for detecting drone aerial photography targets based on PCRS-YOLO network according to claim 4, characterized in that: The residual re-parameterized partial convolution includes PConv, RepConv and a residual structure connected in sequence.

6. The method for detecting drone aerial photography targets based on PCRS-YOLO network according to claim 1, characterized in that: In step 3, the training method is: divide the preprocessed data set into a training set and a validation set, set the number of model training rounds to 200, input 4 pictures for one training, observe the training log in real time through Wandb during the training process, and save the training results after the training is completed.

7. The method for detecting drone aerial photography targets based on PCRS-YOLO network according to claim 1, characterized in that: In the step 4, modifying the parameter file of the network model includes initializing the parameter file of the network model, adjusting the input image size to 640×640, setting the number of training rounds to 200, the batch-size to 4, and the category of the detection object to 10.

Citation Information

Cited By

  • DA-YOLOv11-based unmanned aerial vehicle aerial image target detection method

    CN120014487A