Unmanned aerial vehicle small target detection method, electronic equipment and storage medium

Abstract

The invention relates to the field of image processing target detection, in particular to an unmanned aerial vehicle small target detection method, electronic equipment and a storage medium, and the method comprises the steps: obtaining an image data set, carrying out the preprocessing of each image in the image data set, and enabling the image data set to be composed of an image shot by an unmanned aerial vehicle and the small target verification data of the corresponding image; inputting the preprocessed image into a network detection model, calculating a loss function of an output result of the network detection model and actual small target verification data, and training parameters of the network detection model until the parameters are optimal; and inputting the to-be-detected image into an optimal detection model, and outputting a small target detection result of the unmanned aerial vehicle image, the optimal detection model being a network detection model after training and adjusting model parameters to be optimal, and enhancing the positioning capability and detection capability of the model to the small target.

Description

Technical Field

[0001] The present invention relates to the field of image processing target detection, and in particular to a method for detecting small targets of unmanned aerial vehicles, electronic equipment and storage medium. Background Art

[0002] Small target detection refers to detecting small-sized target objects in an image, which are usually smaller than 1 / 10 of the image size or smaller. This concept is of great significance in the field of computer vision because it involves a variety of application scenarios, such as security monitoring, intelligent transportation, drones, etc. The challenge of small target detection is that these targets usually carry less information, such as limited appearance information such as texture, shape, color, and occupy fewer pixels in the image. After multiple downsampling, the resolution of small targets is further reduced, and the feature information is gradually weakened, resulting in increased detection difficulty. Therefore, general target detectors have low detection accuracy for small targets.

[0003] The significance of small target detection is that it can improve the application scope of the technology and help better understand the details in the image. Small target detection has a wide range of applications in daily life and industrial applications, such as traffic monitoring, medical image analysis, drone aerial photography, etc. For example, in the field of traffic monitoring, small target detection can be used to identify traffic lights, license plates, etc.; in the field of medical image analysis, small target detection can be used to identify tiny tumor cells, etc.; in the field of autonomous driving, small target detection can be used to identify tiny obstacles to make up for the difficulty of lidar detection.

[0004] Among them, the application scenarios of drone small target detection are very wide. For example, in urban traffic monitoring, drones can patrol over the city, detect vehicles and pedestrians through models, and assist in traffic management; in road safety assessment, drones are used to capture road conditions, detect violations of pedestrians and vehicles, and thus assess road safety conditions; in disasters or emergencies, drones can quickly detect trapped people and vehicles and guide rescue operations. However, due to the unstable flight altitude of drones and the wide image field, the objects in drone aerial images are of different scales, there are many small objects, the density is high, and there are occlusions between objects, which are difficult to capture accurately. The above problems have brought challenges to drone aerial target detection. Therefore, accurately handling various complex scenarios of drone target detection has important theoretical significance and application value.

[0005] At present, the target detection technology based on deep learning is mainly divided into two types: the first is a two-stage method based on candidate regions, represented by Fast R-CNN and Faster R-CNN. The second is a single-stage method, represented by SSD and YOLO series algorithms. The current target detection algorithm usually focuses on detection in general scenes. There are a large number of densely distributed small targets in drone aerial images. Due to the small target pixel information, fuzzy feature texture, and complex and diverse background environment, detection is more difficult and it is difficult to achieve the best detection effect. The target detection method based on CNN often only focuses on local information when extracting features, while the target detection method based on Transformer has a stronger global attention mechanism, which improves the target detection accuracy, but at the same time the model parameter volume is also relatively large. Therefore, the present invention uses CNN to reduce the amount of self-attention calculation in Transformer, and at the same time combines the local feature extraction ability of CNN with the global feature extraction ability of Transformer to enhance the feature expression ability of target detection. Considering the real-time detection of small targets of drones, and the pure CNN yolo algorithm is fast, simple and intuitive, and has strong generalization ability, the present invention is innovatively designed based on the latest yolov10. Summary of the invention

[0006] The purpose of the present invention is to provide a small target detection method based on an unmanned aerial vehicle, an electronic device and a storage medium to solve the problems raised in the above background technology.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for detecting small targets of unmanned aerial vehicles, the method comprising:

[0008] Acquire an image data set, and preprocess each image in the image data set, wherein the image data set consists of images taken by a drone and small target verification data of the corresponding images;

[0009] Input the preprocessed image into the network detection model, calculate the loss function between the output result of the network detection model and the actual small target verification data, and train the network detection model parameters until they are optimal;

[0010] The image to be detected is input into the optimal detection model, and the small target detection result of the drone image is output. The optimal detection model refers to the network detection model after the model parameters are adjusted to the optimal value after training.

[0011] Preferably, the network detection model includes: a backbone network, an intermediate network and a detection head network;

[0012] The backbone network extracts features from the preprocessed image;

[0013] The intermediate network uses the PAFPN structure to collect image features of different scales and perform multi-scale feature fusion;

[0014] The detection head network converts the fused feature map into specific information including category, position and confidence required for target detection as the output result of the network detection model.

[0015] Preferably, the backbone network performs feature extraction and downsampling on the preprocessed image, including:

[0016] The preprocessed image is input into the CBS module, which includes a convolution layer, a normalization layer and an activation function layer; the convolution layer uses a convolution operation to extract the local spatial information of the image and downsamples the image feature map; the normalization layer is used to normalize the eigenvalue distribution of the image to accelerate the convergence of the network; the activation function layer introduces nonlinear transformation capabilities to convert and extract image features;

[0017] The image features output by the activation function layer are input into the feature network, which includes the first feature sub-network, the second feature sub-network and the third feature sub-network; the data transmission mode is the first feature sub-network → the second feature sub-network → the third feature sub-network; the first feature sub-network and the second feature sub-network are composed of a CBS module and a C2f module; the third feature sub-network is composed of an SCDown module and a C2f module; the C2f module adopts a Bottleneck structure for fusing image features of different scales; the SCDown module is used for decoupling space and channels;

[0018] The feature data output by the third feature sub-network is input into the JCASA module, which includes the CBAM module and the SA module; the CBAM module uses channel attention and spatial attention to filter out important feature data; the SA module uses self-attention calculation to obtain multi-granularity feature information;

[0019] The feature data output by the SA module is input into the spatial pyramid module, which uses spatial pyramid pooling to efficiently capture multi-scale feature information.

[0020] Preferably, the CBAM module uses channel attention and spatial attention to filter out important feature data, including:

[0021] S1. Channel attention calculation:

[0022] Set the input feature to F ch-in , then according to F avg =AvgPool(F ch-in ) and F max =MaxPool(F ch-in), we get two different spatial context descriptors; among them, F avg 、F max Both represent spatial context descriptors, and AvgPool represents the input feature F ch-in Using the average pooling operation, MaxPool represents the input feature F ch-in Use the maximum pooling operation to aggregate the spatial information of the feature map;

[0023] The two spatial context descriptors are forwarded to the shared network to generate a channel attention map, and two feature data are obtained, where the shared network consists of two convolutional layers plus an activation function:

[0024] F avg-conv =Conv2(Act(Conv1(F avg ))), F max-conv =Conv2(Act(Conv1(F max )));

[0025] Among them, Conv1 and Conv2 both represent 1*1 convolution, and a full connection is made in the direction of the channel. Act represents the Relu activation function, which introduces nonlinear characteristics.

[0026] Get two feature data, and get the weight coefficient M of the two feature data according to the sigmoid activation function: M = σ(F avg-conv +F max-conv ), where σ represents the sigmoid function;

[0027] According to the input feature F ch-in And the weight coefficient M to get the scaled new feature data: F ch-out =M×F ch-in ;

[0028] S2. Spatial attention calculation:

[0029] For the new feature data F ch-out Performing maximum pooling and average pooling operations along the channel dimension respectively obtains two context scale features: F max =MaxPool(F ch-out ), F avg =AvgPool(F ch-out );

[0030] The spatial attention weight M is obtained based on the two context scale features: M = Conv(Cat(F max ,F avg )); Cat represents the concatenation function that concatenates the features after maximum pooling and average pooling along the channel dimension, and Conv represents the convolution operation of size 7*7;

[0031] Apply the sigmoid activation function to the generated spatial attention weights M to limit the weights between 0 and 1: F sp-out =σ(M)×F ch-out ; Confirm output feature F sp_out .

[0032] Preferably, the SA module uses self-attention calculation to obtain multi-granularity feature information, including:

[0033] Self-attention calculation:

[0034] Get the two-dimensional input feature X = F sp_out , where the two-dimensional input feature X∈R H×W×C , H and W represent the width and height of the two-dimensional features of the image, and C represents the number of channels; then X is divided into S×S non-overlapping regions, so that each region contains feature vectors and reshape X into

[0035] According to Q = X r W q , K = X r W k , V = X r W v , where W q , W k , W v ∈R C×C are the projection weights of query, key, and value respectively;

[0036] First, export the region-level query and key projection weights, and then use Q r With the transposed K r The region-to-region affinity graph matrix is ​​derived by matrix multiplication between A r =Q r (K r ) T ; where the affinity graph matrix A r The entries in measure the degree to which two regions are semantically related;

[0037] Using the row-by-row topk operator to derive the index matrix And applies fine-grained token-to-token attention: for each query token in region i, it will pay attention to For all key-value pairs in the union of k routing areas indexed by K, then g =gather(K,I r ), V g =gather(V,I r ) Confirm the key-value tensor

[0038] Get Q, K after routing processing g , V g Then, Q, K g , V g The result is divided into four parts. Three of the parts are convolved using depthwise convolutions of different sizes as aggregators Agg(·). The convolution kernel sizes are set to 3, 5, and 7 respectively. The other part is not processed. Finally, the four parts are concatenated to obtain new Q, K g , V g ;

[0039] According to Q, K g , V g Confirm the self-attention calculation of the two-dimensional input feature X:

[0040] O=Attention(Q,K g ,V g )+LCE(V); where LCE(V) represents the local context enhancement term.

[0041] Preferably, the loss function includes a binary cross entropy loss function, a distribution focus loss function and a complete intersection-over-union loss function. These three loss functions work together to guide the model to learn the target detection task.

[0042] Preferably, the loss function between the output result of the network detection model and the actual small target verification data is calculated, and the parameters of the network detection model are trained to be optimal, including:

[0043] The training images and small target verification images of the preprocessed drone images are obtained and input into the network detection model. After the output results are obtained, the error between the model prediction results and the true labels is calculated according to the loss function. The obtained loss is fed back to the network, and the network model weights are updated so that the model parameters are gradually optimized during the training process to improve the prediction accuracy of the model.

[0044] An electronic device, comprising:

[0045] at least one processor;

[0046] and, a memory communicatively coupled to the at least one processor;

[0047] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned detection method.

[0048] A computer-readable storage medium stores a computer program, and the computer program implements the above-mentioned detection method when executed by a processor.

[0049] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0050] The present invention adds an innovative structure JCASA module to the backbone network. The module is mainly composed of CBAM and SA in series, which ensures accuracy while avoiding excessive overhead caused by the secondary computational complexity of self-attention; in the SA module, the Transformer self-attention calculates Q, K, and V with a unique Group-Mix with Routing mechanism to reduce the number of parameters and increase local feature map information of different sizes; by further increasing the upsampling stage of the Neck network to obtain large-scale feature maps, and adding the corresponding small target detection head to enhance the detection capability of the algorithm for small-size targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0052] Figure 1 It is a flow chart of a method for detecting small targets of unmanned aerial vehicles according to the present invention;

[0053] Figure 2 It is a structural diagram of a network detection model in a method for detecting small targets of unmanned aerial vehicles of the present invention;

[0054] Figure 3 This is a backbone network structure diagram of a UAV small target detection method of the present invention;

[0055] Figure 4 It is a schematic diagram of the detection effect in an embodiment of a method for detecting small targets of unmanned aerial vehicles of the present invention;

[0056] Figure 5 The figure is a comparative experimental result of different detection algorithms in an embodiment of a method for detecting small targets of unmanned aerial vehicles of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] See also Figure 1-Figure 5 The present invention provides a technical solution: a method for detecting small targets of unmanned aerial vehicles, the method comprising:

[0059] Acquire an image data set, and preprocess each image in the image data set, wherein the image data set consists of images taken by a drone and small target verification data of the corresponding images;

[0060] Preprocessing includes operations such as rotating, scaling, reversing, and random cropping of images to enhance image data;

[0061] The present invention adopts the image-rich VisDrone2019 dataset (drone-shot image dataset). The target detection dataset of VisDrone2019 consists of images and corresponding annotation files, including 6471 training set images, 548 validation set images, and 1610 test set images. In the dataset, the image sizes range from 2000×1500 to 480×360. Since the images are taken from the perspective of a drone, they are greatly different from images taken by ground personnel such as MS-COCO and VOC2012 in terms of shooting angle, image content, background, and ambient illumination. The scenes composed of the dataset images are very wide, including streets, squares, parks, schools, residential communities, etc.

[0062] The lighting conditions of the images include well-lit daytime, poorly lit nighttime, cloudy, strong light, and glare conditions; the object types annotated in the images include 10 types: pedestrian, person, bicycle, car, van, truck, tricycle, awning tricycle, bus, and motor.

[0063] Input the preprocessed image into the network detection model, calculate the loss function between the output result of the network detection model and the actual small target verification data, and train the network detection model parameters until they are optimal;

[0064] The image to be detected is input into the optimal detection model, and the small target detection result of the drone image is output. The optimal detection model refers to the network detection model after the model parameters are adjusted to the optimal value after training.

[0065] Preferably, the network detection model includes: a backbone network, an intermediate network and a detection head network;

[0066] The backbone network extracts features from the preprocessed image;

[0067] The intermediate network uses the PAFPN structure to collect image features of different scales and perform multi-scale feature fusion;

[0068] The intermediate network adopts the PAFPN structure, which consists of top-down and bottom-up branches, and can achieve effective multi-scale feature fusion;

[0069] For example, different layers of the top-down branch receive feature maps P5 (20*20*1024), P4 (40*40*512), P3 (80*80*256), and P2 (160*160*128) from the SPPF layer in the spatial pyramid module of the backbone network and the C2f layer in the feature network through lateral connections;

[0070] The top-down branch consists of three parts, each layer consists of upsampling, splicing and C2f modules. Compared with the original Neck network of yolov10, the present invention adds an upsampling stage to obtain large-scale scales and enhance the algorithm's detection ability for small targets;

[0071] In each layer, upsampling is to increase the image size by two times through deconvolution operation, and then the enlarged image features are fused with the features in the backbone network obtained through lateral connection through the Concat module; for example, P5 (20*20*1024) becomes 40*40*512 after upsampling, denoted as N1, and input into the splicing module, which obtains the output P4 (40*40*512) of the third feature sub-network in the backbone network through lateral connection and fuses it with N1, that is, adding element by element to obtain a new feature map; the new feature map then enters the C2f module for feature fusion of different scales, and then input into the next stage; the final output of the Neck network is denoted as H1, and its size is 160*160*128 after three upsamplings.

[0072] The detection head network converts the fused feature map into specific information including category, position and confidence required for target detection as the output result of the network detection model;

[0073] The main function of the detection head network is to convert the feature map into specific information required for target detection, including category, location, and confidence. The detection head network includes several convolutional layers and activation functions. These convolutional layers are used to further process the feature maps output by the intermediate network to extract more advanced features.

[0074] For example, there are four detection heads, which receive H1 (160*160*128), H2 (80*80*256), H3 (40*40*512), and H4 (20*20*1024) as input respectively to detect targets of different sizes. Among them, the detection head that receives large-scale feature maps such as H1 is specially used to detect small targets, which improves the detection ability of the entire model for small targets.

[0075] Preferably, the backbone network performs feature extraction and downsampling on the preprocessed image, including:

[0076] The preprocessed image is input into the CBS module, which includes a convolution layer, a normalization layer and an activation function layer; the convolution layer uses a convolution operation to extract the local spatial information of the image and downsamples the image feature map; the normalization layer is used to normalize the eigenvalue distribution of the image to accelerate the convergence of the network; the activation function layer introduces nonlinear transformation capabilities to convert and extract image features;

[0077] Among them, the convolution kernel size of the convolution layer is 3*3, the step size is 2, and the number of channels is 64. In this way, when the input image size is 640*640*3, the output size after convolution is 320*320*64. Then the output passes through the BN layer to normalize the eigenvalue distribution and speed up the convergence of the network;

[0078] The image features output by the activation function layer are input into the feature network, which includes the first feature sub-network, the second feature sub-network and the third feature sub-network; the data transmission mode is the first feature sub-network → the second feature sub-network → the third feature sub-network; the first feature sub-network and the second feature sub-network are composed of a CBS module and a C2f module; the third feature sub-network is composed of an SCDown module and a C2f module; the C2f module adopts a Bottleneck structure for fusing image features of different scales; the SCDown module is used for decoupling space and channels;

[0079] The feature data output by the third feature sub-network is input into the JCASA module, which includes the CBAM module and the SA module; the CBAM module uses channel attention and spatial attention to filter out important feature data; the SA module uses self-attention calculation to obtain multi-granularity feature information;

[0080] Construct a module JCASA based on the combination of CNN and Transformer self-attention. The JCASA module is placed only after the third feature sub-network with lower resolution, which ensures accuracy while avoiding excessive overhead caused by the secondary computational complexity of self-attention.

[0081] The feature data output by the SA module is input into the spatial pyramid module, which uses spatial pyramid pooling to efficiently capture multi-scale feature information.

[0082] Preferably, the CBAM module uses channel attention and spatial attention to filter out important feature data, including:

[0083] S1. Channel attention calculation:

[0084] Set the input feature to F ch-in , then according to F avg =AvgPool(F ch-in ) and Fmax =MaxPool(F ch-in ), we get two different spatial context descriptors; among them, F avg 、F max Both represent spatial context descriptors, and AvgPool represents the input feature F ch-in Using the average pooling operation, MaxPool represents the input feature F ch-in Use the maximum pooling operation to aggregate the spatial information of the feature map;

[0085] The two spatial context descriptors are forwarded to the shared network to generate a channel attention map, and two feature data are obtained, where the shared network consists of two convolutional layers plus an activation function:

[0086] F avg-conv =Conv2(Act(Conv1(F avg ))), F max-conv =Conv2(Act(Conv1(F max )));

[0087] Among them, Conv1 and Conv2 both represent 1*1 convolution, and a full connection is made in the direction of the channel. Act represents the Relu activation function, which introduces nonlinear characteristics.

[0088] Get two feature data, and get the weight coefficient M of the two feature data according to the sigmoid activation function: M = σ(F avg-conv +F max-conv ), where σ represents the sigmoid function;

[0089] According to the input feature F ch-in And the weight coefficient M to get the scaled new feature data: F ch-out =M×F ch-in ;

[0090] In a neural network, the higher the dimension, the smaller the feature map size and the more channels there are. The channels represent the feature information of the entire image. With so much channel information, it is difficult for a neural network to identify and filter useful channel information. In this case, channel attention can be used to tell the network which ones are important.

[0091] At the same time, for each image, especially large-size images, since the importance of information in different areas is different, the attention required for different areas is also different; at this time, the spatial attention mechanism can be used to assign a weight to each pixel or area in the image; these weights determine the degree to which the model should pay attention to the area when processing the image; areas with high weights will receive more attention, while areas with low weights will be relatively ignored; therefore, the CBAM module composed of channel attention and spatial attention is used in the JCASA module to provide more effective features for the subsequent self-attention calculation.

[0092] S2, spatial attention calculation:

[0093] For the new feature data F ch-out Performing maximum pooling and average pooling operations along the channel dimension respectively obtains two context scale features: F max =MaxPool(F ch-out ), F avg =AvgPool(F ch-out );

[0094] The spatial attention weight M is obtained based on the two context scale features: M = Conv(Cat(F max ,F avg )); Cat represents the concatenation function that concatenates the features after maximum pooling and average pooling along the channel dimension, and Conv represents the convolution operation of size 7*7;

[0095] Apply the sigmoid activation function to the generated spatial attention weights M to limit the weights between 0 and 1: F sp-out =σ(M)×F ch-out ; Confirm output feature F sp_out .

[0096] Preferably, the SA module uses self-attention calculation to obtain multi-granularity feature information, including:

[0097] Self-attention calculation:

[0098] Get the two-dimensional input feature X = F sp_out , where the two-dimensional input feature X∈R H×W×C , H and W represent the width and height of the two-dimensional features of the image, and C represents the number of channels; then X is divided into S×S non-overlapping regions, so that each region contains feature vectors and reshape X into

[0099] According to Q = X r W q , K = X r W k, V = X r W v , where W q , W k , W v ∈R C×C are the projection weights of query, key, and value respectively;

[0100] First, export the region-level query and key projection weights, and then use Q r With the transposed K r The region-to-region affinity graph matrix is ​​derived by matrix multiplication between A r =Q r (K r ) T ; where the affinity graph matrix A r The entries in measure the degree to which two regions are semantically related;

[0101] Using the row-by-row topk operator to derive the index matrix And applies fine-grained token-to-token attention: for each query token in region i, it will pay attention to For all key-value pairs in the union of k routing areas indexed by K, then g =gather(K,I r ), V g =gather(V,I r ) Confirm the key-value tensor

[0102] Get Q, K after routing processing g , V g Then, Q, K g , V g The result is divided into four parts. Three of the parts are convolved using depthwise convolutions of different sizes as aggregators Agg(·). The convolution kernel sizes are set to 3, 5, and 7 respectively. The other part is not processed. Finally, the four parts are concatenated to obtain new Q, K g , V g ;

[0103] According to Q, K g , V g Confirm the self-attention calculation of the two-dimensional input feature X:

[0104] O=Attention(Q,K g ,V g )+LCE(V); where LCE(V) represents the local context enhancement term;

[0105] When calculating self-attention, it can be found that queries in different semantic areas actually focus on completely different key-value pairs; therefore, forcing all queries to process the same set of tokens may not be optimal; the present invention uses a routing mechanism to allow each query to focus only on the most semantically relevant key-value pairs; in addition, most Transformer model attention maps only describe the correlation between each individual tag pair at a single granularity, and multiplying the attention map with the value only linearly recombines a single tag; such a model obviously does not consider the correlation between different token groups at different granularities; in this model, the token is divided into four parts, and some individual tokens are replaced with group proxies generated by a group aggregator; then, the attention map is calculated using Query and Key, and it is used to recombine the group proxies and individual tokens in Value, so that multi-granularity feature information can be obtained, that is, different attentions are mixed together to better understand the token from a comprehensive perspective; in order to generate group proxies of different sizes, deep convolutions with different kernel sizes are used as aggregators, and deep convolutions can reduce the spatial scales of K and V. This calculation method is the Group-Mix with Routing mechanism used in the present invention.

[0106] Preferably, the loss function includes a binary cross entropy loss function, a distribution focus loss function and a complete intersection-over-union loss function. These three loss functions work together to guide the model to learn the target detection task.

[0107] Preferably, the binary cross entropy loss function is used to evaluate the difference between the predicted value and the actual label in the classification model: given a set of actual labels y, its corresponding predicted value is p(y), and BCE Loss is denoted by L BCE , calculated as follows:

[0108]

[0109] Where N is the number of samples, y i is the actual label of the i-th sample, which takes the value of 0 or 1. i ) is the predicted value of the i-th sample, and its value range is between (0, 1);

[0110] Specifically, for each sample, first use the formula y i log(p(y i ))+(1-y i )log(1-p(y i )) Calculate the loss and then negate the above result, that is, -[y i log(p(y i ))+(1-y i)log(1-p(y i ))], and finally take the average of the losses of all samples to find the output loss of the model for a set of size N;

[0111] Calculate L BCE When y=1, the true label indicates that the event occurs. At this time, L BCE =-log(p(y)), when the predicted value p(y) is close to 1, L BCE Close to 0, when the predicted value p(y) is close to 0, L BCE Approaching infinity;

[0112] When y=0, the true label indicates that the event did not occur. BCE =-log(1-p(y)), when the predicted value p(y) is close to 1, L BCE Approaching infinity; when the predicted value p(y) approaches 0, L BCE Close to 0;

[0113] Therefore, when training the model, the model parameters can be optimized by minimizing BCE Loss to make the predicted probability as close to the true label as possible.

[0114] Preferably, the regression loss is focused on the distribution difference of bounding box regression by distribution focus loss. It does not directly predict the bounding box, but predicts the probability distribution of the bounding box. It aims to solve challenging objects with blurred boundaries or partial occlusions. In traditional object detection, especially in complex scenes, the definition of the true bounding box of the target object cannot be accurately given. Considering that the true distribution is usually not too far from the labeled position, DFL chooses to optimize the probabilities of the two positions near the label, so that the network distribution is focused near the label value.

[0115] The loss function is denoted as L DFL , is defined as follows. In layman's terms, during the training process, the bounding box of the target should not be a certain value, but a distribution. For example, for the object of waves, its boundary is not clear. Through such a loss function, the overfitting phenomenon of the network during training can be reduced:

[0116]

[0117] Among them, y is the actual value of the label, y i ,y i+1 are two predicted values ​​near the label y, S i , S i+1 y i ,y i+1Corresponding probability; for example, the center point of a target is 73px away from its upper boundary. In the feature map with the maximum downsampling of 32 times, its mapping value y in DFL is calculated to be 73 / 32=2.3. After rounding, the label value y is obtained i =2, adjacent label value y i+1 =3.

[0118] Preferably, the complete intersection-over-union loss function is applied to measure the difference between the predicted bounding box and the actual bounding box; it is a comprehensive indicator that considers three properties between bounding boxes: overlap ratio, distance between center points, and aspect ratio;

[0119] The loss function is denoted as L CIOU , the calculation process is as follows:

[0120]

[0121] L CIOU = 1-CIoU;

[0122] The three terms of CIOU in the formula correspond exactly to IOU, center point distance, and aspect ratio calculation. α and υ are the aspect ratios, parameter c is the diagonal of the bounding box, ρ is the Euclidean distance, b, w, h, and b gt 、w gt 、h gt Represent the center, width, and height of the predicted box and the true box respectively.

[0123] Preferably, the loss function between the output result of the network detection model and the actual small target verification data is calculated, and the parameters of the network detection model are trained to be optimal, including:

[0124] Obtain the training images and small target verification images of the preprocessed drone images, input them into the network detection model, and after obtaining the output results, calculate the error between the model prediction results and the true labels according to the loss function, feed the obtained loss back to the network, update the network model weights, and gradually optimize the model parameters during the training process to improve the prediction accuracy of the model;

[0125] For example, after preprocessing such as scaling and random cropping, the images in the dataset visdrone2019 are placed in the network detection model, and the loss function value of the image is calculated according to the loss function. The resulting loss is fed back to the network detection model to update the weights of the network detection model.

[0126] Preferably, the network is trained under the Pytorch2.0.1 framework. The initial learning rate parameter determines the step size of the model weight adjustment, which helps to converge quickly in the early stage of training and is set to 0.01. Stochastic gradient descent (SGD) with momentum is used as the optimizer. Momentum is a technique to accelerate the gradient descent process and is used to increase the stability of parameter updates. The momentum of the optimizer is set to 0.937, and the weight decay coefficient is 0.0005. Weight decay is a regularization technique used to reduce model complexity and prevent overfitting. The entire network is trained for 200 rounds to fully learn the data.

[0127] In order to further verify the potential advantages of the algorithm of the present invention, a variety of target detection algorithms were selected for comparative experiments, mainly including the commonly used frameworks and the latest improvements in the field of small target detection; Figure 5 As shown in the figure, by comparing and analyzing the results of mAP values ​​of each category after training the VisDrone2019 dataset, it can be seen that compared with other algorithm models, the algorithm model of the present invention has the best comprehensive performance; on the one hand, it has the best detection accuracy in target categories such as pedestrians and cars, and on the other hand, it has improved the average detection accuracy by 1.9% compared with the latest YOLOv11 model. Compared with the RT-DETR detection algorithm that relies more on the Transformer architecture, the algorithm model of the present invention still maintains its advantage in detection accuracy; the algorithm model of the present invention adopts a unique Group-Mix with Routing mechanism to calculate self-attention, and adds feature extraction and detection heads for small targets in the intermediate network and detection head network, so that the entire model can more fully mine the texture information of small targets and enhance the model's positioning and detection capabilities for small targets.

[0128] An electronic device, comprising:

[0129] at least one processor;

[0130] and, a memory communicatively coupled to the at least one processor;

[0131] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above-mentioned detection method.

[0132] A computer-readable storage medium stores a computer program, and the computer program implements the above-mentioned detection method when executed by a processor.

[0133] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting small targets of unmanned aerial vehicles, characterized by: The method comprises: Acquire an image data set, and preprocess each image in the image data set, wherein the image data set consists of images taken by a drone and small target verification data of the corresponding images; Input the preprocessed image into the network detection model, calculate the loss function between the output result of the network detection model and the actual small target verification data, and train the parameters of the network detection model until they are optimal; The image to be detected is input into the optimal detection model, and the small target detection result of the drone image is output. The optimal detection model refers to the network detection model after the model parameters are adjusted to the optimal after training.

2. The method for detecting small targets of unmanned aerial vehicles according to claim 1, characterized in that: The network detection model includes: a backbone network, an intermediate network and a detection head network; The backbone network performs feature extraction on the preprocessed image; The intermediate network uses a PAFPN structure to collect image features of different scales and perform multi-scale feature fusion; The detection head network converts the fused feature map into specific information including category, position and confidence required for target detection as the output result of the network detection model.

3. The method for detecting small targets of unmanned aerial vehicles according to claim 2, characterized in that: The backbone network performs feature extraction and downsampling on the preprocessed image, including: The preprocessed image is input into the CBS module, which includes a convolution layer, a normalization layer and an activation function layer; the convolution layer uses a convolution operation to extract the local spatial information of the image and downsamples the image feature map; the normalization layer is used to normalize the eigenvalue distribution of the image to accelerate the convergence of the network; the activation function layer introduces nonlinear transformation capabilities to convert and extract image features; The image features output by the activation function layer are input into the feature network, and the feature network includes a first feature subnetwork, a second feature subnetwork and a third feature subnetwork; the data transmission mode is the first feature subnetwork→the second feature subnetwork→the third feature subnetwork; the first feature subnetwork and the second feature subnetwork are composed of a CBS module and a C2f module; the third feature subnetwork is composed of an SCDown module and a C2f module; the C2f module adopts a Bottleneck structure for fusing image features of different scales; the SCDown module is used for decoupling space and channels; The feature data output by the third feature sub-network is input into the JCASA module, wherein the JCASA module includes a CBAM module and an SA module; the CBAM module uses channel attention and spatial attention to filter out important feature data; and the SA module uses self-attention calculation to obtain multi-granularity feature information; The feature data output by the SA module is input into a spatial pyramid module, and the spatial pyramid module uses spatial pyramid pooling to efficiently capture multi-scale feature information.

4. The method for detecting small targets of unmanned aerial vehicles according to claim 3, characterized in that: The CBAM module uses channel attention and spatial attention to filter out important feature data, including: S1. Channel attention calculation: Set the input feature to F ch-in , then according to F avg =AvgPool(F ch-in ) and F max =MaxPool(F ch-in ), we get two different spatial context descriptors; among them, F avg 、F max Both represent spatial context descriptors, and AvgPool represents the input feature F ch-in Using the average pooling operation, MaxPool represents the input feature F ch-in Use the maximum pooling operation to aggregate the spatial information of the feature map; The two spatial context descriptors are forwarded to a shared network to generate a channel attention map, and two feature data are obtained, where the shared network consists of two convolutional layers plus an activation function: F avg-conv =Conv2(Act(Conv1(F avg ))),F max-conv =Conv2(Act(Conv1(F max ))); Among them, Conv1 and Conv2 both represent 1*1 convolution, and a full connection is made in the direction of the channel. Act represents the Relu activation function, which introduces nonlinear characteristics. Obtain the two feature data, and obtain the weight coefficient M of the two feature data according to the sigmoid activation function: M = σ(F avg-conv +F max-conv ), where σ represents the sigmoid function; According to the input feature F ch-in And the weight coefficient M to get the scaled new feature data: F ch-out =M×F ch-in ; S2, spatial attention calculation: For the new feature data F ch-out Performing maximum pooling and average pooling operations along the channel dimension respectively obtains two context scale features: F max =MaxPool(F ch-out ), F avg =AvgPool(F ch-out ); The spatial attention weight M is obtained according to the two context scale features: M = Conv(Cat(F max ,F avg )); Cat represents the concatenation function that concatenates the features after maximum pooling and average pooling along the channel dimension, and Conv represents the convolution operation of size 7*7; Apply the sigmoid activation function to the generated spatial attention weights M to limit the weights between 0 and 1: F sp-out =σ(M)×F ch-out ; Confirm output feature F sp_out .

5. The method for detecting small targets of unmanned aerial vehicles according to claim 3, characterized in that: The SA module uses self-attention calculation to obtain multi-granularity feature information, including: Self-attention calculation: Get the two-dimensional input feature X = F sp_out , where the two-dimensional input feature X∈R H×W×C , H and W represent the width and height of the two-dimensional features of the image, and C represents the number of channels; then X is divided into S×S non-overlapping regions, so that each region contains feature vectors and reshape X into According to Q = X r W q , K = X r W k , V = X r W v , where W q , W k , W v ∈R C×C are the projection weights of query, key, and value respectively; First, export the region-level query and key projection weights, and then use Q r With the transposed K r The region-to-region affinity graph matrix is ​​derived by matrix multiplication between A r =Q r (K r ) T ; where the affinity graph matrix A r The entries in measure the degree to which two regions are semantically related; Using the row-by-row topk operator to derive the index matrix And applies fine-grained token-to-token attention: for each query token in region i, it will pay attention to For all key-value pairs in the union of k routing areas indexed by K, then g =gather(K,I r ), V g =gather(V,I r ) Confirm the key-value tensor Get Q, K after routing processing g , V g Then, Q, K g , V g The result is divided into four parts. Three of the parts are convolved using depthwise convolutions of different sizes as aggregators Agg(·). The convolution kernel sizes are set to 3, 5, and 7 respectively. The other part is not processed. Finally, the four parts are concatenated to obtain new Q, K g , V g ; According to Q, K g , V g Confirm the self-attention calculation of the two-dimensional input feature X: O=Attention(Q,K g ,V g )+LCE(V); where LCE(V) represents the local context enhancement term.

6. The method for detecting small targets of unmanned aerial vehicles according to claim 1, characterized in that: The loss function includes a binary cross entropy loss function, a distribution focus loss function and a complete intersection-over-union loss function. These three loss functions work together to guide the model to learn the target detection task.

7. The method for detecting small targets of unmanned aerial vehicles according to claim 1, characterized in that: The calculating the loss function between the output result of the network detection model and the actual small target verification data, and training the parameters of the network detection model until they are optimal, comprises: The training images and small target verification images of the preprocessed drone images are obtained and input into the network detection model. After the output results are obtained, the error between the model prediction results and the true labels is calculated according to the loss function. The obtained loss is fed back to the network, and the network model weights are updated so that the model parameters are gradually optimized during the training process to improve the prediction accuracy of the model.

8. An electronic device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the detection method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the detection method according to any one of claims 1 to 7 is implemented.

Images