An unmanned aerial vehicle image target detection method based on large kernel equivalent convolution attention mechanism

Abstract

The application provides a UAV image target detection method based on a large kernel equivalent convolution attention mechanism, which comprises the following steps: (1) first, pre-processing the training data; (2) constructing the entire target detection network; (3) after the network is constructed, the network is trained by using the processed data, and a network weight file of training convergence is obtained; (4) using the trained detector to infer a given image, obtaining a detection result, and outputting a detected object category, object position and confidence. Compared with a traditional target detection algorithm, the application improves the UAV image target detection accuracy by applying the attention mechanism, and more targets can be detected.

Description

Technical Field

[0001] This invention relates to the field of target detection in computer vision, specifically to a method for detecting targets in drone images and marking targets in the images. Background Technology

[0002] In recent years, with the gradual maturation of drone technology and the decrease in price, drones have been increasingly widely used in both military and civilian fields, playing a vital role in military reconnaissance, military strikes, agricultural inspection, and marine exploration. In military applications, drones offer advantages such as no personnel loss, low cost and high efficiency, and flexible and diversified configurations. In agricultural applications, the modularity of drone platforms is mainly utilized to assist agricultural workers in various agricultural activities. In 1987, Yamaha Corporation, commissioned by the Japanese Ministry of Agriculture, produced the world's first agricultural drone, the "R50." Currently, Japan has 2,346 registered agricultural drones, making it the world's largest user of agricultural drone spraying. my country began research on low-altitude, low-volume spraying technology using single-rotor drones in 2008. In January 2013, the Agricultural Aviation Industry Technology Innovation Alliance was established in Sanya, Hainan, creating a technology exchange platform encompassing universities, research institutions, and enterprises. Its main focus is on research into related technologies and products to promote the in-depth application of agricultural drones. For the transformation and upgrading of traditional agriculture, drones have an unparalleled and significant role to play. On the one hand, drones can replace manual labor in all aspects of agricultural production, including sowing, spraying, pest control, and monitoring, overcoming the influence of terrain and weather on agricultural production and alleviating the growing shortage of agricultural labor. On the other hand, the application of drones in agriculture can also improve agricultural production efficiency and quality, and ensure agricultural safety. According to relevant data, as of 2018, the market size of agricultural plant protection drones in my country had exceeded 2.5 billion yuan, accounting for nearly 41% of the industrial drone market. In 2019, the market size further expanded, doubling to approximately 5 billion yuan. This demonstrates the enormous application value of drones, both in daily life and for military purposes.

[0003] Among the various applications of drones, visual object detection, as a visual task that identifies specific targets in a given image, plays a crucial role in computer vision, capable of accomplishing a variety of real-world tasks. Furthermore, visual object detection is a major research hotspot in visual tasks and a foundational task for many downstream tasks, its importance being self-evident. Early object detection algorithms were mostly based on hand-designed features such as Haar handcrafted features. With the advent of Deep Neural Networks (DNNs), due to their powerful learning and fitting capabilities, deep learning has been widely applied in object detection in recent years, and now the vast majority of object detection algorithms utilize deep neural network algorithms. However, due to the inherent characteristics of drones, images captured by drones differ significantly from ordinary images:

[0004] 1) Because drones are positioned at different altitudes than ordinary cameras and surveillance cameras, they have different perspectives and a very wide field of view, making the backgrounds of images captured by drones more complex.

[0005] 2) Due to the altitude at which drones take pictures, there are mostly small targets. This also leads to more severe problems with small target clustering and occlusion compared to traditional images;

[0006] 3) Because the altitude of the drone is variable, the scale of the target changes more drastically compared to traditional images, especially since most targets are small.

[0007] Due to these factors, directly applying object detectors trained on traditional image datasets, such as MS COCO, to detect drone-captured images yields significantly worse results than on traditional images. Therefore, to meet current industry needs and achieve better detection performance on drone-captured images, it is necessary to make appropriate improvements to existing object detectors to adapt to the characteristics of drone images and better detect targets captured in drone images. Summary of the Invention

[0008] This invention aims to solve the aforementioned problems. Based on the existing object detection algorithm Faster R-CNN, this invention proposes an object detection method based on an improved large-kernel equivalent convolutional attention mechanism (FASTer R-CNN). This invention can fully utilize images captured by UAVs. By introducing a hybrid attention mechanism object detection algorithm, it fully leverages global information in the image and contextual information surrounding the target to locate the target in a given image.

[0009] The objective of this invention can be achieved through the following technical solutions:

[0010] (1) First, the training data is preprocessed. Specifically, the annotation files of the input data are transformed. The annotation files of the corresponding images are read and converted into the specified format of the MS COCO dataset. Then, data augmentation operations are performed to increase the amount of training data so that the network training is more thorough. The specific data augmentation techniques used include random rotation, translation and scaling, random brightness and contrast transformation, RGB color shift and saturation transformation, image compression, channel transformation and image blurring.

[0011] (2) Construct the entire target detection network. The target detection network is based on the Faster RCNN network. The overall network architecture is divided into four parts: backbone network, neck network, region extraction network and detection head.

[0012] (3) After the network is built, the network is trained using the processed data to obtain the network weight file that has been trained and converged. The training plan is 12 rounds. The learning rate is reduced to one-tenth of the original in the 9th and 11th rounds respectively. The base learning rate is 0.02. The optimization algorithm uses stochastic gradient descent (SGD) with momentum, which can avoid getting trapped in local optima to a certain extent. Among the parameters used, the momentum is 0.9 and the momentum decay regularization term is 0.0001. During training, the deformation size of the image is 640 pixels wide and 512 pixels high. At the same time, multi-scale training is used, and the image is randomly enlarged to 960×768 pixels to enhance the robustness of the model.

[0013] (4) Use the trained detector to reason about the given image and obtain the detection result.

[0014] The backbone network, neck network, region extraction network, and detection head in step (2) specifically include:

[0015] The backbone network uses the ResNet-50 model, which is divided into stages 0 to 4. Stage 0 is the preprocessing stage for the input data, and stages 1 to 4 are the feature extraction stages. The feature maps generated in stages 1 to 4 are used as the output features of this part of the entire network.

[0016] The neck network adopts a multi-scale detection structure based on FPN. Attention features at the corresponding scale are obtained by inserting an Attention Enhance Module (AEM) between the features output by the backbone network and the input features of FPN. In the feature fusion stage of FPN, a Feature Fusion Module (FFM) is used to supervise the features of the corresponding scale by using the attention features at the corresponding scale obtained in AEM and element-wise multiplication. Feature fusion is performed by interpolation amplification and element-wise addition.

[0017] The features at each scale output from the neck of the network are input into the Region Proposal Network (RPN) to obtain the initial detection regions. The RoI Align algorithm is used to map the proposed detection regions back to the input feature map. The obtained detection regions are then input into the detection head for final detection region filtering.

[0018] The detection head consists of a common convolutional layer and detection and classification branches. After the input features are processed by the common convolutional layer, the position of the target detection box is regressed through the detection branch, and the target category and confidence level are predicted by the classification branch.

[0019] The network training in step (3) specifically includes the following steps:

[0020] Training the network requires calculating the loss between the network's predictions and labeled ground truth samples, and minimizing this loss; the loss function used is the L1 loss function:

[0021]

[0022] The smooth L1 loss function is defined as follows:

[0023]

[0024] The loss function used in the classification branch is the cross-entropy loss function, which is commonly used in classification loss:

[0025]

[0026] The final overall loss function of the algorithm is the sum of the losses of all the parts to be trained, which is the sum of the classification loss and the localization loss:

[0027]

[0028] Where i is an anchor point in the current mini-batch, and L cls For classification loss, L loc The localization loss is p, where λ is the regularization parameter. i The classification probability predicted by the model. This represents the classification probability of the ground truth label; it is 1 if the anchor point is a positive sample and 0 if it is a negative sample. i The target location predicted by the parameterized model. For parameterized target localization in the label; N cls With N regThese are the number of classification samples and regression samples in the current mini-batch, respectively, used for regularization; the network is trained by using the back propagation (BP) algorithm of the neural network and an optimizer to minimize the overall loss of the network.

[0029] The image reasoning in step (4) specifically includes the following steps:

[0030] (41) Load the image into memory and perform transformation operations on the image. The specific operations are multi-scale scaling and flipping, with scales of 1280×1024 and 1920×1536.

[0031] (42) Then perform the following operations in sequence: maintain proportion deformation, random flip, regularization, and fill, to convert the image into a tensor;

[0032] (43) After performing the above operations on the input image, the obtained image data is input into the entire network for forward inference to obtain the inference results of the target category, confidence and detection box in the image; based on the obtained inference results, the detection results are visualized on the input image, and finally the image containing the detection results is saved to the specified location as the output result.

[0033] Compared with existing technologies, this invention has the following advantages: It makes appropriate improvements to existing target detectors by applying an attention mechanism to adapt to UAV images. Compared to traditional target detection algorithms, this invention achieves higher target detection accuracy in UAV images and can detect more targets. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0035] Figure 2 This is a network structure diagram of the ResNet-50 backbone network used in this invention;

[0036] Figure 3 This is a schematic diagram of the FPN structure;

[0037] Figure 4 This is a schematic diagram of the structure of a large kernel equivalent convolution.

[0038] Figure 5 This is a schematic diagram of the attention mechanism (LKA) based on large kernel equivalent convolution;

[0039] Figure 6 A schematic diagram of AEM;

[0040] Figure 7 This is a schematic diagram of FFM;

[0041] Figure 8 This is an overall schematic diagram of the multi-scale detection structure used in this invention; Detailed Implementation

[0042] The overall flowchart of this invention is attached. Figure 1 As shown, the process is divided into four steps: data preprocessing, network construction, network training, and image reasoning. The specific implementation methods of this invention will be further described below based on the invention's description.

[0043] 1. Data selection and preprocessing

[0044] This invention uses two datasets for training:

[0045] The TinyPerson dataset, proposed by a team from the University of Chinese Academy of Sciences, contains 1610 images, with 717 images in the training set and 893 images in the test set, totaling 72,561 labeled targets. The team extracted images from drone-captured videos, taking one image every 50 frames and removing nearly duplicate frames while ignoring areas with excessively dense targets. The dataset used in this paper is an augmented and annotated dataset by Hong et al., employing a segmented annotation method. Each slice is 640 pixels wide and 512 pixels high, with a 30-pixel overlap between any two slices, resulting in 111,415 labeled targets—50% more than other datasets. This increased training data ensures more thorough model training and better reflects the performance differences between various algorithms. A key feature of this dataset is its smaller target scale compared to other drone datasets.

[0046] The VisDrone dataset, proposed by a team from Tianjin University, is a drone image dataset containing 400 videos and 10,209 images. It provides annotations for three tracks: object detection, multi-object tracking, and crowd counting. The data was captured by various drone cameras, resulting in a rich variety of scenes, including location (captured in 14 different cities across thousands of kilometers in China), environment (cities and countries), objects (pedestrians, vehicles, bicycles, etc.), and density (sparse and crowded scenes). It also includes various weather and lighting conditions. Over 2.6 million manually annotated object bounding boxes or frequently used object points of interest, such as pedestrians, cars, bicycles, and tricycles, are included. To better utilize the data, important attributes such as scene visibility, object class, and occlusion are also provided. A key characteristic of this dataset is the dramatic variation in object scale.

[0047] Data augmentation is a common data processing method in deep learning that increases the diversity of training data, thereby improving the robustness of the trained model. The data augmentation techniques used are as follows:

[0048] 1) Random rotation, translation, and scaling, with a transformation limit of 0.0625, a scale limit of 0, a rotation angle limit of 45 degrees, and a probability of 0.5.

[0049] 2) Random brightness and contrast transformation, with brightness transformation limited to the range of [-0.1, 0.3] and contrast transformation limited to the range of [0.1, 0.3].

[0050] 3) Randomly apply one of the RGB chromaticity shift and saturation transformations. The probability is 0.1, the RGB chromaticity shift range is 10 for all three types, the hue change range is 20 for the saturation transformation, the saturation change range is 30 for the hue transformation, and the brightness change range is 20 for the brightness transformation.

[0051] 4) Image compression: The compression ratio is as low as 0.85 and as high as 0.95, with a probability of 0.1.

[0052] 5) Image channel transformation, with a probability of 0.1.

[0053] 6) Randomly apply one of three image blurring methods: box blur, median blur, and motion blur, with a kernel size of 3 and a probability of 1 for each.

[0054] 2. Model Building

[0055] This invention is a deep learning-based target detection algorithm for drone images. Based on the classic target detection algorithm Faster R-CNN, this invention incorporates a hybrid attention mechanism. By utilizing the generated hybrid attention features, it simultaneously supervises both the channel and spatial dimensions of the detection process. Through weighted multiplication, it highlights the more important features for drone target detection, thereby improving the accuracy of target detection in drone images. This makes the detection performance of this invention superior to other target detectors. The structure of this target detector will be described in detail below:

[0056] Deep learning-based object detectors all possess a backbone network, whose function is to initially extract features from the input image. Due to historical development, the backbone network for object detection generally uses the same network used for image classification. Unlike image classification tasks, object detection tasks remove the final fully connected layer and pooling layer when applying classification neural networks; these two layers are used to generate classification predictions for the input image. This invention follows this approach, and the backbone network structure of the constructed model is shown in the attached figure. Figure 2As shown. To facilitate comparison with other algorithms, the backbone network used in this invention is ResNet-50, and as mentioned earlier, the pooling layer and fully connected layer used to generate the classification results have been removed. The entire network consists of 5 stages. Stage 0 is for image preprocessing and has a relatively simple structure; the basic unit of the remaining stages is the bottleneck module, abbreviated as BTNK. A brief explanation of the module abbreviations in the network is provided below:

[0057] 1) CONV: Convolutional layer. The parameters that follow are kernel size, number of kernels, and stride.

[0058] 2) BN: Batch Regularization Layer.

[0059] 3) RELU: RELU activation function layer.

[0060] 4) MAXPOOL: Max pooling layer, the following parameters are kernel size and stride, respectively.

[0061] 5) BTNK: Bottleneck module. The parameters listed are, in order, the number of input feature channels, the input feature side length, the number of output feature channels, and the stride. BTNK2 only has two parameters listed because the number of output feature channels is the same as the input, and the stride is the default of 1.

[0062] Figure 2 The shapes of the feature maps in parentheses indicate the stage at which the feature map is located. For example, at the beginning of stage 0, (1, 224, 224) indicates that there is 1 channel and both the width and height are 224 pixels. When the step size S is greater than 1, the width and height of the feature map output by this module will be reduced by a factor of S. (See attached...) Figure 2 The structure in the backplane results in a 32-fold reduction in the final output feature map. In this invention, the feature maps output from stages 1, 2, 3, and 4 of the backbone network are used as the input features for the next multi-scale detection structure.

[0063] In recent years, with the emergence of Feature Pyramid Networks (FPNs), multi-scale detection has become an essential operation in object detection algorithms due to its ability to adapt to targets of different scales and the resulting significant improvement in accuracy. Therefore, adding FPN to the traditional Faster R-CNN algorithm for multi-scale detection has become a common choice. This invention also uses this method as a foundation, adding an attention mechanism to supervise the input features, thereby improving detection accuracy.

[0064] The structure of the Feature Pyramid Network (FPN) is shown in the attached figure. Figure 3As shown, this structure takes feature maps from four stages of the backbone network with different spatial resolutions as input and outputs features with the same spatial resolution. As illustrated, the output features of the backbone network are first reduced in dimensionality through convolution operations with a kernel size of 1, ensuring that the number of channels in the input features of each stage is the same. Then, starting with the features output from stage 3 of the dimensionality-reduced backbone network, the features of each stage are magnified by a factor of 2 through magnification operations (commonly interpolation, deconvolution, etc.) to ensure that the feature map matches the spatial resolution of the features reduced in the previous stage. These two sets of feature maps are then fused element-wise using addition, and used as input for the next step. This process continues until the features from stage 0 are also fused. This results in four sets of feature maps with the same number of channels and progressively increasing spatial resolution. These will serve as input for the next stage's computation.

[0065] One way to enhance small target detection in UAV images is to utilize the image's contextual information, that is, to capture long-range information within the image. There are two common methods for capturing long-range information: one is using self-attention, and the other is using a larger convolutional kernel to capture information surrounding each pixel, thus enabling the perception of more of each pixel. The former's computational mechanism results in a time complexity of O(n^2) for capturing long-range information. 2 Therefore, methods that perform self-attention only in a specific region have emerged to reduce computational cost; however, the latter has not been effectively applied due to the drastically increased number of parameters and computational cost caused by large convolutional kernels. Recent studies have indicated that large kernel convolutions can also achieve their expected performance with proper training techniques. Therefore, this invention ultimately employs an equivalent replacement of large kernel convolutions to generate a hybrid attention feature map, and uses its supervised feature fusion to output the feature map to the next layer, the structure of which is shown in the attached figure. Figure 4 As shown in the figure, this module is essentially an attention module in a certain sense, obtaining the output by element-wise multiplying the original input feature map with the learned feature map. Therefore, this attention method can be extracted and applied to the network independently.

[0066] The modified attention module is shown in the attached image. Figure 5As shown, the module named Large Kernel Attention (LKA) is used to generate hybrid attention feature maps. The key feature of this module is that it uses the feature map obtained from separable large kernel convolutions as the attention feature vector. Experiments in CBAM show that using a larger convolutional kernel can utilize the contextual information around the object, as well as the relationship between the object and its surroundings. Therefore, using this module, as shown in the figure, is equivalent to using a convolutional layer with a kernel size of 21×21, which, compared to the 7×7 convolutional layer in CBAM, can capture global and local relationships more fully. Because this module is added outside the backbone network and is used to generate attention feature tensors, batch regularization and ReLU activation functions are added to correct the distribution offset error generated during convolutional learning and to remove negative values ​​obtained during learning. Finally, a 3×3 convolution operation is used, and the Sigmoid function is used to reduce the final result to the interval [0, 1]. Compared to SENet, LKA is a hybrid attention mechanism that adds spatial dimension attention, which is more useful for object detection. Compared to attention mechanisms like SENet and CBAM, LKA first avoids pooling operations at the channel or spatial levels, thus avoiding loss. Furthermore, the large convolutional layers used, compared to the former two, can fully capture global relationships between objects and local relationships around objects, resulting in better network performance. The k-th layer feature map output by this module is denoted as Ak.

[0067] The modified LKA (Large-Scale Assistive Technologies) of the Attention Enhancement Module (AEM) and Feature Fusion Module (FFM) already delivers performance improvements. However, by utilizing attention features, the feature fusion step in the Feature Fusion Network (FPN) can be supervised to improve FPN feature fusion and prevent interference between the two scales of feature fusion. Therefore, this invention introduces LKA, modifies and applies it, and finally develops AEM and FFM. The structure of AEM is shown in the attached figure. Figure 6As shown in the figure. Experiments demonstrate that the attention Ai generated by Large Kernel Convolutional Attention (LKA) cannot well represent the importance of semantic information in feature map Pi in the Feature Fusion Module (FFM). Therefore, it needs to be modified. The reason why Ai cannot represent the importance of semantic features of Pi in FFM is that FFM uses a residual structure, while LKA is directly applied to the features output by the backbone network. There is an inconsistency in the structure between the two, and this inconsistency will cause the generated attention to not represent the attention features required in FFM. Therefore, using residual learning to modify LKA can enable the attention generated by this structure to effectively supervise the fusion of Pi and Mi. Since the modified structure uses residual learning, it is called Attention Enhancement Module (AEM). Experiments demonstrate that the attention Ai generated by the modified module can effectively supervise the feature fusion of Pi and Mi. The structure of FFM is shown in the attached figure. Figure 7 As shown, one input to the feature fusion section of the FPN is the feature map Pi from its lower layer, supervised by Ai generated from the corresponding layer's AEM; the other input is the dimensionality-reduced feature map Mi from the backbone network. This part of the feature map has already been supervised by the AEM of this layer, so it does not need to be supervised again. Thus, the structure of the entire improved feature fusion module is determined. To simplify the structure, the operation using attention supervision adopts the element-wise multiplication operation common to attention features. Here, Pi represents the i-th layer feature map of the FPN, and Mi represents the output feature map from the AEM.

[0068] The overall structure of the multi-scale detection structure used in this invention is shown in the attached figure. Figure 8 As shown in the diagram. The feature maps output from the backbone network are dimensionality-reduced, and attention features at that scale are generated using the AEM of each branch. Then, FFM is used to fuse the features during the feature fusion stage of FPN. The final output feature maps at the four levels serve as inputs to subsequent networks.

[0069] The Region Proposal Network (RPN) is used to initially distinguish foreground and background regions from pre-defined anchor boxes and predict anchor boxes that may be target bounding boxes. Compared to the selection search algorithm used in Fast R-CNN, the algorithm's running speed is significantly improved, and because the RPN contributes fewer but higher-quality candidate regions compared to the selection search algorithm, it also accelerates the network's convergence speed. The RPN used in this invention is consistent with the RPN in the original version of Faster R-CNN.

[0070] The Faster R-CNN object detection head is divided into an object localization branch and an object classification branch. The entire detection head is used first. The object localization branch utilizes the input features to regress candidate regions output from the RPN and regressed to the original feature map using the RoI Pooling algorithm, further refining the position of the detection boxes. The localization branch used in this invention is consistent with the original Faster R-CNN, and the loss function used is the L1 loss function.

[0071]

[0072] The smooth L1 loss function is defined as follows:

[0073]

[0074] The classification branch of Faster R-CNN uses the input features to classify candidate regions output from the RPN and regressed to the original feature map via the RoIPooling algorithm, determining which category each candidate region belongs to. The localization branch used in this invention is consistent with the original Faster R-CNN, and the loss function used is the commonly used cross-entropy loss function for classification.

[0075]

[0076] The final overall network structure of the algorithm is shown in the attached figure. Figure 7 As shown. The overall loss function of the algorithm is the sum of the losses of all parts to be trained, which is the sum of the classification loss and the localization loss:

[0077]

[0078] Where i is an anchor point in the current mini-batch, and L cls For classification loss, L loc Let λ be the localization loss, and λ be the regularization parameter. i The classification probability predicted by the model. This represents the classification probability of the ground truth label; it is 1 if the anchor point is a positive sample and 0 if it is a negative sample. i The target location predicted by the parameterized model. For parameterized target location in the label. N cls With N reg These are the number of classification samples and the number of regression samples in the current mini-batch, respectively, used for regularization.

[0079] The entire network was constructed according to the relevant content in the invention. The completed network structure, based on Faster R-CNN+FPN, applied AEM+FFM to improve the algorithm's efficiency in fusing features between adjacent scales.

[0080] 3. Network Training

[0081] The network was trained using four NVIDIA Telsa T4 GPUs, each with 16GB of VRAM. The initial learning epochs were 12, with the learning rate reduced to one-tenth of its original value in epochs 9 and 11, resulting in a base learning rate of 0.02. Stochastic gradient descent (SGD) with momentum was used for optimization to avoid getting trapped in local optima. The momentum parameter was 0.9, and the momentum decay regularization term was 0.0001. During training, the deformed image size was 640 pixels wide and 512 pixels high. Multi-scale training was also used, randomly enlarging images to 960×768 pixels to enhance the model's robustness. Backpropagation was used, employing the SGD optimizer to train the network and minimize its loss function to achieve the desired training result.

[0082] 4. Performance Evaluation

[0083] Table 1 shows the experimental results compared with existing algorithms on the TinyPerson dataset. The experimental results demonstrate that the generalization accuracy of this invention on the TinyPerson dataset is superior to other classic algorithms, reflecting the effectiveness of this invention. Furthermore, the experimental data shows that compared to the SSPNet algorithm, which has the highest accuracy on this dataset, this invention achieves better accuracy, demonstrating its superiority.

[0084] Table 1. Accuracy comparison with existing methods (TinyPerson)

[0085]

[0086] Table 2 compares the accuracy of this invention with other existing methods on the VisDrone dataset. The data in the table shows that this invention still achieves competitive accuracy on the VisDrone dataset.

[0087] Table 2. Accuracy comparison with existing methods (VisDrone)

[0088] CornerNet 0.1741 0.3412 - - - CenterNet 0.2603 0.4869 - - - Cascade R-CNN 0.1609 0.3191 - - - RefineDet 0.1490 0.2876 - - - RetinaNet 0.1181 0.2137 - - - This invention 0.2670 0.5150 0.1830 0.3810 0.4370

[0089] In summary, this invention proposes a UAV image target detection algorithm based on the Faster R-CNN target detection algorithm and an attention mechanism based on large kernel equivalent convolution. This invention adds FPN to Faster R-CNN, and further adds an attention mechanism to FPN, which enables the network to better perform feature fusion, thereby improving the performance of UAV image target detection. Experimental results show that this invention has better detection accuracy than existing technologies.

[0090] Obviously, the above examples are merely illustrative and not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations. However, obvious variations or modifications derived therefrom remain within the scope of this invention.

Claims

1. A method for target detection in UAV images based on a large kernel equivalent convolutional attention mechanism, characterized in that, Includes the following steps: (1) First, the training data is preprocessed. Specifically, the annotation files of the input data are transformed. The annotation files of the corresponding images are read and converted into the specified format of the MS COCO dataset. Then, data augmentation operations are performed to increase the amount of training data so that the network training is more thorough. The specific data augmentation techniques used include random rotation, translation and scaling, random brightness and contrast transformation, RGB color shift and saturation transformation, image compression, channel transformation and image blurring. (2) Construct the entire target detection network. The target detection network is based on the Faster R-CNN network. The overall network architecture is divided into four parts: backbone network, neck network, region extraction network and detection head. The backbone network uses the ResNet-50 model, which is divided into stages 0 to 4. Stage 0 is the preprocessing stage of the input data, and stages 1 to 4 are the feature extraction stages. The feature maps generated in stages 1 to 4 are used as the output features of this part in the entire network. The neck network adopts a multi-scale detection structure based on Feature Pyramid Networks (FPN). Attention features at the corresponding scale are obtained by inserting an Attention Enhance Module (AEM) between the features output by the backbone network and the input features of the FPN. In the feature fusion stage of the FPN, a Feature Fusion Module (FFM) is used to perform element-wise multiplication with the attention features at the corresponding scale obtained in the AEM and the features to be fused in the corresponding layer of the FPN, thereby supervising the FPN features. Feature fusion is performed by interpolation amplification and element-wise addition. The features at each scale output from the neck of the network are input into the Region Proposal Network (RPN) to obtain the initial detection regions. The RoI Align algorithm is used to map the proposed detection regions back to the input feature map. The obtained detection regions are then input into the detection head for final detection region filtering. The detection head consists of a common convolutional layer and detection and classification branches. After the input features are processed by the common convolutional layer, the position of the target detection box is regressed through the detection branch, and the target category and confidence score are predicted by the classification branch. (3) After the network is constructed, the processed data is used to train the network to obtain the network weight file that has been trained and converged. The training plan consisted of 12 epochs, with the learning rate reduced to one-tenth of its original value in epochs 9 and 11, respectively. The base learning rate was 0.

02. The optimization algorithm used was stochastic gradient descent (SGD) with momentum. Among the parameters used, the momentum was 0.9 and the momentum decay regularization term was 0.0001. During training, the deformed image size was 640 pixels wide and 512 pixels high. Multi-scale training was also used, randomly enlarging the images to 960×768 pixels to enhance the robustness of the model. (4) Use the trained detector to reason about the given image and obtain the detection result.

2. The UAV image target detection method based on the large kernel equivalent convolution attention mechanism according to claim 1, characterized in that, Step (3) of network training specifically includes the following steps: Training the network requires calculating the loss between the network's predictions and labeled ground truth samples, and minimizing this loss; the loss function used is the L1 loss function: in Indicates category, Represents the bounding box parameter components, The target location predicted by the model. For target localization based on the true label, the smooth L1 loss function is defined as follows: The loss function used in the classification branch is the cross-entropy loss function, which is commonly used in classification loss: in This represents the true label probability of a sample. This indicates the probability that the model predicts the sample belongs to a certain category. The final overall loss function of the algorithm is the sum of the losses of all the parts to be trained, which is the sum of the classification loss and the localization loss. in As an anchor point in the current small batch, For classifying losses, To pinpoint the loss, For regularization parameters; The classification probability predicted by the model. This represents the classification probability of the ground truth label; it is 1 if the anchor point is a positive sample and 0 if it is a negative sample. The target location predicted by the parameterized model. For parameterized target location in the label; and These are the number of classification samples and regression samples in the current mini-batch, respectively, used for regularization; the network is trained by using the back propagation (BP) algorithm of the neural network and an optimizer to minimize the overall loss of the network.

3. The UAV image target detection method based on the large kernel equivalent convolution attention mechanism according to claim 1, characterized in that, Step (4) of image reasoning specifically includes the following steps: (41) Load the image into memory and perform transformation operations on the image. The specific operations are multi-scale scaling and flipping, with scales of 1280×1024 and 1920×1536. (42) Then perform the following operations in sequence: maintain aspect ratio deformation, random flip, regularization, and fill, to convert the image into a tensor; (43) After performing the above operations on the input image, the obtained image data is input into the entire network for forward inference to obtain the inference results of the target category, confidence and detection box in the image; based on the obtained inference results, the detection results are visualized on the input image, and finally the image containing the detection results is saved to the specified location as the output result.

Images