Multi-branch auxiliary fusion improved YOLOv10 unmanned aerial vehicle water surface object target detection method

Abstract

The invention discloses a multi-branch auxiliary fusion improved YOLOv10 unmanned aerial vehicle water surface object target detection method, which comprises the following steps: (1) using an unmanned aerial vehicle to shoot lake water surface floating object images at different heights in a lake water area environment, and recording the overall condition of the lake water surface; (2) selecting a proper labeling tool, and carrying out data set labeling work on the stored pictures; (3) performing data enhancement on the pictures in the data set and the corresponding annotation information, and then dividing the enhanced data set into a training set, a test set and a verification set according to a proportion; (4) a YOLOv10 model is used and improved, a target detection network based on the multi-branch auxiliary fusion improved YOLOv10 model is built, and an obtained data set is used for model training and verification; (5) evaluating an output result; according to the invention, identification of the floating objects on the water surface is realized, so that the cleaning work of the floating objects on the lake water surface is more effectively carried out.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a multi-branch auxiliary fusion improved YOLOv10 unmanned aerial vehicle surface object target detection method. Background Art

[0002] Water resources are an extremely important component of nature. Oceans and lakes account for about 71% of the earth's surface, but only 2.5% of them are fresh water, and only 0.01% of fresh water is suitable for human use. However, with the progress of industry and social development, lakes are facing pollution from various factors such as heavy metals, chemicals, and plastic products. Specifically, floating objects in rivers, including domestic garbage, industrial waste, agricultural residues, and dead plants and animals, have seriously damaged water quality and river landscapes. These pollutants not only affect the natural beauty of rivers, but also destroy aquatic ecosystems and affect the growth and reproduction of aquatic plants and animals. Of particular concern are difficult-to-degrade substances such as plastics and foams, which accumulate in rivers and are difficult to decompose naturally. They not only affect the ecological environment of rivers, but also enter the ocean with river runoff, becoming one of the main causes of marine garbage and microplastic pollution. At present, the method of cleaning floating objects on the water surface still relies on manual salvage and cleaning, but this method is inefficient, costly, and difficult to achieve continuous and comprehensive cleaning. Therefore, the development of efficient automated cleaning technology is crucial to maintaining the ecological balance and beautiful landscape of rivers.

[0003] One of the major advantages of lake surface floating object recognition technology is that it can detect multiple targets at the same time. However, in order to fully utilize this advantage, it is necessary to improve the accuracy of target feature information extraction. In practical applications, the accuracy of image target feature extraction has a huge impact on the dynamic performance of the lake surface floating object recognition system. If the target feature information is not extracted accurately, it may lead to missed recognition or misidentification problems, thereby weakening the advantages of multi-target recognition. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method to solve the problem that inaccurate extraction of target feature information may lead to missed recognition or misrecognition.

[0005] Technical solution: The multi-branch auxiliary fusion improved YOLOv10 UAV surface object target detection method described in the present invention includes the following steps:

[0006] (1) Using drones to capture images of floating objects on the lake surface at different heights in a lake water environment, while recording the overall condition of the lake surface, extracting key frames from the captured video at a set frame rate and saving the image data to a computer;

[0007] (2) Select an appropriate annotation tool to annotate the data set for the images saved in step 1;

[0008] (3) Perform data enhancement on the images and corresponding annotation information in the dataset, and then divide the enhanced dataset into training set, test set, and validation set in proportion;

[0009] (4) Use the YOLOv10 model and make improvements to it, build a target detection network based on the multi-branch assisted fusion improved YOLOv10 model, and use the obtained data set for model training and verification;

[0010] (5) Use labeled data to train classic target detection algorithm models, and evaluate their performance in target detection by comparing the results of the multi-branch assisted fusion improved YOLOv10 model with these classic models.

[0011] Furthermore, step (2) is as follows: Use the LabelImg annotation tool to select the floating object target; convert the annotated VOC format data set, and convert the xml format annotation file into a txt format file for YOLOv10 training.

[0012] Furthermore, step (3) is specifically as follows: data enhancement is performed on the data set and the corresponding annotation information through four methods: horizontal flipping, random brightness, random translation and adaptive histogram equalization.

[0013] Further, step (4) includes the following steps:

[0014] (41) Design a lightweight C2f-EDWIB to replace the traditional C2f module in the backbone network, which effectively realizes the lightweight of the model and significantly reduces the number of parameters and calculations;

[0015] (42) MAFPN is used to replace the original PAN as the Neck network;

[0016] (43) A lightweight attention mechanism Triplet Attention is added after the feature map output by the Neck part;

[0017] (44) A new loss function is introduced to replace the original loss function in YOLOv10.

[0018] Furthermore, step (41) is as follows: first, the input feature map is processed by an additional 3x3 deep convolution layer, then, the feature map is processed by the extended convolution layer, and the number of channels of the feature map is increased by the 1x1 convolution kernel, so that the network can capture more complex features in subsequent layers, and then, another 3x3 deep convolution layer continues to process the feature map to deepen the feature extraction; finally, the projection convolution layer uses a 1x1 convolution kernel to reduce the number of feature channels and prepare the feature map for output or further processing. At the end of the network, a residual connection is added to add the original input to the final feature map.

[0019] Furthermore, step (42) is specifically as follows: in the neck structure, the first bottom-up path uses a shallow auxiliary fusion SAF module to extract multi-scale features from the backbone network and perform preliminary fusion to enhance the model's ability to perceive details; the second top-down path uses an advanced auxiliary fusion AAF module to perform denser connections to integrate gradient information from each layer, thereby generating diverse detection outputs at multiple resolution levels.

[0020] Furthermore, in step (43), Triplet Attention consists of three branches, the first two branches are responsible for the dimensional interaction between the channel and the width or height in the space, and the third branch is used to construct spatial attention.

[0021] Furthermore, MPDIoU is used as a loss function, and the expression is as follows:

[0022] L MPDIoU =1-MPDIoU

[0023] Among them, the definition of MPDIoU is as follows:

[0024]

[0025] Where w and h represent the width and height of the input image respectively; Respectively represent the coordinates of the upper left corner and lower right corner of A; Respectively represent the coordinates of the upper left corner and lower right corner of B; The Euclidean distances between the coordinates of the upper left corner and lower right corner of A and B are calculated respectively; IoU is the intersection over union ratio.

[0026] The multi-branch auxiliary fusion improved YOLOv10 unmanned aerial vehicle surface object target detection system described in the present invention includes:

[0027] Image acquisition module: used to use drones to collect images of floating objects on the lake surface in lake water environments. By adjusting the flight altitude of the drone, images of floating objects on the lake surface at different altitudes can be taken. At the same time, the overall condition of the lake surface can be recorded, key frames can be extracted from the captured video at a set frame rate, and image data can be saved to the computer;

[0028] Image annotation module: select appropriate annotation tools to annotate the saved images;

[0029] Data enhancement module: used to enhance the data by performing data enhancement on the images and corresponding annotation information in the data set, and then divide the enhanced data set into training set, test set and validation set in proportion;

[0030] Model building module: used to use the YOLOv10 model and improve it, build a target detection network based on the multi-branch assisted fusion improved YOLOv10 model, and perform model training and verification based on the data set obtained in step 3;

[0031] Model evaluation module: used to train classic target detection algorithm models using labeled data, and evaluate their performance in target detection by comparing the results of the multi-branch assisted fusion improved YOLOv10 model with these classic models.

[0032] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, any one of the multi-branch assisted fusion improved YOLOv10 unmanned aerial vehicle surface object target detection methods is implemented.

[0033] A storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, any one of the multi-branch assisted fusion improved YOLOv10 unmanned aerial vehicle surface object target detection methods is implemented.

[0034] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: the present invention effectively improves the detection speed and accuracy of the model through the lightweight C2f-EDWIB module; the present invention adopts MAFPN instead of the traditional PAN as the Neck network, and introduces the lightweight attention mechanism Triplet Attention, which significantly reduces the number of network parameters and computational load, while enhancing the detection sensitivity and accuracy of small floating objects; the present invention further improves the alignment accuracy of the predicted frame and the real frame by introducing the MPDIoU loss function, thereby improving the overall positioning performance of the model; the present invention has the ability to use image target feature extraction technology for prediction, so as to realize the identification of floating objects on the water surface, so as to more effectively carry out the cleaning of floating objects on the lake surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Establish a partial flow chart for the model in the present invention;

[0036] Figure 2 The drone of the present invention photographs common floating objects on the water surface; wherein, Figure 2 (a) is foam floating material, Figure 2 (b) is a floating object made of plastic bottles, Figure 2 (c) Plastic bag floating objects, Figure 2 (d) floating objects in cans;

[0037] Figure 3 It is a structural comparison diagram of different convolution methods in the present invention; wherein, Figure 3 (a) is the structure diagram of ordinary convolution. Figure 3 (b) is a diagram of the depth-separable convolution structure;

[0038] Figure 4 It is the structural diagram of EDWIB in the present invention;

[0039] Figure 5 It is a comparison diagram of the structures of C2f and C2f-EDWIB in the present invention, wherein Figure 5 (a) is the organizational diagram of C2f. Figure 5 (b) is the structure diagram of C2f-EDWIB;

[0040] Figure 6 It is the structure diagram of the MAF-YOLOv10 model in the present invention;

[0041] Figure 7 This is a structural diagram of the shallow assisted fusion (SAF) module in the present invention;

[0042] Figure 8 It is a structural diagram of the Advanced Assisted Fusion (AAF) module in the present invention;

[0043] Fig. 9 It is the structural diagram of the TA model in the present invention;

[0044] Fig.10 This is a structural diagram of the multi-branch assisted fusion improved YOLOv10 of the present invention;

[0045] Fig.11 This is a comparison chart of the mAP curves before and after the improvement of the model of the present invention; wherein, Fig.11 (a) shows the comparison curve of mAP@0.5, while Fig.11 (b) shows the comparison curve of mAP@0.5:0.95;

[0046] Fig.12The present invention improves the lightweight YOLOv10 detection result diagram; wherein, Fig.12 (a) Detection results of plastic bottle floating objects, Fig.12 (b) is the test result of plastic bags. Fig.12 (c) is the detection result of floating objects in cans. Fig.12 (d) Foam floating object detection result diagram. DETAILED DESCRIPTION

[0047] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0048] The embodiment of the present invention provides a method for identifying floating objects on the surface of lakes based on an improved lightweight YOLOv10, comprising the following steps:

[0049] Step 1, Image acquisition: Select a suitable lake water environment and use drone camera to take pictures of plastic bottles, plastic bags, cans and foam. According to the experimental design, one picture is extracted every 30 frames to ensure the representativeness and diversity of the data. All the collected image data are then systematically saved in the computer for subsequent analysis and research;

[0050] Step 2: Image annotation: For the image saved in step 1, select the LabelImg annotation tool to annotate the data set;

[0051] Step 2.1: Image annotation: Use the LabelImg annotation tool to clearly identify the types of floating objects on the lake surface when annotating the dataset. Only annotate the objects with clear boundaries and obvious features, and discard some obscured objects with unclear features. Annotate according to the situation in the dataset. When drawing the marking box, make the marking box as close to the boundary of the target object as possible, and name the annotated floating objects.

[0052] Step 2.2: Format conversion: convert the VOC format dataset annotated in step 2.1. The annotation file in xml format needs to be converted into a txt format file for YOLOv10 training.

[0053] Step 3, data enhancement: Process the images collected in step 2 to finally obtain processed images and store them in the control computer;

[0054] Step 3.1, transform the images in the data set and the corresponding annotation information, and perform data enhancement on the data set and the corresponding annotation information through four methods: horizontal flipping, random brightness, random translation, and adaptive histogram equalization;

[0055] Step 3.2: Divide the lake surface floating object dataset in step 3.1 into a training set, a test set, and a validation set in proportion. The division ratio is 8:1:1, that is, the training set has 2000 images, the test set has 200 images, and the validation set has 200 images.

[0056] Step 4: Model building: Establish an improved lightweight YOLOv10 model, and establish the target network model by training the image data obtained in step 3;

[0057] Step 4.1. Based on the UIB architecture in MobileNet-V4, a new structure called Extra Depthwise Inverted Bottleneck (EDWIB) is designed to replace the Bottleneck structure in YOLOv10. EDWIB effectively increases the depth and receptive field of the network by combining the design concept of ConvNext and the efficiency advantage of Inverted Bottleneck (IB), while reducing the computational complexity of the model and alleviating the burden of the model. The EDWIB module is an extension of IB. In order to reduce the number of model parameters, it still uses depth-separable convolution. Figure 3 This is a structural comparison diagram of different convolution methods, where M and N are the number of channels of input and output features respectively; D x and D y are the length and width of the input feature respectively; D k is the size of the convolution kernel; D h and D w are the length and width of the output features, respectively. Figure 3 In (a), the ordinary convolution combines the information of all input channels with the convolution kernel, performs a convolution operation to extract feature information in space and channels, and generates multiple feature maps, each of which corresponds to a convolution kernel. The computational cost of the ordinary convolution operation Q1 is

[0058]

[0059] Figure 3 (b) shows the structure of the depthwise separable convolution, which optimizes network performance by significantly reducing the number of parameters and computational complexity. The depthwise separable convolution first applies an independent convolution kernel to each input channel through depthwise convolution to extract feature maps corresponding to the number of input channels. Then, the information between channels is fused through point-by-point convolution (i.e., 1x1 convolution) to generate the same number of feature maps as ordinary convolution. The amount of convolution operation is Q2.

[0060] Q2=D k ·D k ·M·D w ·D h+M·N·D w ·D h

[0061] From this, we can deduce that the ratio of the computational effort between depthwise separable convolution and ordinary convolution is

[0062]

[0063] From the above formula, we can see that using depthwise separable convolution can reduce the number of model parameters and computation to that of traditional convolution. The model speed will be significantly improved, which is more conducive to deployment in embedded devices.

[0064] like Figure 4 As shown in Figure 1, the core of the EDWIB module is to enhance the module's processing capabilities for input features by adding additional deep convolutional layers before the expansion layer. This design allows EDWIB to maintain or improve the richness and diversity of information by precisely controlling the flow of information while expanding the feature space.

[0065] Specifically, the EDWIB module first inputs the feature map through a 3x3 extra deep convolution layer (ExtraDWLayer), which is specifically used to extract spatial features while keeping the number of channels unchanged. Next, the feature map is processed by the expansion convolution layer (Expansion Layer), which increases the number of channels of the feature map through a 1x1 convolution kernel, enabling the network to capture more complex features in subsequent layers. After that, another 3x3 deep convolution layer (DW Layer) continues to process the feature map to deepen the extraction of features. Finally, the projection convolution layer (Projection Layer) uses a 1x1 convolution kernel to reduce the number of feature channels and prepare the feature map for output or further processing. At the end of the network, a residual connection adds the original input to the final feature map, helping the network avoid the gradient vanishing problem when training deep models while retaining more input information.

[0066] In the C2f module of YOLOv10, there are a series of convolution operations and BottleNeck structures for processing image features. Using the EDWIB structure to replace the original BottleNeck structure can significantly reduce the amount of model calculations, thereby increasing the inference speed. In general, by introducing a faster lightweight module, the computational cost can be effectively reduced without significantly reducing performance. Figure 5 (a) and (b) show the structural comparison of C2f and C2f-EDWIB.

[0067] Step 4.2 Steps, such as Figure 6As shown in the figure, the overall architecture of the MAF-YOLOv10 target detection model is mainly divided into three parts: backbone, neck and head. In the neck structure, the first bottom-up path uses the shallow auxiliary fusion (SAF) module to extract multi-scale features from the backbone network and perform preliminary fusion to enhance the model's ability to perceive details. The second top-down path uses the advanced auxiliary fusion (AAF) module to make more dense connections to integrate the gradient information of each layer, thereby generating diverse detection outputs at multiple resolution levels.

[0068] The Shallow Assisted Fusion (SAF) module is designed to combine high-resolution shallow features and deep information from the same level to preserve important localization details and improve the spatial resolution of the model. Figure 7 , this module strengthens the detection ability of small objects by maintaining the shallow spatial information of the network. Since the shallow information provided by the background network is basic but easily disturbed, we fuse this information into the deep network as an auxiliary channel, thereby improving the stability of the learning process. In addition, in order to effectively control the flow of information during the fusion process, we use a 1x1 convolution kernel to adjust the number of channels of shallow information to ensure that it occupies an appropriate proportion in the network connection to avoid affecting the learning efficiency of subsequent layers. Let P n-1 , P n and P n+1 ∈R H×W×C Represents feature maps of different resolutions, where P n , P n ' and P n ” represents the feature layer of the background network and the two paths of MAFPN. The symbol U(-) represents the upsampling operation. Down represents a 3×3 downsampling convolution accompanied by a batch normalization layer, δ represents a silu function, and C represents a 1×1 convolution to control the number of channels. The output after applying SAF is as follows:

[0069] P n ′=concat(δ(C(Down(P n-1 ))),P n ,U(P n ' +1 ))

[0070] The Advanced Assisted Fusion (AAF) module is used to further enhance the interaction and utilization of feature layers in the deep layers of the MAFPN architecture. Specifically, Figure 8 As shown in Figure 1, the AAF module enhances feature processing by aggregating information across multiple levels, including high- and low-resolution information as well as shallow information at the same level, as well as data from the previous layer. The AAF connection involves the aggregation of information across multiple levels, including high-resolution shallow layers,n ' +1 , low-resolution shallow P n ' -1 , shallow P of the same level n ' and the previous layer P n ″ -1 In this design, the final output layer P4 can simultaneously merge information from four different layers, significantly enhancing the processing performance for medium-sized objects. AAF also uses a 1×1 convolution control channel to adjust the impact of each layer on the result, balancing the number of channels in each layer to ensure that the model obtains diverse outputs. The output results after applying AAF are as follows:

[0071] P n ″=concat(δ(C(Down(P n ' -1 ))),δ(C(Down(P n ″ -1 ))),P n ′,C(U(P n ' +1 )))

[0072] Step 4.3: Add a lightweight attention mechanism TripletAttention after the feature map output by the Neck part; its model structure is as follows Fig. 9 , Triplet Attention consists of three branches. The first two branches are responsible for the dimensional interaction between the channel and the width or height in the space, and the third branch is used to construct spatial attention. Specifically: X∈R C×H×W Input into three branches respectively. In the first two branches, first rotate the input tensor X by 90° counterclockwise according to its width or height, and then rotate X ^ After the Z-Pool operation, the dimension of channel C is reduced to 2, that is, the maximum eigenvalue and the average eigenvalue in the dimension are obtained, and then they are spliced ​​to obtain a two-dimensional tensor X ^* , so that the layer can retain the rich representation of the actual tensor, while reducing the depth and further reducing the amount of calculation. The definition of Z-Pool is shown in formula (1)

[0073] X ^* =[MaxPool od (X ^ ,AvgPool od (X ^ )]

[0074] Then, batch normalization is performed through a convolution layer with a convolution kernel size of k, and then the attention weight is generated through a Sigmoid activation layer. Finally, the tensor is rotated 90° clockwise along the width or height corresponding to the first step operation to make the output tensor consistent with the size of the input tensor. In the third branch, the input tensor width and height are interacted and the rotation operation is removed. Finally, the input tensors of the three branches are aggregated together, that is, the average value is removed after addition. The specific step formula is as follows:

[0075]

[0076] Step 4.4, the YOLOv10 model uses CIoU loss when calculating regression loss. The formula for CIoU loss during calculation is as follows:

[0077]

[0078] Where v is a parameter used to measure the consistency of aspect ratio, and a is a weight function, defined as

[0079]

[0080] Among them, b and b gt Represent the center points of the predicted box and the true box, ρ represents the Euclidean distance, and c represents the diagonal distance of the minimum outer rectangle of the predicted box and the true box. CIoU has shortcomings in measuring the similarity of two bounding boxes, especially when the target is partially occluded and overlapped, it cannot accurately predict the actual situation of the target and is prone to missed detection. Therefore, the introduction of the MPDIoU loss function helps the model to more accurately capture the position and shape information of the target, solve the problem of missed detection in complex environments, and further improve the performance of the task of detecting floating objects on the water surface in complex environments. MPDIoU is a bounding box similarity comparison measure based on the minimum point distance, which proposes a comprehensive improvement in the accuracy and performance of the target detection task. By considering the local distribution of the target box and cross-comparing the distribution of the target area, MPDIoU can capture the shape information of the target more comprehensively, thereby more accurately evaluating the similarity between the two boxes. Moreover, it simplifies the calculation process by directly minimizing the distance between the upper left and lower right points of the predicted bounding box and the actual marked bounding box.

[0081] The definition of MPDIoU is as follows:

[0082]

[0083] The calculation formula is as follows:

[0084]

[0085] Where: w, h represent the width and height of the input image respectively; Respectively represent the coordinates of the upper left corner and lower right corner of A; Respectively represent the coordinates of the upper left corner and lower right corner of B; The Euclidean distances between the coordinates of the upper left corner and lower right corner of A and B are calculated respectively; IoU is the intersection over union ratio.

[0086] MPDIoU is used as the loss function, and the expression is as follows:

[0087] L MPDIoU =1-MPDIoU

[0088] Step 5: Use the labeled data to train the classic target detection algorithm model, and evaluate its performance in target detection by comparing the results of the multi-branch assisted fusion improved YOLOv10 model with these classic models.

[0089] Step 5.1. Use a computer based on Ubuntu 18.04 operating system, equipped with NVIDIA GeForce RTX4090 graphics card, and video memory capacity of 24GB. Select PyTorch deep learning framework as the main development tool, version 1.8.0, and use CUDA10.2.89 to accelerate the training process. In addition, Python interpreter version 3.8 is used, and the SGD optimizer is used to adjust the model parameters. Table 1 provides the detailed configuration of key experimental parameters.

[0090] Table 1 Hyperparameter configuration

[0091]

[0092]

[0093] The present invention uses the commonly used evaluation indicators for target detection: precision (P), recall (R) and mean average precision (MAP). The specific calculation formula is as follows. TP (true positive): true positive, refers to the target in the image being identified as the correct target. FP (false positive): false positive, refers to the location of the target being identified, but the target category is incorrectly identified. FN (false negative): false negative, the correct target is not identified and is considered to be another target, that is, missed detection. N represents the number of categories. AP iIt represents the area under the precision-recall curve of each class. The larger the area, the better the classifier. In addition, in order to reflect the advantages of other models on small devices, the number of parameters, the amount of calculation and the size of the model file are added as evaluation indicators.

[0094]

[0095] In order to verify the effectiveness of the improved algorithm of the multi-branch assisted fusion YOLOv10 model, YOLOv10 was used as the baseline model, and ablation experiments were performed on each module under the same experimental conditions and datasets. The results are shown in Table 2. In Method 2, after replacing the C2f module of the backbone network with C2f-EDWIB, the number of parameters and the amount of calculation of the model decreased by 26% respectively, while the mAP@0.5 decreased by 0.6%, indicating that the model sacrificed some detection performance while reducing complexity and size. Method 3 further reduced the number of parameters of the model while maintaining computational efficiency by integrating the C2f-EDWIB and MAFPN structures. Method 4 further added the TA attention mechanism on the basis of Method 3, significantly improving the performance of mAP@0.5 and mAP@0.5:0.95. Finally, the multi-branch assisted fusion YOLOv10 model proposed in Method5 showed significant performance improvements in mAP and mAP@0.5:0.95, which were 1.8% and 2.7% higher than the YOLOv10 model, respectively, thus demonstrating the effective optimization of the model and significant performance improvement.

[0096] Table 2 Ablation experiment

[0097]

[0098] Step 5.2, input the data set into the classic target detection model and the multi-branch assisted fusion improved YOLOv10 model for comparative experiments, and obtain the results in Table 3. In the evolutionary path of the YOLO series, the multi-branch assisted fusion YOLOv10 model improvement algorithm shows the best performance balance in terms of computational complexity, parameter quantity, accuracy, recall rate and average precision (mAP), including in the Map0.5 index, the multi-branch assisted fusion YOLOv10 model is 4.4% higher than YOLOX, 2.5% higher than YOLOv5, 2.0% higher than YOLO8, and 1.4% higher than Yolov10. In terms of comprehensive consideration of the accuracy and model complexity of lake surface floating object detection, the multi-branch assisted fusion YOLOv10 model improvement algorithm is superior to other mainstream algorithms.

[0099] Table 3 Detection results of different algorithms

[0100]

[0101] Step 5.3: In order to evaluate the performance of multi-branch assisted fusion improved YOLOv10 in actual application environments, the present invention selects various scene images under complex lighting conditions, at different heights, and with small targets at long distances for testing. The detection results are presented visually, where Fig.11 The mAP curve comparison chart of multi-branch assisted fusion improved YOLOv10 and multiple classic target detection algorithms is shown. Fig.12 The detection results of YOLOv10 improved by multi-branch assisted fusion are shown.

[0102] The embodiment of the present invention also provides a multi-branch auxiliary fusion improved YOLOv10 unmanned aerial vehicle surface object target detection system, including:

[0103] Image acquisition module: used to use drones to collect images of floating objects on the lake surface in lake water environments. By adjusting the flight altitude of the drone, images of floating objects on the lake surface at different altitudes can be taken. At the same time, the overall condition of the lake surface can be recorded, key frames can be extracted from the captured video at a set frame rate, and image data can be saved to the computer;

[0104] Image annotation module: select appropriate annotation tools to annotate the saved images;

[0105] Data enhancement module: used to enhance the data by performing data enhancement on the images and corresponding annotation information in the data set, and then divide the enhanced data set into training set, test set and validation set in proportion;

[0106] Model building module: used to use the YOLOv10 model and improve it, build a target detection network based on the multi-branch assisted fusion improved YOLOv10 model, and perform model training and verification based on the data set obtained in step 3;

[0107] Model evaluation module: used to train classic target detection algorithm models using labeled data, and evaluate their performance in target detection by comparing the results of the multi-branch assisted fusion improved YOLOv10 model with these classic models.

[0108] An embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, any one of the multi-branch assisted fusion improved YOLOv10 unmanned aerial vehicle surface object target detection methods is implemented.

[0109] An embodiment of the present invention also provides a storage medium, which stores a computer program. When the computer program is executed by a processor, it implements any one of the multi-branch assisted fusion improved YOLOv10 unmanned aerial vehicle surface object target detection methods.

Claims

1. A multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method, characterized in that: The following steps are involved: (1) Using drones to capture images of floating objects on the lake surface at different heights in a lake water environment, while recording the overall condition of the lake surface, extracting key frames from the captured video at a set frame rate and saving the image data to a computer; (2) Select an appropriate annotation tool to annotate the data set for the images saved in step 1; (3) Perform data enhancement on the images and corresponding annotation information in the dataset, and then divide the enhanced dataset into training set, test set, and validation set in proportion; (4) Use the YOLOv10 model and make improvements to it, build a target detection network based on the multi-branch assisted fusion improved YOLOv10 model, and use the obtained data set for model training and verification; (5) Use labeled data to train classic target detection algorithm models, and evaluate their performance in target detection by comparing the results of the multi-branch assisted fusion improved YOLOv10 model with these classic models.

2. According to claim 1, a multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method is characterized in that: Step (2) is as follows: Use the LabelImg annotation tool to select the floating object target; convert the annotated VOC format data set, and convert the xml format annotation file into a txt format file for YOLOv10 training.

3. According to claim 1, a multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method is characterized in that: Step (3) is as follows: data enhancement is performed on the data set and the corresponding annotation information through four methods: horizontal flipping, random brightness, random translation and adaptive histogram equalization.

4. According to claim 1, a multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method is characterized in that: Step (4) comprises the following steps: (41) Design a lightweight C2f-EDWIB to replace the traditional C2f module in the backbone network, which effectively realizes the lightweight of the model and significantly reduces the number of parameters and calculations; (42) MAFPN is used to replace the original PAN as the Neck network; (43) A lightweight attention mechanism Triplet Attention is added after the feature map output by the Neck part; (44) A new loss function is introduced to replace the original loss function in YOLOv10.

5. According to claim 4, a multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method is characterized in that: Step (41) is as follows: first, the input feature map is processed by an additional 3x3 deep convolution layer. Then, the feature map is processed by the extended convolution layer, which increases the number of channels of the feature map through a 1x1 convolution kernel, so that the network can capture more complex features in subsequent layers. Then, another 3x3 deep convolution layer continues to process the feature map to deepen the feature extraction; finally, the projection convolution layer uses a 1x1 convolution kernel to reduce the number of feature channels and prepare the feature map for output or further processing. At the end of the network, a residual connection is added to add the original input to the final feature map.

6. According to claim 4, a multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method is characterized in that: Step (42) is as follows: In the neck structure, the first bottom-up path uses the shallow auxiliary fusion SAF module to extract multi-scale features from the backbone network and perform preliminary fusion to enhance the model's ability to perceive details; the second top-down path uses the advanced auxiliary fusion AAF module to make denser connections to integrate the gradient information of each layer, thereby generating diverse detection outputs at multiple resolution levels.

7. According to claim 4, a multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method is characterized in that: In step (43), Triplet Attention consists of three branches. The first two branches are responsible for the dimensional interaction between the channel and the width or height in the space, and the third branch is used to construct spatial attention.

8. According to claim 4, a multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method is characterized in that: MPDIoU is used as the loss function, and the expression is as follows: L MPDIoU =1-MPDIoU Among them, the definition of MPDIoU is as follows: Among them, w and h represent the width and height of the input image respectively; Respectively represent the coordinates of the upper left corner and lower right corner of A; Respectively represent the coordinates of the upper left corner and lower right corner of B; The Euclidean distances between the coordinates of the upper left and lower right corners of A and B are calculated respectively; IoU is the intersection over union ratio.

9. A multi-branch assisted fusion improved YOLOv10 UAV surface object target detection system, characterized in that: include: Image acquisition module: used to use drones to collect images of floating objects on the lake surface in lake water environments. By adjusting the flight altitude of the drone, images of floating objects on the lake surface at different altitudes can be taken. At the same time, the overall condition of the lake surface can be recorded, key frames can be extracted from the captured video at a set frame rate, and image data can be saved to the computer; Image annotation module: select appropriate annotation tools to annotate the saved images; Data enhancement module: used to enhance the data by performing data enhancement on the images and corresponding annotation information in the data set, and then divide the enhanced data set into training set, test set and validation set in proportion; Model building module: used to use the YOLOv10 model and improve it, build a target detection network based on the multi-branch assisted fusion improved YOLOv10 model, and perform model training and verification based on the data set obtained in step 3; Model evaluation module: used to train classic target detection algorithm models using labeled data, and evaluate their performance in target detection by comparing the results of the multi-branch assisted fusion improved YOLOv10 model with these classic models.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into the processor, it implements the multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method according to any one of claims 1-8.

11. A storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the multi-branch assisted fusion improved YOLOv10 UAV surface object target detection method according to any one of claims 1-8.

Images