An unmanned aerial vehicle water surface floating garbage detection method and system based on multi-scale dynamic feature fusion

Abstract

The application discloses a kind of based on multi-scale dynamic feature fusion unmanned aerial vehicle water surface floating garbage detection method and system, the unmanned aerial vehicle water surface floating garbage detection method constructs the data set suitable for unmanned aerial vehicle water surface floating garbage detection and garbage detection model.The garbage detection model of the application includes main network, detail enhancement block, neck network and detection head;Variable deformation convolution is introduced in the main network of garbage detection model to construct deformable feature extraction module to enhance the adaptability to garbage target deformation and complex background;At the same time, the neck network of garbage detection model is designed with reparameterization structure to obtain more rich scale information, so as to improve the detection accuracy of model.In addition, a multi-scale coordination attention module is connected between the neck network and the detection head of the garbage detection model to improve the ability of the model to focus on important target features and reduce the interference caused by background noise.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection, and particularly relates to a method and system for detecting water surface floating garbage of an unmanned aerial vehicle based on multi-scale dynamic feature fusion. BACKGROUND

[0002] The global water environment pollution problem is becoming increasingly serious, especially the threat to the ecological system, shipping safety and human health caused by a large amount of plastic garbage entering the water body. Effective monitoring and cleaning of water surface floating garbage has become an urgent task. Traditional water surface garbage patrol mainly relies on manual methods, which is not only inefficient, time-consuming and labor-intensive, but also difficult to achieve real-time and high-precision monitoring of a large range of water areas. Especially in complex water environment or adverse weather conditions, the limitations of manual patrol are more prominent, resulting in problems such as garbage accumulation and water quality deterioration that are difficult to solve in a timely manner.

[0003] In recent years, with the rapid development of computer vision, deep learning and unmanned aerial vehicle technology, new solutions have been provided for intelligent detection of water surface floating garbage. Target detection algorithms based on deep learning can be roughly divided into two-stage detection algorithms and one-stage detection algorithms. Two-stage detection algorithms perform well in detection accuracy, but have limited processing speed, making it difficult to meet the needs of most real-time detection scenarios. One-stage detection algorithms eliminate the step of generating candidate boxes, greatly improving the detection speed, and the accuracy can meet the needs of most real-time scenarios. Although deep learning technology has been gradually applied to the environmental protection field, general deep learning models still face multiple challenges such as poor deformation adaptability, small target detection failure and real-time bottlenecks in view of the characteristics of Hangzhou waterways, such as vertical and horizontal layout, dense ships and strong water surface reflection.

[0004] With the continuous progress of information technology and photography technology, deep learning-based methods provide a new way to improve the efficiency of water surface garbage cleaning. High-definition real-time images of the river channel are obtained using unmanned aerial vehicle equipment, and computer vision technology is used to identify, locate and classify water surface floating garbage in the images. However, the existing technology has the following problems:

[0005] 1. Small target detection problem: In images taken by unmanned aerial vehicles at high altitudes, water surface floating garbage often appears as small targets with small pixel areas. Traditional target detection algorithms may lose features and have high miss detection rates when dealing with small targets, affecting the accuracy of detection.

[0006] 2. Complex background interference: The water surface environment is complex and variable, with various background interference factors such as water ripples, light reflection, shadows and aquatic plants. These factors may reduce the discrimination between targets and backgrounds, increasing the risk of false positives and missed detections.

[0007] 3. Real-time and deployment constraints: Although the YOLO series algorithms have fast detection speed, when deployed on edge devices such as drones, the computational complexity and parameter amount of the model need to be considered to ensure real-time and efficient detection under limited computing resources.

[0008] To address the above challenges, although some improvement methods have been proposed in existing research, such as introducing attention mechanisms, optimizing network structures, or using data augmentation strategies. However, these methods have more or less limitations, such as attention mechanisms that may increase model parameter amount and affect inference efficiency; image enhancement methods that may cause detail loss and are difficult to adapt to real-time scenarios; insufficient attention to target overlap issues, which can easily cause missed detection in heavily polluted areas. Therefore, developing an intelligent detection method that can effectively deal with various complex situations in water surface floating garbage detection while balancing high accuracy, low latency, and edge device deployment capabilities has important theoretical significance and practical application value. SUMMARY

[0009] The present application aims to solve the problems of low efficiency, insufficient real-time performance, difficulty in detecting small targets, complex background interference, target deformation and overlap leading to missed detection, and limited deployment on edge devices in existing water surface floating garbage detection methods. A UAV water surface floating garbage detection method and system based on multi-scale dynamic feature fusion are proposed, which significantly improves the accuracy, robustness, and real-time performance of water surface floating garbage detection, making it more suitable for deployment on UAV edge devices, thereby promoting the intelligent transformation of water environment governance.

[0010] In a first aspect, the present application provides a UAV water surface floating garbage detection method based on multi-scale dynamic feature fusion, which comprises:

[0011] Collecting original images containing water surface floating garbage and constructing a dataset; preprocessing the original images in the dataset to obtain enhanced images;

[0012] Constructing a garbage detection model; the garbage detection model includes a backbone network, a detail enhancement block, a neck network, and a detection head; the backbone network includes four layers of feature extraction layers connected in series;

[0013] The neck network includes a first branch and a second branch arranged side by side; the first branch and the second branch both use asymmetric structure to fuse output feature maps of different scales; the first branch includes three feature fusion modules connected in series for fusing output feature maps of the backbone network; the second branch includes two feature optimization modules connected in series for fusing output feature maps of the first branch;

[0014] The output feature maps of the third feature fusion module and the two feature optimization modules are taken as the output feature maps of the neck network and input into the detection head to obtain the output results of the garbage detection model;

[0015] The garbage detection model is trained using the data set, and the trained garbage detection model is used to perform garbage detection on the image under test.

[0016] As preferred, in the first branch, the input of the first feature fusion module is the splicing result of the convolution processing result of the high-level feature output by the backbone network and the down-sampling processing result of the middle-level feature output by the backbone network; the input of the second feature fusion module and the third feature fusion module is the splicing result of the down-sampling processing result of the low-level feature output by the backbone network and the dynamic up-sampling processing result of the previous feature fusion module;

[0017] In the second branch, the input of the first feature optimization module is the splicing result of the down-sampling processing result of the output feature map of the third feature fusion module and the output feature map of the second feature fusion module; the input of the second feature optimization module is the splicing result of the output feature map of the first feature fusion module, the down-sampling processing result of the output feature map of the second feature fusion module and the down-sampling processing result of the output feature map of the first feature optimization module.

[0018] As preferred, the low-level feature and the middle-level feature are respectively the output feature map of the second layer feature extraction layer and the third layer feature extraction layer in the backbone network; and the high-level feature is the result after the output feature map of the fourth layer feature extraction layer in the backbone network is processed by the detail enhancement block.

[0019] As preferred, the first layer feature extraction layer in the backbone network includes two convolution layers and a deep feature extraction module connected in series; the structures of the remaining feature extraction layers are the same, and each includes a convolution layer and a deep feature extraction module connected in series; the deep feature extraction modules in the first layer feature extraction layer and the third layer feature extraction layer adopt dynamic feature extraction modules; and the deep feature extraction modules in the second layer feature extraction layer and the fourth layer feature extraction layer adopt deformable feature extraction modules.

[0020] As preferred, the dynamic feature extraction module and the deformable feature extraction module each include a bottleneck layer; the bottleneck layer in the dynamic feature extraction module sequentially processes the input feature map of the bottleneck layer through two convolution layers, and fuses the processing result with the input feature map of the bottleneck layer to obtain the output feature map of the bottleneck layer;

[0021] The bottleneck layer in the deformable feature extraction module sequentially processes the input feature map of the bottleneck layer through a convolution layer and a deformable convolution network v2, and fuses the processing result with the input feature map of the bottleneck layer to obtain the output feature map of the bottleneck layer.

[0022] As preferred, the input feature map of the dynamic feature extraction module and the deformable feature extraction module is processed in sequence by a convolution block and a plurality of deep fusion modules connected in series, and the processing result is fused with the input feature map of the first deep fusion module, and finally a convolution block is used to process the fusion result to obtain the output feature map of the dynamic feature extraction module and the deformable feature extraction module.

[0023] As preferred, the deep fusion module comprises a convolution layer, a plurality of bottleneck layers connected in series and a convolution block; the input feature map of the deep fusion module is processed in sequence by a convolution layer and a plurality of bottleneck layers connected in series, and the processing result is fused with the input feature map of the first bottleneck layer, and finally a convolution block is used to process the fusion result to obtain the output feature map of the deep fusion module.

[0024] As preferred, the feature fusion module and the feature optimization module both adopt a cross-stage fusion module; in the cross-stage fusion module, the input feature map of the cross-stage fusion module is processed in sequence by a convolution layer and three reparameterization feature extraction modules, and the output feature maps of the convolution layer and each reparameterization feature extraction module are fused, and the fusion result is processed by a convolution layer to obtain the output feature map of the cross-stage fusion module; the reparameterization feature extraction module comprises a reparameterization convolution layer and a regular convolution layer.

[0025] As preferred, a multi-scale coordination attention module is connected in series between the neck network and the detection head; the multi-scale coordination attention module comprises a first branch and a second branch arranged side by side;

[0026] In the first branch, the input feature map of the multi-scale coordination attention module is processed by adaptive average pooling along the width and height dimensions respectively to obtain vertical context descriptors and horizontal context descriptors; the vertical context descriptors and the horizontal context descriptors are spliced in the spatial dimension, and the spliced feature vectors are transformed in sequence by two convolution layers and an activation function to generate spatial attention weights; the spatial attention weights are divided into two parts according to the width and height to obtain features corresponding to the vertical direction and features corresponding to the horizontal direction, and the two features are fused with the vertical context descriptors and the horizontal context descriptors respectively; the two fusion results are processed by an activation function to generate spatial weights in the vertical direction and spatial weights in the horizontal direction;

[0027] In the second branch, the input feature map of the multi-scale coordination attention module is processed by global average pooling to obtain global channel descriptors; the result processed by a convolution layer from the global channel descriptors is fused with the spatial attention weights, and the fusion result is processed by an activation function to generate channel weights;

[0028] The vertical direction space weight, the horizontal direction space weight and the channel weight are fused with the input feature map of the multi-scale coordinated attention module to obtain an output feature map of the multi-scale coordinated attention module.

[0029] In a second aspect, the present application provides a UAV water surface floating garbage detection system based on multi-scale dynamic feature fusion, which is used to execute the above-mentioned UAV water surface floating garbage detection method.

[0030] The present application has the beneficial effects that:

[0031] 1. The present application introduces a deformable convolution to construct a deformable feature extraction module, and a multi-scale coordinated attention module is connected in series between the neck network and the detection head, which significantly enhances the adaptability and robustness of the garbage detection model to garbage target deformation, complex background (such as water ripples, light reflection) and multi-scale targets.

[0032] 2. The present application performs content-aware scale transformation, asymmetric multi-path information fusion and efficient reparameterization structure in the neck network, realizes efficient fusion of multi-scale features, significantly improves the feature pyramid's representation ability for small targets and overall detection accuracy, effectively solves the problem of easy loss of small target features under high-altitude shooting of the UAV, and provides the subsequent detection head with multi-scale feature maps with higher information density and richer semantic clues; at the same time, the present application introduces a dynamic sampling mechanism in the neck network, which can dynamically generate up-sampling or down-sampling kernels according to the local semantic information of the input feature map, thereby intelligently retaining key details and suppressing noise during scale transformation, which is crucial for maintaining the structural integrity of small targets.

[0033] 3. The present application sets up a cross-stage fusion module in the neck network to perform feature fusion, which widely uses efficient reparameterization structure and cross-stage local network ideas to realize the balance between computational complexity and information flow; at the same time, the reparameterization convolution is used to improve the performance of the model without increasing the inference cost, and the internal dense connection is used to maximize feature reuse.

[0034] 4. While improving performance, the model maintains the lightweight characteristics and can be deployed on UAV edge devices, providing high-precision target positioning for unmanned ship cooperation cleaning and promoting the intelligent transformation of water environment governance. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1Flow chart of the unmanned aerial vehicle water surface floating garbage detection method in the application.

[0036] Figure 2 Collection route map of the unmanned aerial vehicle for collecting original images in the application.

[0037] Figure 3 Part of the original image schematic diagram of the data set in the application.

[0038] Figure 4 Data information schematic diagram of the data set in the application; wherein, (a) is the original image quantity schematic diagram corresponding to each category label in the data set; (b) is the label size distribution schematic diagram in the data set; (c) is the different size target distribution schematic diagram.

[0039] Figure 5 Garbage detection model structure diagram in the application.

[0040] Figure 6 Neck network structure schematic diagram in the application.

[0041] Figure 7 Deformable feature extraction module structure schematic diagram in the application.

[0042] Figure 8 Deep feature extraction module structure schematic diagram in the application.

[0043] Figure 9 Cross-stage fusion module structure schematic diagram in the application.

[0044] Figure 10 Multi-scale coordinated attention module structure schematic diagram in the application. DETAILED DESCRIPTION

[0045] The application is further described below in combination with the drawings.

[0046] The unmanned aerial vehicle water surface floating garbage detection method based on multi-scale dynamic feature fusion adopts an unmanned aerial vehicle water surface floating garbage detection system including a data processing module, a target detection module, and an image acquisition module carried on the unmanned aerial vehicle. The image acquisition module has the characteristics of low delay and high resolution image transmission, and can effectively realize reliable transmission of image data between the unmanned aerial vehicle and the ground station; the image data collected by the image acquisition module is transmitted to the data processing module. The data processing module is used for preprocessing the image data collected by the unmanned aerial vehicle, including data transmission decoding, format conversion, etc., to optimize the input performance of the image data and provide high-quality input for the edge computing device. The target detection module is used for deep learning model reasoning, and the generated detection result is transmitted to the ground workstation through the network and displayed through the visualization interface. Finally, the processing result is fed back to the man-machine interface of the ground end in real time through the data transmission link for real-time viewing and analysis by the operator.

[0047] As shown in Figure 1 , the unmanned aerial vehicle water surface floating garbage detection method includes the following steps:

[0048] Step one, construct a data set

[0049] In order to ensure the effectiveness and robustness of the model in the actual water surface floating garbage detection scene, a data set FlowWaste containing multiple original images is constructed. The collection process of the original images in the data set fully considers the complexity and diversity of the water surface environment, and the collection route of part of the original images in the data set is as shown in Figure 2 . A multi-period (9:00-19:00), multi-height (10m~40m) and multi-view (30°-90° camera pitch angle) aerial photography strategy is adopted for original image collection. This multi-dimensional collection strategy aims to simulate various complex scenarios that may be encountered in actual applications, ensuring the adaptability of the model to different environments, and part of the original images are as shown in Figure 3 . The data set mainly focuses on five common types of water surface floating garbage: paper sheets, express paper boxes, plastic bottles, foams and plastics, which are common pollutants in water surface environments and are representative. The original images in the data set are accurately labeled, i.e., a bounding box is defined for each floating garbage target in the original image, and a corresponding class label is assigned. The labeling tool uses LabelImg, and the labeling format is the TXT format required by YOLO. The data set is divided into training set, test set and validation set according to the ratio of 7:2:1, and the information of the divided data set is as shown in Figure 4 .

[0050] Step two, data augmentation

[0051] The original images are processed through image normalization and mosaic enhancement strategies respectively to obtain enhanced images, completing the preprocessing of image data.

[0052] Step three, constructing a spam detection model

[0053] As shown in Figure 5 , the spam detection model includes a backbone network, a detail enhancement block, a neck network, and a detection head. The backbone network includes four layers of feature extraction layers connected in series; the input of the backbone network is an enhanced image. Except for the first layer of feature extraction layer, the structures of the remaining feature extraction layers are the same, and each includes a convolution layer (convolution kernel size is 3×3, step is 2) and a deep feature extraction module connected in series. The first layer of feature extraction layer includes two convolution layers (convolution kernel size is 3×3, step is 2) and a deep feature extraction module connected in series. The deep feature extraction modules in the first layer of feature extraction layer and the third layer of feature extraction layer adopt dynamic feature extraction modules; the deep feature extraction modules in the second layer of feature extraction layer and the fourth layer of feature extraction layer adopt deformable feature extraction modules (DCSP). The output feature maps of the second layer of feature extraction layer and the third layer of feature extraction layer are respectively taken as low-level features , middle-level features ; the result of the output feature map of the fourth layer of feature extraction layer after being processed by the detail enhancement block is taken as high-level features ; the detail enhancement block includes a fast spatial pyramid pooling module (SPPF) and a cross-stage local spatial attention module (C2PSA) connected in series.

[0054] The traditional feature pyramid network (FPN) and its variants face two major bottlenecks when processing unmanned aerial vehicle images: 1) fixed up-sampling / down-sampling operators are prone to cause loss of key information or introduce artifacts when processing targets with a sharp scale reduction; 2) the symmetrical and strictly hierarchical feature fusion path limits the flexibility and sufficiency of cross-scale information flow. To overcome these limitations, the design of the neck network follows three core principles: content-aware dynamic scale transformation, asymmetric deep cross-scale fusion, and efficient re-parameterization fusion units.

[0055] As shown in Figure 6 , the neck network includes a first branch and a second branch arranged side by side; the first branch includes three feature fusion modules connected in series; the second branch includes two feature optimization modules connected in series.

[0056] In the first branch, the middle-level features are down-sampled and concatenated and fused with the high-level features converted through point convolution (convolution kernel size is 3×3, step is 2) to generate an initial high-level feature map rich in context information. Then, the initial high-level feature map is up-sampled using a content-aware dynamic sampling (DySample) operator, and the processing result is concatenated and fused with the low-level features The splicing and fusion operation is performed to obtain an initial middle-layer feature map The initial middle-layer feature map is dynamically up-sampled and spliced with a low-layer feature The splicing and fusion operation is performed to obtain a final low-layer feature map The fusion operation in the first branch is performed through a feature fusion module.

[0057] In the second branch, the final low-layer feature map is down-sampled and spliced with the initial middle-layer feature map The splicing and fusion operation is performed to obtain a final middle-layer feature map The initial middle-layer feature map and the final middle-layer feature map are both down-sampled and spliced with the initial high-layer feature map The splicing and fusion operation is performed to obtain a final high-layer feature map The fusion operation in the second branch is performed through a feature optimization module.

[0058] In this embodiment, the convolution kernel size in the up-sampling and down-sampling is 3x3, and the step is 2.

[0059] The final low-layer feature map , the final middle-layer feature map and the final high-layer feature map are sequentially processed through a multi-scale coordination attention module (MPCA) and a detection head to obtain the output result of the garbage detection model.

[0060] To solve the problems of small target and large deformation of water-surface floating garbage and complex background interference in the original image taken by a UAV, a deformable convolution network v2 (DCNv2) is introduced into the backbone network to construct a deformable feature extraction module, which can more accurately locate and identify water-surface floating garbage, especially in challenging scenarios such as target deformation, small target and complex background, and shows significant performance improvement. Figure 7 and Figure 8 As shown in the figures, the deformable feature extraction module and the dynamic feature extraction module each include two convolution blocks and a plurality of serially connected deep fusion modules; the input feature map of the deformable feature extraction module is sequentially processed through a convolution block and a plurality of serially connected deep fusion modules, and the processing result is fused with the input feature map of the first deep fusion module, and finally a convolution block is used to process the fusion result to obtain the output feature map of the deformable feature extraction module.

[0061] The deep fusion module comprises a convolutional layer, a plurality of bottleneck layers connected in series, and a convolutional block; the input feature map of the deep fusion module is processed in turn by the convolutional layer and the plurality of bottleneck layers connected in series, and the processing result is fused with the input feature map of the first bottleneck layer, and finally the fusion result is processed by the convolutional block to obtain the output feature map of the deep fusion module. By introducing the deep fusion module in the deformable feature extraction module, the details of the small target can be captured more effectively, and the robustness of the model to complex backgrounds can be enhanced.

[0062] The bottleneck layer in the deformable feature extraction module comprises a 1x1 convolutional layer and a deformable convolutional network v2 (DCNv2); the input feature map of the bottleneck layer is processed in turn by the convolutional layer and the deformable convolutional network v2, and the processing result is fused with the input feature map to obtain the output feature map of the bottleneck layer in the deformable feature extraction module. The bottleneck layer in the dynamic feature extraction module comprises a 1x1 convolutional layer and a 3x3 convolutional layer; the input feature map of the bottleneck layer is processed in turn by the two convolutional layers, and the processing result is fused with the input feature map to obtain the output feature map of the bottleneck layer in the dynamic feature extraction module. The traditional standard convolutional operation adopts a fixed shape of the convolutional kernel, and the sampling point position is predetermined. This makes it difficult for the standard convolution to effectively capture the features of irregularly deformed targets or complex backgrounds. Deformable convolution (Deformable Convolutional Networks, DCN) learns an additional two-dimensional offset at each sampling point, so that the sampling position of the convolutional kernel can be adaptively adjusted according to the content of the input feature map. DCNv2 further introduces a learnable modulation mechanism to learn a weight for each sampling point, so that it can more flexibly adjust the contribution of each sampling point to the output feature. At the same time, by introducing two-dimensional offset and modulation scalar in DCNv2, DCNv2 can adaptively adjust the shape and size of the receptive field, so as to better adapt to the geometric deformation of the target and effectively distinguish the target from the background. The mathematical expression of DCNv2 is as follows:

[0063]

[0064] wherein, is any position on the output feature map y ; K is the total number of sampling points in the convolutional kernel; is the weight of the convolutional kernel; is the input feature map; is the predefined standard convolutional kernel sampling point position; is the learned two-dimensional offset, so that the sampling point can deviate from its original position; is a learned modulation scalar used to adjust the contribution of each sample point to the output, whose value is usually between [0, 1].

[0065] In this embodiment, the convolutional block in the deformable feature extraction module and the dynamic feature extraction module includes a convolutional layer, a batch normalization layer (BN) and a SiLU activation function connected in sequence.

[0066] In order to enhance the feature fusion capability without introducing excessive computational burden, the feature fusion module and the feature optimization module in the neck network both adopt a cross-stage fusion module (RCSP, Reparameterized Cross Stage Partial), which widely uses an efficient reparameterization structure and a cross-stage partial (CSP, Cross Stage Partial Stage) network idea. The cross-stage fusion module balances the computational load and the information flow through the CSP architecture; it improves the performance of the model without increasing the inference cost by using the RepConv; and it maximizes feature reuse by means of internal dense connections.

[0067] As shown in Figure 9 , the cross-stage fusion module includes two 1x1 convolutional layers and three reparameterization feature extraction modules; in the cross-stage fusion module, the input feature map of the cross-stage fusion module is processed through a convolutional layer and three reparameterization feature extraction modules in sequence, the output feature maps of the convolutional layer and each reparameterization feature extraction module are fused, and the fusion result is processed through a convolutional layer, thereby efficiently aggregating features from different paths and different depths to generate a feature map with strong representation ability as the output feature map of the cross-stage fusion module.

[0068] In this embodiment, the reparameterization feature extraction module includes a 3x3 reparameterization convolutional layer and a 3x3 regular convolutional layer connected in series.

[0069] Due to the complexity of the water surface environment, background noise such as light reflection, water ripples and shadows seriously interferes with target detection. In order to enhance the robustness of the model to these disturbances and improve the feature expression ability, a multi-scale coordination attention module is set before the large, medium and small scale detection heads of the garbage detection model, and the structural details of the module are as shown in Figure 10 . The multi-scale coordination attention module includes a first branch and a second branch arranged side by side.

[0070] In the first branch, the input feature map of the multi-scale coordination attention module is processed through adaptive average pooling along the width and height dimensions respectively to obtain vertical context descriptors and horizontal context descriptors ; the vertical context descriptors and the horizontal context descriptors The splicing is performed in the spatial dimension to form a unified tensor , and the spliced feature vector is transformed through a 3x1 convolution layer, a 1x1 convolution layer and a Sigmoid activation function in turn to generate refined spatial attention weights . The spatial attention weights are divided into two parts according to the width and height to obtain the corresponding vertical direction feature and the corresponding horizontal direction feature . The features and are fused with the vertical context descriptor and the horizontal context descriptor respectively, and the two fusion results are processed through a Sigmoid activation function to generate the vertical direction spatial weight and the horizontal direction spatial weight .

[0071] In the second branch, the input feature map of the multi-scale coordinated attention module is processed by global average pooling (GAP) to obtain a global channel descriptor ; the result after convolution layer processing of the global channel descriptor is fused with the spatial attention weight , and the fusion result is processed through a Sigmoid activation function to generate a channel weight .

[0072] The generated spatial weight , spatial weight and channel weight are multiplied element by element with the input feature map of the multi-scale coordinated attention module through the broadcast mechanism, and the output feature tensor after re-standardization is output as the output feature map of the multi-scale coordinated attention module. In this way, the multi-scale coordinated attention module can dynamically and adaptively enhance the information-rich features while suppressing noise, thereby improving the overall performance of the network model. In order to comprehensively model the features, while considering the correlation between channels and the long-distance dependence in space, the multi-scale coordinated attention module constructs three parallel encoding paths to effectively capture the three different dimensional context information respectively. Independently calculating each dimensional attention will ignore the internal correlation between them, so the multi-scale coordinated attention module designs an information interaction path to realize the synergistic enhancement of attention, rather than simple linear superposition.

[0073] Step four, training the model

[0074] The garbage detection model is trained using the training set and the training result is evaluated through the test set and the validation set. In terms of garbage detection model training, the application is trained in the following software and hardware environment: a graphic processing unit NVIDIA RTX4060Ti, a central processing unit Intel Core i5-14600K, a GPU memory of 16G, an operating system Win11, a computing platform CUDA12.4, and a deep learning framework PyTorch2.5.1. In addition, the parameters during training are as follows: an AdamW optimizer is used, the initial learning rate is 0.001, and the weight decay coefficient is 0.0005; a cosine annealing learning rate scheduling strategy is adopted; the batch size is 16; the training round is 300; the input image size is 1280x1280; and the loss function of the garbage detection model includes a classification loss, a confidence loss and a bounding box regression loss.

[0075] Step five, model evaluation

[0076] To measure the performance of the garbage detection model in the application, the evaluation indexes including mAP (mean average precision), Recall (recall rate), Precision (precision rate), FPS (frames per second) and Prams (parameter quantity) are adopted. The mean average precision refers to the average value of AP (average precision) of all categories, and is used to measure the overall detection performance of the model on all categories. The application mainly focuses on mAP@0.5 (mAP when IoU=0.5) and mAP@0.5:0.95 (average value of mAP when the IoU threshold is from 0.5 to 0.95 with a step of 0.05). The recall rate is the proportion of the number of correctly detected targets to the number of all real targets, and is used to measure the ability of the model to find all related targets. The precision rate is the proportion of the number of correctly detected targets to the number of all detection results, and is used to measure the accuracy of the detection results of the model. In addition, the frames per second is used to measure the real-time processing capability of the model on the given hardware. The parameter quantity is the model parameter quantity and the model file size, and is used to measure the deployment feasibility of the model on the edge device.

[0077] The garbage detection model is fully trained and evaluated on the data set FlowWaste, and a comparative experiment is conducted with a benchmark model (YOLOv11), and the experimental results are shown in Table 1.

[0078] Table 1 Comparative experimental results of the application and YOLOv11

[0079]

[0080] As can be seen from Table 1, in terms of accuracy, the present application is superior to the benchmark model YOLOv11 in precision rate, recall rate, F1 score, average precision mean (mAP@0.5) at IoU = 0.5 and average precision mean (mAP@0.5:0.95) at IoU = 0.5 to 0.95, with an increase of 5.30% and 3.46%. This shows that the improvement strategy proposed in the present application effectively improves the detection accuracy of the model for water surface floating garbage. While maintaining high accuracy, the inference speed of the present application slightly decreases, but still reaches 76FPS, meeting the real-time detection requirements. The model parameter quantity and size slightly increase, but still remain in the lightweight range, suitable for deployment on unmanned aerial vehicles and other edge devices.

[0081] In order to verify the effectiveness of each improvement strategy, the present application conducts an ablation experiment, and the experimental results are shown in Table 2.

[0082] Table 2 Ablation experiment results

[0083]

[0084] As can be seen from Table 2, each improvement strategy positively contributes to the model performance. DCSP1 represents modifying DCSP only in the shallow layer (P2, P3) of the backbone network, and DCSP2 represents modifying DCSP only in the deep layer (P4, P5) of the backbone network. It can be seen that using deformable convolution in the deep network better adapts to the geometric deformation of the target. The precision and speed indicators of the neck network designed in the present application are superior to the generalized feature pyramid (GFPN). In order to suppress the interference problem caused by background noise, a multi-scale coordination attention module is designed and inserted in front of the detection head at different scales, effectively improving the detection accuracy of the model. When all the improvement strategies are combined, the model performance reaches the best, verifying the effectiveness of the overall scheme of the present application.

Claims

1. A method for detecting garbage floating on water surface based on multi-scale dynamic feature fusion, characterized in that: The method comprises: Collecting original images containing floating garbage on the water surface and constructing a data set; preprocessing the original images in the data set to obtain enhanced images; Constructing a garbage detection model; the garbage detection model comprises a backbone network, a detail enhancement block, a neck network and a detection head; the backbone network comprises four layers of feature extraction layers connected in series; The first layer of feature extraction layers in the backbone network comprises two convolution layers and a deep feature extraction module connected in series; the structures of the remaining feature extraction layers are the same, each comprising a convolution layer and a deep feature extraction module connected in series; the deep feature extraction modules in the first layer of feature extraction layers and the third layer of feature extraction layers adopt dynamic feature extraction modules; the deep feature extraction modules in the second layer of feature extraction layers and the fourth layer of feature extraction layers adopt deformable feature extraction modules; The dynamic feature extraction module and the deformable feature extraction module each comprise a bottleneck layer; the bottleneck layer in the dynamic feature extraction module processes the input feature map of the bottleneck layer through two convolution layers in sequence, and fuses the processing result with the input feature map of the bottleneck layer to obtain the output feature map of the bottleneck layer; The bottleneck layer in the deformable feature extraction module processes the input feature map of the bottleneck layer through a convolution layer and a deformable convolution network v2 in sequence, and fuses the processing result with the input feature map of the bottleneck layer to obtain the output feature map of the bottleneck layer; The neck network comprises a first branch and a second branch arranged side by side; the first branch and the second branch each adopt an asymmetric structure to fuse output feature maps of different scales; the first branch comprises three feature fusion modules connected in series and used for fusing the output feature maps of the backbone network; the second branch comprises two feature optimization modules connected in series and used for fusing the output feature maps of the first branch; The feature fusion module and the feature optimization module each adopt a cross-stage fusion module; in the cross-stage fusion module, the input feature map of the cross-stage fusion module is processed through a convolution layer and three reparameterization feature extraction modules in sequence, and the output feature maps of the convolution layer and each reparameterization feature extraction module are fused; the fusion result is processed through a convolution layer to obtain the output feature map of the cross-stage fusion module; the reparameterization feature extraction module comprises a reparameterization convolution layer and a regular convolution layer; The output feature maps of the third feature fusion module and the two feature optimization modules are taken as the output feature maps of the neck network and input into the detection head to obtain the output result of the garbage detection model; The garbage detection model is trained using the data set, and the trained garbage detection model is used to detect garbage in the image to be detected.

2. The unmanned aerial vehicle water surface floating garbage detection method based on multi-scale dynamic feature fusion according to claim 1, characterized in that: In the first branch, the input of the first feature fusion module is the splicing result of the convolution processing result of the high-level features output by the backbone network and the down-sampling processing result of the middle-level features output by the backbone network; the inputs of the second feature fusion module and the third feature fusion module are each the splicing result of the down-sampling processing result of the low-level features output by the backbone network and the dynamic up-sampling processing result of the previous feature fusion module. In the second branch, the input of the first feature optimization module is the splicing result of the down-sampling processing result of the output feature map of the third feature fusion module and the output feature map of the second feature fusion module; the input of the second feature optimization module is the splicing result of the output feature map of the first feature fusion module, the down-sampling processing result of the output feature map of the second feature fusion module and the down-sampling processing result of the output feature map of the first feature optimization module.

3. The unmanned aerial vehicle water surface floating garbage detection method based on multi-scale dynamic feature fusion according to claim 2, characterized in that: The low-level feature and the middle-level feature are respectively the output feature maps of the second feature extraction layer and the third feature extraction layer in the backbone network; and the high-level feature is the result obtained by processing the output feature map of the fourth feature extraction layer in the backbone network through the detail enhancement block.

4. The unmanned aerial vehicle water surface floating garbage detection method based on multi-scale dynamic feature fusion according to claim 1, characterized in that: The input feature map of the dynamic feature extraction module and the deformable feature extraction module is processed through a convolution block and a plurality of serial depth fusion modules in turn, and the processing result is fused with the input feature map of the first depth fusion module, and finally a convolution block is used to process the fusion result to obtain the output feature map of the dynamic feature extraction module and the deformable feature extraction module.

5. The unmanned aerial vehicle water surface floating garbage detection method based on multi-scale dynamic feature fusion according to claim 4, characterized in that: The depth fusion module comprises a convolution layer, a plurality of serial bottleneck layers and a convolution block; the input feature map of the depth fusion module is processed through a convolution layer and a plurality of serial bottleneck layers in turn, and the processing result is fused with the input feature map of the first bottleneck layer, and finally a convolution block is used to process the fusion result to obtain the output feature map of the depth fusion module.

6. The unmanned aerial vehicle water surface floating garbage detection method based on multi-scale dynamic feature fusion according to claim 1, characterized in that: The neck network is connected in series with a multi-scale coordination attention module between the detection head; the multi-scale coordination attention module comprises parallel first and second branches; In the first branch, the input feature map is subjected to average pooling in the width and height dimensions respectively to obtain vertical and horizontal context descriptors; the vertical and horizontal context descriptors are spliced and then processed through two convolution layers and an activation function in turn to obtain spatial attention weights, and the spatial attention weights are divided into two parts according to the width and height to obtain features corresponding to the vertical direction and the horizontal direction; the two features are fused with the vertical and horizontal context descriptors respectively and then processed through an activation function to obtain spatial weights in the vertical direction and the horizontal direction; In the second branch, the input feature map is processed through global average pooling to obtain a global channel descriptor; the global channel descriptor is processed through a convolution layer, and the result is fused with the spatial attention weights, and the fusion result is processed through an activation function to generate channel weights; The vertical spatial weight, the horizontal spatial weight and the channel weight are fused with the input feature map of the multi-scale coordination attention module to obtain the output feature map of the multi-scale coordination attention module.

7. A multi-scale dynamic feature fusion-based unmanned aerial vehicle water surface floating garbage detection system, characterized in that: The unmanned aerial vehicle water surface floating garbage detection system comprises an unmanned aerial vehicle, a data processing module and a target detection module. An image acquisition module is arranged on the unmanned aerial vehicle, and is used for acquiring image data; a data processing module is used for pre-processing the acquired image data; and a target detection module is used for obtaining a detection result according to the pre-processed image data.

Images