A multi-scale feature filtering and reorganization method for water surface floating object monitoring

Abstract

This invention discloses a multi-scale feature filtering and reconstruction method for monitoring floating objects on the water surface, belonging to the field of water surface floating object monitoring technology. The method includes the following steps: S1, inputting UAV images of the water surface into a hybrid backbone network; S2, performing feature fusion through a neck network. This invention possesses advantages such as low computational complexity, strong anti-interference capability against complex water surface backgrounds, high accuracy in detecting multi-scale and long-tailed targets, and accurate positioning of slender and deformed floating objects. It is very suitable for deployment on UAVs or unmanned surface vessels with limited computing power, and can meet the real-time, high-precision monitoring needs of floating objects on the water surface in various scenarios of natural and complex water surfaces.

Description

Technical Field

[0001] This invention relates to the field of water surface floating object monitoring technology, specifically a multi-scale feature filtering and reconstruction method for water surface floating object monitoring. Background Technology

[0002] Traditional methods of monitoring floating objects on water surfaces rely primarily on manual inspections or simple image processing techniques, which are inefficient and resource-intensive. With the development of drone technology and deep learning, image target recognition algorithms based on drone perspectives have been widely applied in floating object monitoring. However, the natural water environment is extremely complex, often accompanied by disturbances such as water surface fluctuations, strong light reflections, dynamic light and shadow, and fog. Furthermore, floating objects exhibit characteristics such as a wide scale range, susceptibility to non-rigid distortions in shape, and a severe "long-tailed distribution" of categories. Existing detection algorithms based on conventional deep learning often face technical bottlenecks when dealing with these complex and variable scenarios, including high false alarm rates, difficulty in locating slender and small targets, and significant missed detections of rare categories.

[0003] Patent CN116416462A discloses a method for identifying floating objects on lake surfaces based on PP-PicoDet. This method, building upon the original PP-PicoDet, uses ESNet as the backbone network and introduces a contextual attention module (CAM) at the neck to enhance the connection between features and context. However, while focusing on improving the attention mechanism, this method lacks a mechanism to address the severe "long-tail distribution" problem prevalent in natural floating object datasets. When faced with categories that are scarce but extremely harmful, the model is easily dominated by common head categories, resulting in extremely low recall rates for rare categories. Furthermore, the purely lightweight convolutional network extracts only a single feature, failing to address the problem of highly similar floating objects with different categories and materials, as well as the fine-grained feature confusion between floating objects and similar backgrounds, leading to frequent misclassifications.

[0004] Patent CN116597314A discloses a method for identifying floating objects on the water surface based on a lightweight improved SSD algorithm. This method aims to improve target detection speed for operation on small embedded platforms. Specifically, it replaces the original VGG16 network structure with the MobileNetV2 network structure and significantly reduces the number of parameters and computational cost by utilizing depthwise separable convolutions. However, the purely lightweight convolutional neural network (CNN) architecture is limited by its local receptive field and lacks the ability to capture global long-range dependent semantic information. When facing complex water surface backgrounds in drone aerial photography (such as large areas of oil spills or interference from overall water flow trends), its low-level feature maps often contain a large amount of noise and lack deep semantics, resulting in extremely limited feature extraction.

[0005] Patent CN117036656B discloses a method for identifying floating objects on the water surface in complex scenarios. This method optimizes the YOLOv7 algorithm by proposing a YOLOv7-FC model. It incorporates a CBAM convolutional attention mechanism in the feature extraction stage and uses an SPPFCSPC module for feature enhancement in the neck region. It also employs CIoU (Complete Intersection over Union) loss in the position loss. However, CIoU loss is extremely sensitive to the overlap of bounding boxes, failing to address the issue that when the drone's flight altitude or the unmanned vessel's viewpoint changes, the pixel proportion of small-scale floating objects at long distances is extremely small. This leads to gradient vanishing and severe missed detections in the IoU-based regression process.

[0006] Patent CN117523462A discloses a method for detecting floating objects on the water surface based on an improved Faster-RCNN. This method uses ResNet50 to extract rich features and applies K-means clustering before the RPN network. However, the two-stage Faster-RCNN architecture is bulky and fails to address the problems of excessive parameter quantity and computational complexity in existing high-precision floating object detection algorithms, resulting in poor timeliness and inability to meet real-time detection requirements on computing platforms with limited computing power and memory, such as UAVs or unmanned surface vessels. Furthermore, its feature pyramid fusion process fails to effectively address the problem of high-frequency background noise generated by water ripples, sunlight reflection, and reflections when UAVs or unmanned surface vessels are cruising on complex water surfaces, leading to floating objects easily being confused with the background and generating numerous false alarms.

[0007] Patent CN120388288A discloses a method for detecting floating objects on the water surface based on an improved YOLOv8 model. This model constructs a CSP_DCNv2CoordConv module, combining coordinate convolution and deformable convolution, and uses a C2S_Upsample module for super-resolution reconstruction. However, despite the introduction of coordinate convolution, its detection head structure is not completely decoupled. Even with shared features between classification and regression tasks, it still struggles to completely resolve the problem of inaccurate localization caused by the difficulty in extracting spatial orientation information from slender or irregularly shaped floating objects. Furthermore, its multi-scale super-resolution fusion mechanism does not actively suppress shallow noise, making it susceptible to false alarms due to high-frequency background noise on the water surface.

[0008] Therefore, existing methods or systems have failed to comprehensively address the complex problems in natural water surface scenes, such as high-frequency background noise interference, multi-scale feature spatial misalignment, vanishing regression gradients of small targets, insufficient learning of long-tailed distributed samples, confusion of fine-grained features, and inaccurate localization of irregular targets. To address these issues, we propose a method for monitoring floating objects on the water surface based on multi-scale feature filtering and reconstruction of UAV imagery. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a multi-scale feature filtering and reconstruction method for monitoring floating objects on the water surface, solving the problems mentioned in the background section.

[0010] To achieve the above objectives, the present invention provides a multi-scale feature filtering and reconstruction method for monitoring floating objects on the water surface, comprising the following steps: S1. Input the surface drone imagery into the hybrid backbone network and extract multi-scale features of local texture and global semantics; S2. Feature fusion is performed through the neck network, top-down background suppression is performed using the Semantic Guided Filtering (SGFM) module, and bottom-up feature alignment is performed using the Depth Reconstruction (DRM) module. S3. Input the fused features into the four-scale decoupled detection head of the CoordAtt fusion coordinate attention mechanism, and output the category confidence and bounding box coordinates of the floating objects on the water surface. S4. During the training phase, the network parameters are optimized using a joint loss function based on NWD and EIoU, and the long-tailed distribution of samples is handled using a targeted Copy-Paste data augmentation strategy.

[0011] Furthermore, the hybrid backbone network sequentially includes a ConvNeXt module and a Swin Transformer module; wherein, the ConvNeXt module is used to output shallow features downsampled by 4 times (C2) and 8 times (C3), and the Swin Transformer module is used to receive the C3 features and output deep features downsampled by 16 times (C4) and 32 times (C5), and after the output of the C5 layer, it is connected to the SPPF spatial pyramid pooling module.

[0012] Furthermore, the processing logic of the semantic guided filtering module (SGFM) is as follows: after upsampling the deep features, they are input into a lightweight gated convolutional network, which sequentially passes them through depthwise separable convolution, batch normalization, ReLU activation, 1×1 convolution, and the Sigmoid function to generate a background probability mask M with values ​​ranging from 0 to 1; the inverse value (1−M) of this mask is multiplied element-wise with the shallow features, and a learnable residual parameter α is used for baseline fusion, resulting in the following output formula: .

[0013] Furthermore, the processing logic of the detail reconstruction module (DRM) is as follows: the shallow features are downsampled and concatenated with the deep features, and the spatial offset is predicted by 3×3 convolution; The offset is used to drive deformable convolution (DCN) to perform grid resampling and alignment of shallow features; finally, the aligned features are injected into deep features through residual structures.

[0014] Furthermore, the four-scale decoupled detection head includes four levels of downsampling: 4x, 8x, 16x, and 32x. Each level is divided into two independent branches: classification and regression. A coordinate attention (CoordAtt) module is embedded between the convolutional layers of each branch. This module captures position-sensitive information through adaptive average pooling along the horizontal and vertical directions, respectively, and performs attention weighting through cross-multiplication.

[0015] Furthermore, the joint loss function is composed of a weighted average of QualityFocalLoss, EIoULoss, and NWDLoss; the NWDLoss models the predicted box and the ground truth box as a two-dimensional Gaussian distribution and calculates the normalized Wasserstein distance as the regression loss.

[0016] Furthermore, the targeted Copy-Paste data augmentation strategy specifically involves: constructing an instance sample pool during the training phase to trigger augmentation operations with a set probability; when determining the object category to be pasted, if the random number r < 0.8, forcibly randomly selecting target instances from a rare category set with insufficient sample size; and adaptively pasting the selected target instances into the background image after random scaling and out-of-bounds truncation protection.

[0017] This invention provides a method for monitoring floating debris on the water surface using multi-scale feature filtering and reconstruction, which has the following advantages: This method for monitoring floating debris on the water surface based on UAV imagery combines the ConvNeXt module with Swin. The hybrid backbone network of the Transformer module enables efficient extraction of local fine-grained texture and global long-range semantics, overcoming the challenges of fine-grained feature confusion and excessive computational complexity. To address interference from water ripples and strong light reflection, a semantic guided filtering module (SGFM) is designed for top-down active background suppression, reducing the false alarm rate. To address the misalignment problem in multi-scale feature fusion, a detail reconstruction module (DRM) is employed, utilizing deformable convolutions to achieve precise spatial alignment of shallow and deep features. To solve the problems of gradient vanishing from small floating objects and inaccurate localization of slender targets, a joint loss function based on NWD and EIoU is introduced, and a four-scale decoupled detection head incorporating coordinate attention (CoordAtt) is constructed, ensuring accurate bounding selection of small and irregular targets. Furthermore, to address the low recognition rate of rare floating objects, a class-biased directional copy-paste data augmentation strategy is adopted, fundamentally breaking down the barrier of extreme long-tail distribution of floating object data.

[0018] This invention has advantages such as low computational complexity, strong anti-interference ability in complex water surface backgrounds, high detection accuracy for multi-scale and long-tailed targets, and accurate positioning of slender and deformed floating objects. It is very suitable for deployment on UAVs or unmanned vessel platforms with limited computing power and can meet the real-time, high-precision monitoring needs of floating objects on natural complex water surfaces in multiple scenarios. Attached Figure Description

[0019] Figure 1 This is a flowchart of the multi-scale feature filtering and reconstruction method for monitoring floating objects on the water surface provided in this embodiment; Figure 2 This is a flowchart of the semantic guided filtering module (SGFM) provided in this embodiment; Figure 3 This is a flowchart of the Detail Reorganization Module (DRM) provided in this embodiment; Figure 4 This is a flowchart of the four-scale decoupled detection head based on the CoordAtt fusion coordinate attention (CoordAtt) provided in this embodiment; Figure 5 This is a category distribution diagram based on the open-source dataset provided in this embodiment; Figure 6 This is a graph showing the detection accuracy (PR curve) on an open-source dataset, based on the example provided in this embodiment. Figure 7 This is a confusion matrix diagram based on the embodiment provided. Figure 8 This is a recognition effect diagram in a calm scene provided in this embodiment; Figure 9 This is a recognition effect diagram in a ripple scene provided in this embodiment; Figure 10 This is a recognition effect diagram in a strong light scene provided in this embodiment; Figure 11 This is a recognition effect diagram in a dense scene provided in this embodiment; Figure 12 This is a recognition effect diagram in a small-scale scene provided in this embodiment; Figure 13 This is a comparison chart of the accuracy of different algorithms provided in this embodiment. Detailed Implementation

[0020] Figure 1 A flowchart illustrating the multi-scale feature filtering and reconstruction method for monitoring floating debris on the water surface provided in this embodiment is shown, as follows: Figure 1 As shown, the method includes the following steps: S1. Input the water surface UAV imagery into the hybrid backbone network to extract multi-scale features of local texture and global semantics.

[0021] Convolutional Neural Networks (CNNs) are primarily used to extract fine-grained local features in shallow layers of images. They can efficiently capture the edge, color, and texture information of floating objects on the water surface through the sliding of convolutional kernels, improving the extraction efficiency of shallow features. However, traditional pure CNN architectures, due to their limited receptive field, are confined to feature aggregation within local windows and lack the ability to capture global semantic information. When facing water surface reflections, wave interference, and complex backgrounds in drone aerial photography, the low-level feature maps often contain a large amount of noise, easily leading to missed detections of small or occluded targets.

[0022] To overcome the impact of changes in UAV flight altitude and trajectory on the imaging resolution of floating objects on the water surface, as well as the interference caused by complex water surface backgrounds, this invention proposes an improved hybrid backbone network based on ConvNeXt and Swin Transformer modules. Specifically, by combining the advantages of CNN and Swin Transformer modules, the extraction structure of deep low-resolution feature layers and shallow high-resolution feature layers is adjusted. Through the cascade fusion of ConvNeXt and Swin Transformer modules, efficient extraction of local texture and global semantics is achieved.

[0023] A hybrid backbone network generates feature maps of different scales at different layers to cover floating targets of various sizes. First, the input image is processed by a lightweight ConvNeXt module to generate shallow local feature maps, including layers C2 (downsampled by 4x) and C3 (downsampled by 8x). Then, the C3 feature map is passed as input to the Swin Transformer module. The core idea of ​​the Swin Transformer module is to establish long-range dependencies by calculating self-attention within non-overlapping local windows using a moving window mechanism. The self-attention calculation formula with relative positional bias is as follows: (1) Where Q, K, and V represent the query, key, and value matrices within the local window, respectively, and d is the feature dimension, used to scale the dot product result to prevent gradient vanishing. B is the relative position bias matrix. The introduction of the relative position bias B enables the network to better handle the relative spatial relationships between floating objects on the water surface, as well as between floating objects and water ripples.

[0024] This hybrid backbone network, after being processed by Stage 3 and Stage 4 of the Swin Transformer module, generates deep feature maps C4 (downsampled by 16x) and C5 (downsampled by 32x). Since traditional networks have limited receptive field expansion during semantic extraction at the highest layer, making it difficult to extract large-scale floating objects, this invention incorporates the SPPF (Spatial Pyramid Pooling - Fast) module after the output of layer C5. SPPF fuses multi-scale global features through multiple cascaded max pooling operations with fixed kernel sizes. The cascaded pooling formula is as follows: (twenty one) (twenty two) (twenty three) (twenty four) Where x is the input C5 feature map, and k is the pooling kernel size. In this network architecture, the pooling kernel size is set to k=5.

[0025] The traditional single backbone network is divided into four hybrid feature layers (C2, C3, C4, and C5) for output. Since traditional methods lack utilization of features downsampled to four times higher resolution, they struggle to extract extremely small-scale floating objects on the water surface. Therefore, this invention specifically retains the C2 layer output from the ConvNeXt module for subsequent network fusion. This hybrid backbone network improvement addresses the problem of missed detection of small targets caused by the variability in altitude and the complexity of the water surface background during UAV imaging. It not only preserves extremely high-resolution local textures for multi-scale floating targets but also introduces anti-interference global semantics, effectively detecting extremely small-scale floating objects on the water surface.

[0026] S2. Feature fusion is performed through the neck network, top-down background suppression is performed using the semantic guided filtering module (SGFM), and bottom-up feature alignment is performed using the detail reconstruction module.

[0027] Strong light reflection from the water surface and high-frequency wave textures are the core sources of interference leading to false alarms in the system. In multi-scale object detection, traditional Feature Pyramid Networks (FPNs) typically use simple element-wise addition or concatenation to fuse deep semantic features with shallow spatial features. However, in natural water environments, the shallow high-resolution feature maps of UAV aerial images not only contain the edge textures of floating objects but are also filled with a large amount of background noise such as water ripples, strong light reflections, and dynamic lighting. Direct fusion leads to the contamination of deep, pure semantic information by shallow noise, thereby causing false detections and alarms regarding floating objects on the water surface.

[0028] To overcome the interference of complex water surface background noise on multi-scale feature fusion, this invention proposes a Semantic Guided Filtering Module (SGFM). Specifically, in the top-down path of the neck network, a background probability-gated mask is generated using the high-level semantic features rich in the deep network (i.e., the network has already understood "where the large water surface is") as guidance. Through this intelligent gating switch, the network adaptively suppresses the response intensity of the water surface reflective areas in the shallow features, allowing the real floating object signals to stand out naturally. SGFM eliminates background artifacts from the bottom layer of the network structure, directly cutting off the unnecessary mechanical energy consumption caused by false detections.

[0029] The semantic-guided filtering module performs layer-by-layer filtering between adjacent upper and lower feature layers. Figure 2 A flowchart of the semantic guided filtering module (SGFM) according to this embodiment is shown, as follows: Figure 2 As shown, assume the input deep feature is fhigh (e.g., the P5 layer after dimensionality reduction), and the shallow feature to be fused is flow (e.g., the P4 layer after dimensionality reduction). First, to achieve spatial dimensionality alignment, the deep feature fhigh is upsampled using bilinear interpolation to ensure its spatial resolution matches that of the shallow feature flow. (3) Subsequently, the aligned deep features fhighup are input into a lightweight gated convolutional network to generate a background probability mask. To capture spatial adjacency while reducing computational complexity, the gated network sequentially employs depthwise separable convolutions (DWConv) with the number of groups equal to the channel dimension, batch normalization (BatchNorm), the ReLU activation function, and a 1×1 pointwise convolution for cross-channel information interaction. Finally, the output features are mapped to the 0-1 interval using the Sigmoid activation function to generate a single-channel spatial probability mask M. The specific formula for mask generation is: (4) Here, σ represents the Sigmoid function. The closer the pixel value in the mask M is to 1, the higher the probability that the area is water surface background noise; conversely, it represents a floating object target area.

[0030] After obtaining the background probability mask, this module employs an adaptive filtering mechanism to suppress shallow features. The shallow feature flow is multiplied element-wise with the inverse mask (1-M) to accurately filter out background areas such as water surface reflections and ripples. Simultaneously, to ensure the stability of feature propagation and smooth backpropagation of network gradients, this module introduces a learnable baseline residual parameter α (initialized to 0). The final filtered output feature f... outThe calculation formula is: (5) in, This represents the Hadamard product (element-by-element multiplication).

[0031] This invention improves upon the traditional simple feature addition process by employing a top-down semantic-guided filtering method. SGFM modules are deployed in the feature fusion stages of P5 to P4, P4 to P3, and the newly added P3 to P2 (Stride=4, specifically for extremely small targets). Through the improved SGFM algorithm, the problem of shallow feature contamination in UAV imagery caused by water surface ripples and strong light reflection can be effectively solved. While preserving high-resolution positioning information of small floating objects, it significantly reduces the false alarm rate and background interference in complex water environments.

[0032] In multi-scale object detection, while shallow feature maps contain rich location and fine-grained texture information, traditional bottom-up feature enhancement paths typically employ fixed pooling or conventional stride convolution for downsampling, followed by direct addition with deeper features. However, floating objects on the water surface under natural conditions (such as plastic bags and fishing nets) are prone to morphological distortion and non-rigid deformation under the influence of water flow and waves. Conventional downsampling fusion methods, due to their fixed receptive fields, struggle to adapt to such irregular deformations, easily leading to severe spatial misalignment of shallow textures when propagating to deeper layers.

[0033] To overcome the feature fusion misalignment problem caused by the variable shape of floating objects on the water surface, this invention proposes a Detail Reorganization Module (DRM). Specifically, it introduces Deformable Convolution (DCN) into the enhancement path, uses deep features as a benchmark, and adaptively adjusts the sampling grid of shallow features to achieve accurate detail texture alignment and reorganization.

[0034] The detailed reorganization module is deployed layer by layer from bottom to top in the enhancement path of the neck network. Figure 3 This is a flowchart of the detail reorganization module (DRM) provided in this embodiment, as follows: Figure 3 As shown, assuming the input deep feature is fhigh (as the spatial reference), and the shallow feature to be injected is flow (containing rich texture), firstly, to unify the resolution for offset calculation, the shallow feature flow is downsampled using bilinear interpolation to ensure its spatial dimensions are strictly aligned with the deep feature fhigh. (6) Subsequently, in order to perceive the deformation trend of the floating object, this module compares the deep feature fhigh with the aligned shallow feature. The concatenation is performed along the channel dimension. The concatenated fused features are then input into a 3×3 ordinary convolutional layer to learn and predict the spatial offset tensor. Since the 3×3 convolutional kernel contains 9 sampling points, and each sampling point is offset in both the x and y directions, the number of output channels for this convolutional layer is set to 18. The offset calculation formula is: (7) Obtaining spatial offset Subsequently, this module utilizes deformable convolution (DCN) to adaptively align shallow features. This differs from conventional convolution's fixed grid sampling points p. k DCN will predict the dynamic offset Superimposed onto the receptive field mesh. Shallow features are then processed on this adaptive mesh. Sampling and convolution calculations are performed to accurately extract the texture details of deformable floating objects. The deformable convolution alignment formula is expressed as: (8) Where p is the current position on the feature map, K is the total number of sampling points in the convolution kernel (K=9 here), and wk is the learning weight at the corresponding position.

[0035] Finally, the shallow features aligned by DCN are input into a fusion layer consisting of 3×3 convolutions, batch normalization (BatchNorm), and ReLU activation functions for feature extraction. These features are then directly superimposed into the deep baseline feature fhigh via residual connections to complete the detailed reconstruction. The output feature f... out The calculation formula is: (9) This invention improves upon traditional fixed downsampling fusion by employing adaptive detail reconstruction based on dynamic offset. The DRM module is comprehensively deployed in the enhancement paths of P2 to P3, P3 to P4, and P4 to P5 (where layer P2 preserves high-resolution shallow features, with a stride of 4). This DRM algorithm improvement addresses the positional deviation problem caused by morphological distortion of floating objects due to water surface undulations, enabling the model to successfully reconstruct accurate shallow boundary textures in deep semantics, effectively improving the positioning accuracy of non-rigid floating objects.

[0036] S3. Input the fused features into the four-scale decoupled detection head of the fused coordinate attention mechanism, and output the category confidence and bounding box coordinates of the floating objects on the water surface.

[0037] Traditional single-stage object detection networks typically employ coupled detection heads, where feature mapping is performed simultaneously on the same convolutional feature map, directly outputting the object's class probability and bounding box location. However, classification and localization / regression tasks have fundamentally conflicting spatial response preferences for features: classification tasks require features to possess translation invariance to identify "what it is," while localization / regression tasks require features to possess translation covariance to determine "where it is." Furthermore, floating objects on natural water surfaces often include slender and irregularly shaped objects such as ropes and nets. Traditional two-dimensional convolutional operations, due to their regular square receptive fields, lack the ability to perceive features in specific directions and long-span spatial locations, easily leading to missed detections of slender targets or inaccurate bounding box regression.

[0038] To overcome the performance bottleneck caused by the aforementioned coupling features and the difficulty in locating slender floating objects, this invention proposes a four-scale decoupled detection head that integrates a coordinate attention mechanism (CoordAtt). Specifically, the classification and regression tasks are decoupled at the end of the network, and a coordinate attention module is embedded in the feature extraction process. High-precision localization of complex-shaped floating objects is achieved through cross-dimensional direction perception weighting.

[0039] Figure 4 The flowchart of the four-scale decoupled detection head with fused coordinate attention (CoordAtt) according to this embodiment is shown. Figure 4 As shown, the decoupled detection head designed in this invention receives fused feature maps at four scales (i.e., layers P2, P3, P4, and P5, with corresponding downsampling strides of 4, 8, 16, and 32, respectively) from the output of the neck network. Specifically, layer P2, with a downsampling of only 4 times, is retained, greatly preserving the spatial resolution of extremely small floating objects. For the input features at each scale, the network divides them into two independent branches in parallel: a classification branch and a regression branch. Each branch performs feature mapping independently, using 3×3 convolution, group normalization, and ReLU activation function for initial feature extraction.

[0040] To enhance the network's sensitivity to slender target boundaries, a coordinate attention module (CoordAtt) is embedded within both decoupled branches. The CoordAtt module decomposes the two-dimensional positional information into two one-dimensional orientation-aware feature maps, encoded along the horizontal (X-axis) and vertical (Y-axis) directions, respectively. Assuming the input feature map is X∈ℝC×H×W, adaptive average pooling kernels of size (H, 1) and (1, W) are first used to perform one-dimensional feature aggregation on channel c along the vertical and horizontal directions, respectively. The formulas for the output zch(h) with a vertical height of h and the output zcw(w) with a horizontal width of w are as follows: (10) (11) Subsequently, the aggregated features from these two directions are concatenated spatially, and an intermediate feature map f containing information from both directions is generated using a 1×1 convolution F1, batch normalization (BatchNorm), and the SiLU activation function δ. (12) Next, the intermediate feature f is re-split along the spatial dimension into two independent tensors fh and fw, and then passed through two additional 1×1 convolutions F. h and F w By combining the sigmoid activation function σ, attention weights a in the vertical and horizontal directions are generated. h and a w : (13) (14) Finally, the attention weights of these two dimensions are multiplied back into the original input features. Figure X At the corresponding position, the final output feature Y with the coordinate attention mechanism applied is obtained: (15) This invention improves upon the traditional single-scale coupled detection head by transforming it into a four-scale decoupled detection head with orientation awareness, effectively eliminating feature conflicts between classification and regression tasks. The improved CoordAtt algorithm solves the problem of accurately selecting slender, irregular floating objects such as ropes and nets on the water surface. The combined effect of the outputs of the four scale feature layers (especially the P2 layer which retains extremely high-resolution shallow features) and coordinate attention weights significantly enhances the model's localization accuracy and multi-classification ability for complex floating objects even when the UAV's perspective changes.

[0041] S4. During the training phase, the network parameters are optimized using a joint loss function based on NWD and EIoU, and the long-tailed distribution of samples is handled using a targeted Copy-Paste data augmentation strategy.

[0042] In existing object detection algorithms, Intersection over Union (IoU) and its derivatives (such as GIoU and CIoU) are commonly used as the loss function for bounding box regression. However, in UAV surface monitoring scenarios, there are numerous extremely small floating objects. For such tiny targets, even a one or two-pixel deviation in the bounding box position can cause a precipitous drop in the IoU value, making it difficult for the network to generate smooth, continuous gradients for effective optimization. Furthermore, traditional static label assignment strategies are prone to causing a "misalignment" problem between classification scores and localization accuracy; that is, predicted boxes with high classification confidence often have inaccurate positions.

[0043] To overcome the difficulties in retrieving the location of small targets and the decrease in detection accuracy caused by inconsistencies in classification and localization tasks, this invention proposes a joint loss function based on NWD (Normalized Wasserstein Distance) and EIoU, combined with a task alignment allocation strategy. The specific details are as follows: 1) Normalized Wasserstein Distance Loss (NWD Loss) for Small Targets To address the issue of extremely small targets being overly sensitive to positional deviations, this invention abandons the overlap calculation method that relies entirely on IoU. Instead, it models both the predicted and ground truth bounding boxes as two-dimensional Gaussian distributions. The similarity between the two is measured by calculating the Wasserstein distance between the two two-dimensional Gaussian distributions. Assume the center point coordinates, width, and height of the predicted box P and the ground truth box G are (cx... p ,cy p ,w p, h p ) and (cx a ,cy a ,w a ,h a Its second-order Wasserstein distance squared W 2 The formula for calculating 2(P,G) is: (16) To map the distance to the interval 0-1 for easier network optimization, a normalization constant C is introduced (C=12.8 in this invention), resulting in a normalized NWD index and the corresponding NWD loss function: (17) (18) The NWD loss, by measuring distribution characteristics, can provide smooth gradient guidance even when two bounding boxes do not intersect at all, greatly improving the convergence effect of floating objects at extremely small scales on the water surface.

[0044] 2) Enhanced Cross-Union Loss (EIoU Loss) For conventional and large-scale floating objects, this invention introduces EIoU loss in the regression branch. Unlike traditional CIoU, EIoU calculates the difference penalties for the predicted and ground truth bounding boxes in the width and height directions separately, solving the convergence slowdown problem caused by the inability to simultaneously and independently optimize width and height in the aspect ratio penalty term. The formula for calculating EIoU loss is: (19) Where ρ2 is the square of the Euclidean distance between the centers, c w and c h These are the width and height of the smallest closure region that simultaneously encloses the predicted bounding box and the ground truth bounding box, respectively.

[0045] 3) Quality Focal Loss (QFL) and Dynamic Label Allocation To address the inconsistency between classification and localization, this invention employs QFL loss in the classification branch. QFL incorporates the IoU quality between the predicted and ground truth bounding boxes as a soft label into the cross-entropy. Assuming the joint prediction score of the model is σ, and the soft label target quality is y, the formula for QFL is: (20) Simultaneously, a dynamic task alignment allocator is employed to calculate the alignment metric t=s based on the classification score s and the IoU value u. α ∙u β During training, the network dynamically selects the Top-K anchor points with the highest alignment as positive samples for each real target based on the metric t.

[0046] 4) Calculation of joint loss function The overall network loss of this invention is composed of the weighted sum of the above three parts. By assigning different weight coefficients to classification, EIoU regression, and small-objective NWD regression, the optimal balance of multi-task collaborative optimization of the model is achieved. (twenty one) Based on experiments using a large dataset of water surface data, the weighting coefficients were set to λ1=3.0, λ2=2.5, and λ3=1.0, respectively. This improvement to the joint loss function addresses the divergence problem in localizing small targets caused by variations in UAV altitude, ensuring high-precision bounding box regression and accurate classification of various floating objects on complex, dynamically lit water surfaces.

[0047] Figure 5 The following diagrams illustrate the categories of the open-source dataset provided in this embodiment: Figure 5As shown, in real-world scenarios of monitoring floating objects on natural water surfaces, datasets often exhibit a severe "long-tail distribution." This means that the number of common floating object samples is extremely large, while samples of certain floating objects with specific hazards or unique morphologies are extremely scarce. Traditional image-level data augmentation algorithms (such as random cropping, horizontal flipping, or color transformation) can only change the appearance of existing samples and cannot fundamentally increase the number of instances of rare categories. This causes the neural network to be dominated by the common categories at the head during training, resulting in extremely low recall rates for detecting rare categories at the tail.

[0048] To overcome the problem of missed detection of some categories due to the long-tail distribution of floating objects on the water surface, this invention proposes a targeted copy-paste data augmentation strategy for the long-tail distribution of water surface objects. Specifically, during the training phase, an independent instance-level sample library is established, and rare targets are injected into the complex water surface background through class-biased probability sampling and adaptive boundary fusion. The targeted copy-paste data augmentation strategy is dynamically executed during the data loading phase of each training batch of the model. First, during the training initialization phase, the annotation files of the entire dataset are scanned to extract the normalized bounding box coordinates (cx, cy, w, h) of all floating objects, and a structured multi-class instance sample pool is constructed according to the class labels.

[0049] Before the image is input into the network, a copy-paste operation is triggered with a set probability (50% in this invention). Unlike traditional completely random sampling, this module introduces a targeted long-tail compensation mechanism: categories with fewer than a threshold of samples are predefined as the rare category set Crare. When determining the object category to paste, a random number r between 0 and 1 is generated. When r < 0.8 (i.e., a high probability of 80%), the model is forced to randomly sample target instances only from the rare category set Crare; otherwise, it samples from the entire category set. This biased sampling formula greatly increases the exposure frequency of tail categories during training.

[0050] After selecting the source instance image Isrc to be pasted, its absolute pixel coordinates in the source image are calculated based on its normalized coordinates, and the region of interest (ROI) is extracted. To prevent the pasted object from being too large and obscuring the original features of the main image Isrc, it is randomly scaled. Assuming the scaling factor is S (S~U(0.5, 1.0)), the scaled target size (w) new ,h new )for: (twenty two) (twenty three) Among them, Wmain and Hmain are the width and height of the main image.

[0051] Subsequently, in the main image I main Randomly generate the top left corner coordinates (x) of the pasted text. p , y p To address training errors or gradient anomalies caused by the target bounding box exceeding the image boundaries, this module performs strict double out-of-bounds truncation and protection calculations. The formula for calculating the actual bottom-right corner coordinates (xp2, yp2) of the pasted data is as follows: (twenty four) (25) The actual effective pasting area after truncation is And the corresponding ROI pixel values ​​are overlaid on the area, while the new bounding box coordinates (x) are set. p ,y p ,x p2 ,y p2 The category labels are attached to the set of real labels of the main image.

[0052] In addition, to make the pasted rare objects blend more naturally with the water background, this invention combines the directional Copy-Paste strategy with a high-probability (80%) Mosaic multi-image stitching technology, and supplements it with adaptive photometric distortion (including color jitter and Gaussian noise to simulate water reflection and fog noise) and perspective transformation (to simulate the imaging distortion of the UAV at different pitch angles).

[0053] This invention improves upon traditional random data augmentation by injecting targeted instances based on long-tail distribution. This augmentation strategy effectively breaks down the class imbalance barrier in natural water surface datasets, effectively compensates for the characteristics of tail-class samples, and significantly enhances the model's ability to learn rare objects. Through targeted copy-paste improvement, it solves the problem of missed detections of rare and crucial floating objects such as fishing nets and buoys from a drone's perspective due to insufficient samples, greatly improving the model's generalization robustness and the balance of detection accuracy across classes in complex water surface environments.

[0054] To verify the application effect and robustness of the proposed multi-scale feature filtering and reconstruction method for monitoring floating objects on the water surface based on UAV imagery in real-world scenarios, this invention conducted comprehensive experiments and tests on an open-source floating object dataset.

[0055] It should be noted that, having solved the problem of long-tailed data distribution, this invention further evaluates the overall detection accuracy of the model. Figure 6The diagram shows the PR (Precision-Recall) curve provided according to this embodiment, from... Figure 6 The fullness of the curves for each category and the overall integral area show that, under the synergistic optimization of the multi-scale feature filtering and recombination mechanism and the NWD-based joint loss function, the model's mean accuracy (mAP@0.5) across all categories reaches a high level of approximately 86%. This objective data demonstrates that even under the harsh conditions of a UAV's high-altitude overhead view, facing extremely small target pixels and the high likelihood of bounding box regression gradient divergence, this invention can still maintain extremely high target recall and detection accuracy, effectively overcoming the technical challenge of easily missing small-scale floating objects at long distances.

[0056] To verify the model's comprehensive performance in fine-grained classification and complex-shaped target localization, this invention statistically analyzed the classification prediction results on the test set. Figure 7 An obfuscation matrix diagram according to this embodiment is shown, such as Figure 7 As shown, the values ​​on the diagonal of the normalized confusion matrix remain at very high levels, while the values ​​in the off-diagonal region and the background false alarm region are extremely low. This intuitively demonstrates that the four-scale decoupled detection head with fused coordinate attention (CoordAtt) used in this invention can accurately extract the spatial orientation features of slender and irregular objects. This structure not only completely decouples the feature conflicts between classification and regression tasks, but also effectively overcomes the problem of frequent misclassification and false alarms caused by similar shapes, similar materials, or background interference of floating objects on the water surface, ensuring accurate identification of targets in complex environments.

[0057] In practical drone patrol applications, the aquatic environment changes rapidly. To further demonstrate the engineering application value of this invention in real natural waters, the system underwent visual reasoning tests for several typical environments. Figures 8 to 12 The following diagrams illustrate the recognition results under this embodiment in calm scenes, ripple scenes, strong light scenes, dense scenes, and small-scale scenes, respectively. Figures 8 to 12 As shown, the model of this invention exhibits exceptionally strong anti-interference capabilities in various environments, including relatively calm waters, complex waters with dense, small-scale floating debris, and challenging imaging conditions accompanied by severe surface ripple disturbances and strong light reflection. Through the powerful active suppression of high-frequency background light and shadow noise by the Semantic Guided Filtering (SGFM) module and the precise spatial alignment of shallow and deep features by the Details Reconstruction (DRM) module, the model successfully achieves high-precision bounding selection of multi-scale floating debris against complex dynamic water surface backgrounds. These application results fully demonstrate that this invention comprehensively meets the real-time, high-precision, and highly robust dynamic monitoring requirements of mobile edge computing platforms in real, complex water environments.

[0058] To verify the advancement and effectiveness of the proposed algorithm, a quantitative performance comparison experiment was conducted on an open-source floating object dataset, comparing it with current mainstream classical object detection algorithms. The dataset used in this experiment was strictly divided into training, validation, and test sets. The training set contained 1901 images, the validation set contained 238 images, and the test set contained 238 images. To ensure the fairness and objectivity of the comparison experiment, all algorithms were trained and tested under the same hardware and software experimental environment. The experimental hardware platform was equipped with a Tesla-V100-PCIE-32GB GPU, an 8-core CPU with 16GB of memory, and the deep learning framework used was PyTorch. The training input parameters are shown in Table 1.

[0059] Table 1 Training Input Parameters

[0060] Figure 13 A comparison chart of the accuracy of different algorithms provided in this embodiment is shown, such as... Figure 13 As shown, under the unified experimental environment and dataset division described above, this invention selected the classic single-stage algorithm SSD, the authoritative two-stage algorithm Faster R-CNN, and the modern lightweight industrial-grade algorithm YOLOv5 as benchmark comparison models. Table 2 shows the performance comparison of different algorithms on the open-source floating debris dataset.

[0061] Table 2 Performance Comparison of the Invention with Different Algorithms

[0062] As can be seen from the comparison of the data in the charts, this invention achieves optimal detection performance while maintaining computational efficiency: (1) Compared with the modern lightweight algorithm YOLOv5: In order to overcome the problem of extremely complex light and shadow and long tail distribution on the water surface, this invention introduces a multi-scale decoupling and filtering recombination mechanism, which increases GFLOPs from 15.86 to 135.71 and the inference speed from 274.69 FPS to 25.13 FPS. However, mAP@0.5 is significantly improved from 78.4% to 85.8% (a significant improvement of 7.4 percentage points), which reflects the reasonable balance between sacrificing some redundant ultra-high frame rate in exchange for extreme detection accuracy in complex scenes. (2) Compared with the classic two-stage algorithm Faster R-CNN: the computational cost of the present invention is reduced by 35.4% (from 210.24 to 135.71), and while the inference speed remains basically the same (25.13 FPS vs. 26.70 FPS), the mAP@0.5 is significantly improved by 14.5 percentage points (from 71.3% to 85.8%), which reflects the dual advantages of the present invention in feature representation efficiency and detection accuracy; (3) Compared with the early single-stage algorithm SSD: Although the increased feature extraction layer increases the computational load, thanks to the optimization of the network architecture, the inference speed is improved by 46.4% (from 17.17 FPS to 25.13 FPS), and mAP@0.5 achieves a leapfrog improvement of 36.5 percentage points (from 49.3% to 85.8%), which is comprehensively and thoroughly superior to the traditional SSD algorithm.

[0063] This invention achieves a high accuracy of 85.8% mAP@0.5 on an open-source floating surface object dataset, while maintaining a near real-time inference speed of 25.13 FPS. It effectively solves the problems in floating surface object monitoring tasks based on UAV imagery, such as strong light ripple background interference, loss of small target features, easy confusion of fine-grained materials, and the high likelihood of missing rare floating objects due to extreme long-tail distribution. The comprehensive comparative results strongly demonstrate that the method proposed in this invention possesses superior detection accuracy and extremely high practical engineering application value in floating object monitoring tasks in complex natural water environments.

Claims

1. A method for monitoring floating debris on water surfaces using multi-scale feature filtering and reconstruction, characterized in that, Includes the following steps: S1. Input the surface drone imagery into the hybrid backbone network and extract multi-scale features of local texture and global semantics; S2. Feature fusion is performed through the neck network, and a semantic guided filtering module is used for top-down background suppression, while a detail reconstruction module is used for bottom-up feature alignment. S3. Input the fused features into the four-scale decoupled detection head of the fused coordinate attention mechanism, and output the category confidence and bounding box coordinates of the floating objects on the water surface. S4. During the training phase, the network parameters are optimized using a joint loss function based on NWD and EIoU, and the long-tailed distribution of samples is handled using a targeted Copy-Paste data augmentation strategy.

2. The method for monitoring floating debris on the water surface using multi-scale feature filtering and reconstruction according to claim 1, characterized in that, The hybrid backbone network includes a ConvNeXt module and a Swin Transformer module. The ConvNeXt module is used to output shallow features downsampled by 4x and 8x, and the Swin Transformer module is used to receive shallow features downsampled by 8x and output deep features downsampled by 16x and 32x. After the 32x downsampled output, it is connected to the SPPF spatial pyramid pooling module.

3. The method for monitoring floating debris on the water surface using multi-scale feature filtering and reconstruction according to claim 1, characterized in that, The semantic-guided filtering module's processing logic is as follows: Deep features are upsampled and input into a lightweight gated convolutional network. The network sequentially passes through depthwise separable convolution, batch normalization, ReLU activation, 1×1 convolution, and the Sigmoid function to generate a background probability mask M with values ​​ranging from 0 to 1. The inverse value (1−M) of this mask is then multiplied element-wise with the shallow features, and a learnable residual parameter α is used for baseline fusion. The output formula is: ; In the formula, f out is a filter output feature, f low is a shallow feature to be fused.

4. The method for monitoring floating debris on the water surface using multi-scale feature filtering and reconstruction according to claim 1, characterized in that, The processing logic of the detailed reconstruction module is as follows: the shallow features are downsampled and concatenated with the deep features, and the spatial offset is predicted by 3×3 convolution. The offset is used to drive deformable convolution to perform grid resampling and alignment of shallow features; finally, the aligned features are injected into deep features through residual structures.

5. The method for monitoring floating debris on the water surface using multi-scale feature filtering and reconstruction according to claim 1, characterized in that, The four-scale decoupled detection head includes four levels of downsampling: 4x, 8x, 16x, and 32x. Each level is divided into two independent branches: classification and regression. A coordinate attention module is embedded between the convolutional layers of each branch. This module captures position-sensitive information through adaptive average pooling along the horizontal and vertical directions, respectively, and performs attention weighting through cross-multiplication.

6. The method for monitoring floating objects on the water surface using multi-scale feature filtering and reconstruction according to claim 1, characterized in that, The joint loss function is composed of a weighted average of QualityFocalLoss, EIoULoss, and NWDLoss; the NWDLoss models the predicted box and the ground truth box as a two-dimensional Gaussian distribution and calculates the normalized Wasserstein distance as the regression loss.

7. The method for monitoring floating objects on the water surface using multi-scale feature filtering and reconstruction according to claim 1, characterized in that, The targeted Copy-Paste data augmentation strategy is as follows: during the training phase, an instance sample pool is constructed to trigger augmentation operations with a set probability; when determining the object category to be pasted, if the random number r < 0.8, a target instance is forcibly randomly selected from the rare category set with insufficient sample size; after random scaling and out-of-bounds truncation protection, the selected target instance is adaptively pasted into the background image.

Images