Method and related apparatus for underwater target detection

Abstract

This application discloses an underwater target detection method and related apparatus. The method includes: inputting an image to be detected into a pre-trained target detection model; extracting features from the image through the backbone network of the target detection model to obtain multiple feature maps of different resolutions; fusing the multiple feature maps through the neck network of the target detection model to obtain fused feature maps at least three scales; wherein, the scale represents the downsampling factor of the fused feature map relative to the image to be detected, and different scales of fused feature maps correspond to different sizes of target detection ranges; and performing category prediction and bounding box position determination on the fused feature maps through the detection head of the target detection model to obtain the detection result of the underwater target. This application, based on a target detection model, fuses feature maps of different resolutions and performs predictions at multiple scales, enabling more precise identification of small targets, thereby improving the accuracy of underwater target detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to underwater target detection methods and related devices. Background Technology

[0002] In swimming pool cleaning and underwater inspection scenarios, accurate analysis of debris, structural defects, and environmental conditions at the pool bottom is typically required for target detection and recognition of acquired underwater images. This provides a basis for equipment path planning and operational decisions. Related technologies usually employ a single target detection model or deploy separate detection and segmentation models to process underwater images. However, due to limitations such as poor image quality caused by underwater environmental factors like light attenuation and water turbidity, extremely small targets, and target stacking at the bottom, as well as computational constraints, existing detection algorithms generally have limited ability to detect small targets. This can easily lead to missed detections of small targets and decreased detection accuracy, thus affecting the recognition accuracy of underwater equipment and overall operational efficiency. Summary of the Invention

[0003] In view of the above problems, this application provides an underwater target detection method and related apparatus to improve the detection recall and localization accuracy of extremely small underwater targets, while reducing the number of model parameters and computational complexity, facilitating real-time deployment on embedded devices such as pool cleaning robots. The specific solution is as follows:

[0004] The first aspect of this application provides an underwater target detection method, comprising:

[0005] The image to be detected is input into a pre-trained target detection model, and the backbone network of the target detection model is used to extract features from the image to be detected, resulting in multiple feature maps of different resolutions.

[0006] The neck network of the target detection model fuses multiple feature maps to obtain fused feature maps at least three scales; wherein, the scale represents the downsampling factor of the fused feature map relative to the image to be detected, and the fused feature maps of different scales correspond to different target detection ranges;

[0007] The detection head of the target detection model performs category prediction and bounding box position determination on the fused feature map to obtain the detection result of the underwater target.

[0008] Optionally, the step of extracting features from the image to be detected through the backbone network of the target detection model to obtain multiple feature maps of different resolutions includes:

[0009] The image to be detected is subjected to an initial convolutional transformation to obtain an initial feature map;

[0010] The initial feature map is subjected to multi-level convolution and downsampling processing through the backbone network to output at least four-stage feature maps; wherein the at least four-stage feature maps include a first-stage feature map, a second-stage feature map, a third-stage feature map, and a fourth-stage feature map; the resolution of the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map decreases sequentially.

[0011] Optionally, the plurality of feature maps with different resolutions include a first-stage feature map, a second-stage feature map, a third-stage feature map, and a fourth-stage feature map; the resolution of the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map decreases sequentially; the feature fusion of the plurality of feature maps through the neck network of the target detection model to obtain a fused feature map at least three scales includes:

[0012] The fourth-stage feature map is upsampled for the first time, and the first upsampling result is concatenated and fused with the third-stage feature map to obtain the first intermediate fused feature map;

[0013] The first intermediate fused feature map is upsampled a second time, and the result of the second upsampling is concatenated and fused with the second stage feature map to obtain the second intermediate fused feature map;

[0014] The second intermediate fusion feature map is upsampled a third time, and the result of the third upsampling is concatenated and fused with the first stage feature map to obtain the third intermediate fusion feature map;

[0015] After performing a convolutional transformation on the first stage feature map, it is concatenated and fused with the third intermediate fusion feature map to obtain the first scale fusion feature map;

[0016] The first scale fusion feature map is downsampled for the first time, and the first downsampling result is concatenated and fused with the second intermediate fusion feature map to obtain the second scale fusion feature map;

[0017] The second-scale fused feature map is downsampled a second time, and the second downsampling result is spliced ​​and fused with the first intermediate fused feature map to obtain the third-scale fused feature map; wherein the resolution of the first-scale fused feature map, the resolution of the second-scale fused feature map, and the resolution of the third-scale fused feature map decrease sequentially.

[0018] Optionally, the step of performing category prediction and bounding box location determination on the fused feature map using the detection head of the target detection model to obtain the detection result of the underwater target includes:

[0019] The detection head of the target detection model is configured with multiple detection branches; wherein each detection branch is associated with a specific fusion feature. Figure 1 One-to-one connection;

[0020] For each of the fused feature maps, a first convolution process is performed on the fused feature map through the classification prediction sub-branch corresponding to the fused feature map in each detection branch, so as to predict the target category of the candidate target in the fused feature map and obtain the target category result;

[0021] For each fused feature map, a sub-branch is determined by the bounding box corresponding to the fused feature map in each detection branch, and a second convolution process is performed on the fused feature map to determine the bounding box position parameters of the candidate targets in the fused feature map, thereby obtaining the target bounding box result;

[0022] Based on the target category results and the target bounding box results, the detection results of the underwater target are generated.

[0023] Optionally, before performing category prediction and bounding box location determination on the fused feature map using the detection head of the target detection model to obtain the detection result of the underwater target, the method further includes:

[0024] Obtain annotation information from multiple labeled samples in the model training data;

[0025] Cluster analysis is performed on the size of the annotation boxes of multiple annotation information to obtain a set of candidate anchor boxes corresponding to different scales;

[0026] The candidate anchor frame set is divided to obtain multiple sets of anchor frames corresponding to different scales;

[0027] Based on the multiple sets of anchor boxes, anchor boxes of corresponding scales are assigned to the detection branches corresponding to each fused feature map, so that each detection branch determines the bounding box position parameters of the candidate targets in the fused feature map corresponding to each detection branch based on the assigned anchor boxes.

[0028] Optionally, the target detection model is obtained through the following steps:

[0029] Raw videos under various underwater environmental conditions are collected, and image frames are extracted from the raw videos as raw image samples;

[0030] The initial detection model is used to infer the original image samples to generate pseudo-labels containing the target category and bounding box location;

[0031] The pseudo-labels are corrected to obtain labeled samples as model training data;

[0032] The target detection model is trained based on the model training data to obtain the pre-trained target detection model.

[0033] Optionally, training the target detection model based on the model training data to obtain the pre-trained target detection model includes:

[0034] The labeled samples of the model training data are subjected to sample augmentation processing to obtain augmented samples;

[0035] The enhanced samples are input into the target detection model for forward computation to obtain prediction results;

[0036] The loss is calculated based on the prediction results and the annotation information in the labeled samples to obtain classification loss and regression loss; wherein, the annotation information includes target category and bounding box location;

[0037] The parameters of the target detection model are updated based on the classification loss and the regression loss.

[0038] Repeat the forward computation, loss calculation, and parameter update until the preset training termination condition is met.

[0039] A second aspect of this application provides an underwater target detection device, comprising:

[0040] The feature extraction unit is used to input the image to be detected into the pre-trained target detection model, and extract features from the image to be detected through the backbone network of the target detection model to obtain multiple feature maps of different resolutions.

[0041] The feature fusion unit is used to fuse multiple feature maps through the neck network of the target detection model to obtain fused feature maps at least three scales; wherein, the scale represents the downsampling factor of the fused feature map relative to the image to be detected, and the fused feature maps of different scales correspond to different target detection ranges;

[0042] The detection output unit is used to perform category prediction and bounding box position determination on the fused feature map through the detection head of the target detection model to obtain the detection result of the underwater target.

[0043] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the underwater target detection method described in the first aspect or any implementation thereof.

[0044] A fourth aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein:

[0045] The memory is used to store computer programs;

[0046] The processor is used to execute the computer program so that the electronic device can implement the underwater target detection method of the first aspect or any implementation thereof.

[0047] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to perform the underwater target detection method described in the first aspect or any implementation thereof.

[0048] Based on the above technical solution, this application inputs the image to be detected into a target detection model comprising a backbone network, a neck network, and a detection head. The backbone network extracts multiple feature maps of different resolutions, enabling the model to simultaneously acquire both detailed and overall information from the image. Since feature maps of different resolutions correspond to different downsampling factors, they possess different receptive fields. High-resolution feature maps retain more details, which is beneficial for identifying smaller targets; low-resolution feature maps have a larger receptive field, which is beneficial for identifying larger targets.

[0049] Building upon this, the neck network fuses the feature maps at different resolutions, combining high-level semantic information with low-level detail information. This results in a fused feature map that possesses both structural and detailed information about the target. Since the fused feature map is set to multiple scales, each corresponding to a different target detection range, it allows for more targeted representation of targets of different sizes at different scales. This enhances the model's ability to distinguish between small targets and targets in complex backgrounds, thereby reducing the problem of false positives and false negatives caused by the difficulty in recognizing small targets.

[0050] Furthermore, by performing category prediction and bounding box location determination on the fused feature maps at different scales using the detection head, target classification and localization can be completed simultaneously at different scales. Since the fused feature maps at different scales are used to detect targets of different sizes, small targets can be detected more precisely on high-resolution feature maps, while large targets can be detected as a whole on low-resolution feature maps. This improves the overall detection accuracy, reduces missed detections and false detections, and ultimately enhances the reliability of underwater target identification. Attached Figure Description

[0051] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0052] Figure 1 This is a schematic flowchart of an underwater target detection method provided in an embodiment of this application;

[0053] Figure 2 This is a schematic diagram of another underwater target detection method provided in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of the structure of an underwater target detection device provided in an embodiment of this application;

[0055] Figure 4 This is a schematic diagram of a computer device structure provided in an embodiment of this application. Detailed Implementation

[0056] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments and is not intended to limit the application. Those skilled in the art will understand that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems. The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be interchanged where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0057] With the development of intelligent robot technology, underwater robots and other equipment are increasingly being applied to underwater maintenance scenarios. Among these applications, vision-based target detection technology is widely used to identify debris at the bottom of pools and to assist in path planning and cleaning decisions. Target detection technology typically relies on deep learning models to analyze acquired underwater images to achieve automatic detection and localization of different types of targets. For visual detection of debris at the bottom of pools, existing technologies mainly employ two methods. One is based on traditional image processing, which extracts manually designed features such as color, texture, and edges, and combines this with a classifier to discriminate candidate regions in the image, distinguishing debris from the pool bottom background. The other is a deep learning-based target detection method, which utilizes a pre-built image dataset containing multiple debris categories to train lightweight convolutional neural network models such as the YOLO series and SSD. The trained model is then deployed to an embedded computing platform on the robot to achieve real-time inference and target localization of the input image. However, in the complex underwater environment at the bottom of a swimming pool, the small size of the target to be detected, the significant impact of water turbidity and lighting changes on the image, and the insufficient utilization of high-resolution detail features by existing detection algorithm models lead to a decrease in the information expression ability of small targets during feature extraction and fusion, thus affecting the accuracy of the detection results.

[0058] To address the aforementioned problems, this application provides an underwater target detection method. The underwater target detection method of this application embodiment will be described in detail below with reference to the accompanying drawings.

[0059] Reference Figure 1 , Figure 1 This is a flowchart illustrating an underwater target detection method provided in an embodiment of this application, as shown below. Figure 1 As shown in the embodiment of this application, an underwater target detection method may include steps 101 to 103, which are described in detail below.

[0060] 101. Input the image to be detected into the pre-trained target detection model, and extract features from the image to be detected through the backbone network of the target detection model to obtain multiple feature maps of different resolutions.

[0061] The target detection model in this application embodiment comprises three sequentially connected functional modules: a backbone network, a neck network, and a detection head. The backbone network is used to extract multi-scale feature maps from the input image, specifically employing a lightweight convolutional neural network structure based on CSPDarknet26. The image to be detected is typically an RGB image captured by an underwater camera, or it can be preprocessed image data. Before being input into the target detection model, the image data can be adjusted to a uniform-sized three-channel image tensor to meet the model's input requirements. For example, it can be processed into image data with a fixed spatial resolution (e.g., 640×640 pixels) and input into the backbone network in tensor form. After the image is input, the backbone network first performs a preliminary convolutional transformation on the input image through an initial convolutional layer. By introducing downsampling operations (e.g., stride convolution or pooling operations) at different levels, the feature map gradually decreases in spatial resolution while gradually increasing in channel dimension, thereby forming multiple feature maps with different resolutions. Here, feature maps of different resolutions refer to feature representations that have different downsampling factors in spatial size relative to the input image, such as corresponding to 1 / 4, 1 / 8, 1 / 16 and 1 / 32 scales of the input image, so that each feature map has a receptive field of different size.

[0062] 102. By fusing features from multiple feature maps through the neck network of the target detection model, at least three scales of fused feature maps are obtained; where scale represents the downsampling factor of the fused feature map relative to the image to be detected, and different scales of fused feature maps correspond to different sizes of target detection ranges.

[0063] The neck network in this embodiment employs an improved PANet (Path Aggregation Network) structure, specifically including a top-down feature fusion path and a bottom-up feature aggregation path. In the top-down path, the neck network starts with the deepest feature map corresponding to high-level semantic features (i.e., the feature map with the largest downsampling factor), progressively increasing its resolution through multiple upsampling operations, and then sequentially concatenating and fusing it with feature maps of corresponding levels in the backbone network, thereby transferring deep semantic information to shallow high-resolution feature maps. In the bottom-up path, the neck network starts with shallow fused features, progressively aggregating them through downsampling operations, further enhancing the localization capability of feature maps at each scale. After the above fusion processing, the neck network outputs fused feature maps at at least three scales, each corresponding to a specific downsampling factor, such as 4x, 8x, and 16x downsampling, thus being suitable for different target detection ranges. In this context, scale refers to the downsampling factor of the fused feature map relative to the image to be detected. The smaller the downsampling factor, the higher the resolution of the feature map, which is more conducive to detecting smaller targets. Conversely, the larger the downsampling factor, the lower the resolution of the feature map, but the larger the receptive field, which is more conducive to detecting larger targets.

[0064] 103. The detection head of the target detection model is used to predict the category and determine the bounding box position of the fused feature map to obtain the detection results of underwater targets.

[0065] The detection head in this embodiment includes multiple parallel detection branches, each corresponding to a fused feature map at a specific scale. Each detection branch employs a decoupled structure, containing independent classification prediction sub-branch and bounding box determination sub-branch. The classification prediction sub-branch uses several convolutional layers to predict the class probability of each candidate location in the fused feature map, outputting the confidence score of the target belonging to each class. The bounding box determination sub-branch uses another set of convolutional layers to predict the center point coordinates, width, and height of the target's bounding box. Subsequently, the final detection result is generated based on the class prediction result and the bounding box position parameters. This detection result indicates the class label of underwater targets (such as leaves, pebbles, and other debris) and their positional distribution in the image, thus completing the entire underwater target detection process.

[0066] Based on the above technical solution, this application inputs the image to be detected into a target detection model comprising a backbone network, a neck network, and a detection head. The backbone network extracts multiple feature maps of different resolutions, enabling the model to simultaneously acquire both detailed and overall information from the image. Since feature maps of different resolutions correspond to different downsampling factors, they possess different receptive fields. High-resolution feature maps retain more details, which is beneficial for recognizing smaller targets; low-resolution feature maps have a larger receptive field, which is beneficial for recognizing larger targets. Furthermore, by fusing these feature maps of different resolutions through the neck network, high-level semantic information can be combined with low-level detailed information, resulting in a fused feature map that simultaneously possesses both structural and detailed information about the target. Because the fused feature map is set to multiple scales, each corresponding to a different target detection range, it allows for more targeted representation of targets of different sizes at different scales, enhancing the model's ability to distinguish between small targets and targets in complex backgrounds, thereby reducing the problem of false positives and false negatives caused by the difficulty in recognizing small targets. Further, by using the detection head to perform category prediction and bounding box location determination on the fused feature maps at each scale, target classification and localization can be completed simultaneously at different scales. Since fused feature maps of different scales are used to detect targets of different sizes, small targets can be detected more precisely on high-resolution feature maps, while large targets can be detected as a whole on low-resolution feature maps. This improves the overall detection accuracy, reduces missed detections and false detections, and thus enhances the reliability of underwater target identification.

[0067] It should be understood that the target detection model used in the embodiments of this application is an improved design based on the YOLO framework; for details, please refer to [link / reference needed]. Figure 2The model is composed of three main parts: the backbone, the neck, and the head. The backbone uses a lightweight CSPDarknet26 architecture, and within its C3 module, depthwise separable convolutions replace some standard 3×3 convolutions, significantly reducing the number of parameters while maintaining feature extraction capabilities. The backbone contains four stages, outputting four different resolution feature maps: Stage 1 outputs a 160×160 first-stage feature map (P2, downsampling factor 4), Stage 2 outputs an 80×80 second-stage feature map (P3, downsampling factor 8), Stage 3 outputs a 40×40 third-stage feature map (P4, downsampling factor 16), and Stage 4 outputs a 20×20 fourth-stage feature map (P5, downsampling factor 32). Notably, the high-resolution P2 feature map from Stage 1 is retained to preserve detailed information about extremely small targets at the bottom of the pool. In this embodiment, the C3k2 module is mainly used as the core feature extraction unit in the backbone and neck network. The C3k2 module is mainly constructed based on the Cross-Stage Partial Network (CSPNet). The processing flow of the C3k2 module is as follows: the input features first undergo convolutional transformation to complete channel adjustment and preliminary feature extraction, and then are divided into two feature branches in the channel dimension through a splitting operation. One feature branch undergoes feature transformation through the bottleneck structure (Bottleneck module) and the attention enhancement module (PSABlock) in sequence to extract deeper semantic information. When the configuration parameter Attn=True, the attention enhancement module is activated to weight the features in the spatial and channel dimensions, thereby highlighting the features of the target region and suppressing background interference. The other branch acts as a shortcut to directly pass the original or shallow features to retain detailed information and enhance gradient propagation ability. Subsequently, the two feature branches are fused through a concatenation operation and integrated and output through a convolutional layer. Through the above structural design, the C3k2 module can effectively fuse semantic and detailed information while controlling computational complexity, thereby improving feature representation capabilities, especially suitable for small target detection scenarios in complex underwater environments. Furthermore, the backbone network in this embodiment is equipped with an SPPF (Spatial Pyramid Pooling Fast) module to further expand the receptive field of features and enhance multi-scale information representation capabilities. The SPPF module's processing flow is as follows: input features are first compressed and channel-adjusted through convolutional layers, followed by multiple pooling operations (such as max pooling). Feature responses are extracted at different scales through continuously stacked pooling layers to obtain a wider range of contextual information.The features obtained from each pooling layer are fused with the original features through a concatenation operation to form a feature representation containing multi-scale receptive field information. Then, convolutional layers integrate the concatenated features, and residual connection structures are used to stack the input features for output. Thus, this SPPF module can improve the network's ability to perceive large-scale targets and complex backgrounds without significantly increasing computational overhead, providing richer contextual information support for subsequent feature fusion and object detection.

[0068] The neck network portion of this application embodiment is mainly improved based on the classic PANet structure, adding a feature fusion path at layer P2 to achieve the transfer of high-level semantic features to high-resolution features. Specifically, in the top-down path, the fourth-stage feature map is upsampled three times in sequence. After each upsampling, it is concatenated and fused with the third, second, and first-stage feature maps to obtain the first, second, and third intermediate fused feature maps. Simultaneously, the first-stage feature map is convolved and fused again with the third intermediate fused feature map to generate the first-scale fused feature map (S1, 160×160), which is specifically used for the detection of extremely small targets. Subsequently, through the bottom-up aggregation path, the first-scale fused feature map is downsampled twice in sequence and concatenated and fused with the second intermediate and first intermediate fused feature maps to generate the second-scale fused feature map (S2, 80×80) and the third-scale fused feature map (S3, 40×40), which are used for the detection of small-to-medium-sized targets and medium-sized targets, respectively.

[0069] In the detection head section, a multi-scale decoupling design is adopted, including three parallel detection branches, P2, P3, and P4, which are respectively connected to the first, second, and third scale fusion feature maps. Each detection head contains an independent classification prediction sub-branch and a bounding box determination sub-branch, and outputs target category information and bounding box position parameters through convolution operations. To further reduce computational overhead and improve deployment efficiency on edge devices, the convolutional structures in each detection branch are also constructed using depthwise separable convolutions. Through the above structural design, this embodiment significantly reduces the number of parameters and computational load while improving the detection capability of complex small targets at the bottom of swimming pools, making it easy to deploy on embedded devices.

[0070] In one possible implementation, the backbone network of the object detection model extracts features from the image to be detected, resulting in multiple feature maps of different resolutions. This includes: performing an initial convolutional transformation on the image to be detected to obtain an initial feature map; and performing multi-level convolutional and downsampling processing on the initial feature map through the backbone network to output at least four-stage feature maps. The at least four-stage feature maps include a first-stage feature map, a second-stage feature map, a third-stage feature map, and a fourth-stage feature map. The resolutions of the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map decrease sequentially.

[0071] First, it should be noted that the backbone network of this embodiment adopts the CSPDarknet26 architecture, and the input end of the backbone network includes a Focus module. The Focus module is used to perform slicing operations on the image to be detected, for example, sampling the input image with a resolution of 640×640×3 at pixel intervals to obtain a 320×320×12 feature map; then, a standard convolutional layer is used to compress the number of channels, thereby generating the initial feature map. The initial convolutional transformation can effectively reduce the amount of subsequent computation without losing spatial information.

[0072] The backbone network then performs multi-level convolution and downsampling on the initial feature map to output at least four stages of feature maps. Specifically, the CSPDarknet26 architecture can contain four sequentially connected stages, each stage consisting of multiple C3 modules. The C3 modules adopt the CSPNet structure, dividing the input feature map into two parts by channel. One part undergoes feature transformation through multiple Bottleneck blocks composed of depthwise separable convolutions, while the other part is directly connected across channels. Finally, the two parts are concatenated and output via convolution. Depthwise separable convolution decomposes the standard 3×3 convolution into depthwise convolution and pointwise convolution, thus requiring only about 11.9% of the parameters of standard convolution. For example, when the number of input channels... and number of output channels When both are 128, the standard convolution parameter count is: That is, 147456, while the number of parameters for depthwise separable convolution is That is, 17536. Through the above multi-stage processing, the backbone network can output at least four stage feature maps, namely the first stage feature map, the second stage feature map, the third stage feature map, and the fourth stage feature map. The resolution of the first stage feature map, the second stage feature map, the third stage feature map, and the fourth stage feature map decreases sequentially. As the network layer deepens, the spatial size of the feature map decreases step by step, while the corresponding receptive field increases step by step.

[0073] To illustrate with a specific example, in the CSPDarknet26-based implementation, Stage 1 outputs a first-stage feature map with a size of 160×160 and 128 channels; Stage 2 outputs a second-stage feature map with a size of 80×80 and 256 channels; Stage 3 outputs a third-stage feature map with a size of 40×40 and 512 channels; and Stage 4 outputs a fourth-stage feature map with a size of 20×20 and 1024 channels. Thus, the first-stage feature map focuses more on preserving local details and edge information of the target, suitable for representing smaller targets; while the fourth-stage feature map has stronger high-level semantic expression capabilities, suitable for representing larger targets or targets with more complex semantic features. The resolutions of the first, second, third, and fourth-stage feature maps correspond to downsampling of the original image by 4x, 8x, 16x, and 32x, respectively, forming a pyramid-shaped feature hierarchy from high-resolution details to low-resolution semantics.

[0074] In one possible implementation, multiple feature maps of different resolutions include a first-stage feature map, a second-stage feature map, a third-stage feature map, and a fourth-stage feature map; the resolutions of the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map decrease sequentially; feature fusion is performed on the multiple feature maps through the neck network of the object detection model to obtain a fused feature map of at least three scales, including: performing top-down feature fusion processing on the multiple feature maps of different resolutions, by upsampling at each level and concatenating them with the feature maps of adjacent stages to obtain a multi-level intermediate fused feature map; and performing bottom-up feature fusion processing based on the multi-level intermediate fused feature map, by downsampling at each level and concatenating them with the corresponding intermediate fused feature map to obtain a fused feature map of at least three scales.

[0075] This application embodiment performs top-down feature fusion processing on multiple feature maps of different resolutions output by the backbone network: starting with the feature map with the lowest resolution (i.e., the largest downsampling factor), its resolution is gradually increased through upsampling at each level. At each level, the upsampling result is concatenated and fused with the adjacent feature map of the corresponding layer in the backbone network with a higher resolution, thereby transferring deep semantic information layer by layer to the shallow high-resolution features. This generates a series of intermediate fused feature maps with progressively increasing resolution, enabling low-level features to enhance semantic expressiveness while preserving spatial detail information. Subsequently, based on the multi-level intermediate fused feature maps obtained from the top-down approach, bottom-up feature fusion processing is performed: starting with the intermediate fused feature map with the highest resolution, its resolution is reduced through downsampling at each level. At each level, the downsampling result is concatenated and fused again with the intermediate fused feature map of the corresponding layer in the top-down path, thereby further enhancing the localization ability of feature maps at each scale, allowing high-level features to supplement detailed information while maintaining overall semantic information. After the bidirectional fusion described above, the neck network finally outputs at least three fused feature maps with different resolutions. Each fused feature map corresponds to a specific downsampling factor and is used to detect underwater targets of different sizes. This effectively takes into account the spatial details of extremely small targets and the semantic information of medium and large targets. It can obtain feature representations that contain both semantic and detailed information at different scales, thereby improving the overall performance of multi-scale target detection.

[0076] For the aforementioned top-down and bottom-up feature fusion process, in one specific implementation, it can be achieved through the following steps: First, upsample the fourth-stage feature map, and then concatenate and fuse the first upsampled result with the third-stage feature map to obtain a first intermediate fused feature map; second, upsample the first intermediate fused feature map, and then concatenate and fuse the second upsampled result with the second-stage feature map to obtain a second intermediate fused feature map; third, upsample the second intermediate fused feature map, and then concatenate and fuse the third upsampled result with the first-stage feature map to obtain a third intermediate fused feature map. After performing a convolutional transformation on the first-stage feature map, it is concatenated and fused with the third intermediate fusion feature map to obtain the first-scale fusion feature map. The first-scale fusion feature map is downsampled for the first time, and the result of the first downsampling is concatenated and fused with the second intermediate fusion feature map to obtain the second-scale fusion feature map. The second-scale fusion feature map is downsampled for the second time, and the result of the second downsampling is concatenated and fused with the first intermediate fusion feature map to obtain the third-scale fusion feature map. The resolution of the first-scale fusion feature map, the second-scale fusion feature map, and the third-scale fusion feature map decreases sequentially.

[0077] In one specific implementation, a top-down first fusion operation is performed: the fourth-stage feature map is upsampled for the first time. Upsampling typically uses bilinear interpolation to double the resolution of the feature map, for example, enlarging a 20×20 fourth-stage feature map to 40×40. Then, the upsampled result is concatenated and fused with the third-stage feature map (40×40 resolution) along the channel dimension, i.e., the two feature maps are stacked together along the channel direction. A C3 module then performs convolutional transformation and feature extraction on the concatenated features to obtain the first intermediate fused feature map. This first intermediate fused feature map has the same resolution as the third-stage feature map, 40×40, but incorporates the deeper semantic information of the fourth-stage feature map.

[0078] Next, the first intermediate fused feature map is upsampled a second time, doubling its resolution, for example, from 40×40 to 80×80. The result of this second upsampling is then concatenated and fused with the second-stage feature map (80×80 resolution), and feature extraction is performed using the C3 module to obtain the second intermediate fused feature map. This second intermediate fused feature map has the same resolution as the second-stage feature map, 80×80, and now contains deep semantic information from the fourth and third stages, achieving preliminary fusion with the spatial detail information of the second stage.

[0079] Subsequently, the second intermediate fused feature map is upsampled a third time, doubling its resolution again, for example, from 80×80 to 160×160. The result of this third upsampling is then concatenated with the first-stage feature map (160×160 resolution) and processed by module C3 to obtain the third intermediate fused feature map. This third intermediate fused feature map has the same resolution as the first-stage feature map, 160×160. This completes all the fusion operations along the top-down path, enabling the high-resolution P2 layer feature map to acquire semantic information from deeper layers (P5, P4, P3).

[0080] After obtaining the third intermediate fused feature map, the first-stage feature map is subjected to a separate convolutional transformation. Specifically, a CBS module (containing convolutional layers, batch normalization layers, and the SiLU activation function) is used to adjust its channel count so that it matches the channel count of the first-stage feature map with that of the third intermediate fused feature map. Then, the convolutionally transformed first-stage feature map and the third intermediate fused feature map are concatenated and fused again, and processed by the C3 module to obtain the first-scale fused feature map. This first-scale fused feature map retains a resolution of 160×160, but also contains high-resolution detail information from the original first-stage feature map and deep semantic information passed down through a top-down path. Therefore, it can be specifically used to detect extremely small targets with an area less than or equal to 32×32 pixels.

[0081] Then, based on the first-scale fused feature map, a bottom-up aggregation operation is performed. First, the first-scale fused feature map is downsampled. Downsampling typically uses a convolution operation with a stride of 2 to reduce the resolution by half, for example, downsampling from 160×160 to 80×80. Then, the first downsampling result is concatenated and fused with the second intermediate fused feature map (resolution 80×80), and processed by the C3 module to obtain the second-scale fused feature map. This second-scale fused feature map has a resolution of 80×80 and can be used to detect small to medium-sized targets with areas between 32×32 pixels and 96×96 pixels.

[0082] Finally, the second-scale fused feature map is downsampled a second time, reducing its resolution by half again, for example, from 80×80 to 40×40. The result of this second downsampling is then concatenated with the first intermediate fused feature map (40×40 resolution) and processed by the C3 module to obtain the third-scale fused feature map. This third-scale fused feature map has a resolution of 40×40 and can be used to detect medium-sized targets with an area greater than or equal to 96×96 pixels. Thus, through the above top-down and bottom-up bidirectional fusion, three scale fused feature maps with progressively decreasing resolutions (160×160, 80×80, 40×40) are obtained, corresponding to the detection of extremely small targets, small to medium-sized targets, and medium-sized targets, respectively.

[0083] In one possible implementation, the detection head of the target detection model performs category prediction and bounding box location determination on the fused feature map to obtain the detection result of the underwater target. This includes setting multiple detection branches in the detection head of the target detection model; wherein each detection branch is associated with a different fused feature map. Figure 1One-to-one connection; for each fused feature map, the first convolution process is performed on the fused feature map through the classification prediction sub-branch corresponding to the fused feature map in each detection branch to predict the target category of the candidate target in the fused feature map, and obtain the target category result; for each fused feature map, the second convolution process is performed on the fused feature map through the bounding box determination sub-branch corresponding to the fused feature map in each detection branch to determine the bounding box position parameters of the candidate target in the fused feature map, and obtain the target bounding box result; based on the target category result and the target bounding box result, the detection result of the underwater target is generated.

[0084] In this embodiment, multiple detection branches are set in the detection head of the target detection model, and each detection branch is associated with a different fused feature. Figure 1 The one-to-one correspondence between the fusion feature maps and the detection branches is primarily due to the fact that different scales of the fused feature maps correspond to different downsampling factors and target detection regions. Therefore, a dedicated detection branch needs to be configured for each scale of the fused feature map to ensure that each scale feature can independently complete target prediction within its corresponding detection space. The detection head can include three parallel detection branches, connected to the first-scale fused feature map, the second-scale fused feature map, and the third-scale fused feature map, respectively, i.e., connected to feature maps P2, P3, and P4. The detection branch corresponding to the first-scale fused feature map can be used to detect extremely small targets, the detection branch corresponding to the second-scale fused feature map can be used to detect small to medium-sized targets, and the detection branch corresponding to the third-scale fused feature map can be used to detect medium or large targets. This one-to-one correspondence between the fused feature maps and the detection branches allows each detection branch to process candidate target information matching its scale, thereby avoiding interference between targets of different scales in the same detection path.

[0085] For each fused feature map, a first convolution process is performed on the fused feature map through the classification prediction sub-branch in the corresponding detection branch to predict the target category of the candidate objects. The classification prediction sub-branch is used to determine the category of local regions in the fused feature map and output the prediction result of the region belonging to each target category. The first convolution process here refers to performing a convolution operation on the corresponding fused feature map in the classification prediction sub-branch to obtain the feature response related to the target category. The classification prediction sub-branch is specifically responsible for target category prediction, so that category judgment and bounding box regression are performed independently. Since the classification task mainly focuses on what the candidate target is, this sub-branch can learn category distinguishing features more focused on, thereby improving the ability to identify the category of leaves, stones or other targets in underwater scenes. Specifically, the classification prediction sub-branch is usually composed of two depthwise separable convolutional layers and a 1×1 convolutional layer connected in sequence. After inputting the fused feature map, the number of output channels is equal to the number of target categories multiplied by the number of anchor boxes preset by the detection branch. Each output value represents the probability that the corresponding anchor box position belongs to each category.

[0086] Simultaneously, for each fused feature map, a second convolution process is performed on the fused feature map through the bounding box determination sub-branch within the same detection branch to determine the bounding box position parameters of the candidate target. Here, the bounding box determination sub-branch primarily learns the spatial distribution information of the candidate target in the image. The second convolution process refers to performing convolution operations on the fused feature map within the bounding box determination sub-branch to obtain the feature representation used for bounding box regression. The structure of the bounding box determination sub-branch is similar to that of the classification prediction sub-branch, consisting of two depthwise separable convolutional layers and one 1×1 convolutional layer. Its output channel number is equal to 4 times the number of anchor boxes, where 4 corresponds to the x-coordinate, y-coordinate, width, and height of the bounding box's center point. The output of the bounding box determination sub-branch is the offset relative to the anchor boxes. After decoding, the actual bounding box position of the candidate target in the original image can be obtained, thus yielding the target bounding box result. By separating bounding box regression from classification prediction, the bounding box determination sub-branch can focus on learning the target's localization features, thereby improving the inaccurate localization problem of small underwater targets caused by edge blurring and target stacking.

[0087] After obtaining the target category and target bounding box results, the final detection result for underwater targets is generated based on these results. In the specific implementation, three parallel detection branches perform detection tasks on feature maps of different scales, so the final output detection result is a set of prediction results from multiple scale detection branches. By using multi-scale parallel detection, extremely small, medium, and medium-sized targets can be detected on feature maps of more suitable scales, thereby improving the overall detection effect for targets of different sizes.

[0088] In one possible implementation, before performing category prediction and bounding box position determination on the fused feature map using the detection head of the target detection model to obtain the detection result of the underwater target, the method of this application embodiment further includes: acquiring annotation information from multiple labeled samples in the model training data; performing cluster analysis on the size of the labeled boxes of the multiple labeled information to obtain a set of candidate anchor boxes corresponding to different scales; dividing the set of candidate anchor boxes to obtain multiple sets of anchor boxes corresponding to different scales; and assigning the anchor boxes of the corresponding scales to the detection branches corresponding to each fused feature map according to the multiple sets of anchor boxes, so that each detection branch determines the bounding box position parameters of the candidate targets in the fused feature map corresponding to each detection branch based on the assigned anchor boxes.

[0089] Annotated samples refer to sample data in the model training dataset that have completed target location annotation. Annotation information refers to the target category and bounding box position data corresponding to each labeled sample. The bounding box position includes at least the size information of the bounding box, specifically the bounding box coordinates (center x, center y, width, height) of all targets in each sample image.

[0090] Since the size distribution of underwater targets in real-world scenarios is scene-dependent, especially with a high proportion of extremely small targets, directly utilizing the annotation information from the training data can more accurately reflect the target scale characteristics. Then, cluster analysis is performed on the bounding box dimensions (i.e., width and height) from multiple annotation sets to obtain candidate anchor boxes corresponding to different target scales. Specifically, the K-means clustering method can be used to cluster the bounding box dimensions in the model training dataset (such as a self-built underwater dataset) to obtain candidate anchor boxes adapted to different detection scales. Because the candidate anchor boxes are derived from statistical results of real training data, they are more closely aligned with the underwater target scale characteristics targeted in this application compared to using a general anchor box configuration. Next, the clustered candidate anchor box set is further divided to obtain multiple groups of anchor boxes corresponding to different scales. The division is based on the size range of the anchor boxes, dividing the clustered anchor boxes into a first group, a second group, and a third group. The first group corresponds to the extremely small target scale, the second group corresponds to the small to medium target scale, and the third group corresponds to the medium target scale. This grouping method allows different detection branches to use reference anchor boxes that are more suitable for the current target scale when determining the bounding box position parameters.

[0091] Finally, based on the aforementioned multiple sets of anchor boxes, the anchor boxes of corresponding scales are assigned to the detection branches corresponding to each fused feature map. This allows each detection branch to determine the bounding box position parameters of the candidate targets in the fused feature map corresponding to each detection branch based on the assigned anchor boxes. Bounding box regression typically uses preset anchor boxes as a reference to predict the offset and size change of the target relative to the anchor boxes. Therefore, the anchor box size used by each detection branch directly affects the regression accuracy of the bounding box position parameters. Specifically, the first set of anchor boxes can be assigned to the detection branch corresponding to the first-scale fused feature map, the second set to the detection branch corresponding to the second-scale fused feature map, and the third set to the detection branch corresponding to the third-scale fused feature map. For example, the first-scale detection branch can correspond to the P2 detection head, whose anchor frame size can be 3, 6, 9, 12, 15, or 18 pixels, used to detect extremely small targets with an area smaller than 32×32 pixels; the second-scale detection branch can correspond to the P3 detection head, whose anchor frame size can be 10, 14, 16, 22, 28, or 36 pixels; and the third-scale detection branch can correspond to the P4 detection head, whose anchor frame size can be 30, 45, 60, 80, 100, or 120 pixels. Through this anchor frame allocation method, each detection branch can perform more accurate bounding box position parameter prediction within its respective target size range, thereby improving the accuracy and convergence speed of bounding box regression.

[0092] In one possible implementation, the object detection model is trained through the following steps: acquiring raw videos under various underwater environmental conditions and extracting image frames from the raw videos as raw image samples; using the initial detection model to infer from the raw image samples to generate pseudo-labels containing the object category and bounding box location; correcting the pseudo-labels to obtain labeled samples as model training data; and training the object detection model based on the model training data to obtain a pre-trained object detection model.

[0093] In this embodiment, the target detection model constructs training data by acquiring raw videos under various underwater environmental conditions and extracting image frames from the raw videos as raw image samples. Specifically, a high-definition underwater camera deployed at the bottom of a swimming pool acquires raw videos of the pool bottom under different light intensities, water turbidity, and target densities. Light intensity is categorized into strong light scenes (pool surface light intensity greater than or equal to 1000 lux), weak light scenes (200 to 500 lux), and dark light scenes (less than 50 lux); water turbidity is categorized into clear water (turbidity value less than 1 NTU), slightly turbid (1 to 5 NTU), and heavily turbid (5 to 10 NTU); target density is categorized into sparse scenes (0 to 2 targets per image), medium-density scenes (3 to 5 targets), and dense scenes (6 or more targets). Image frames are extracted from the acquired raw videos at fixed frame intervals to form a raw image sample set.

[0094] Then, the initial detection model is used to infer from the original image samples, generating pseudo-labels containing target categories and bounding box locations. By automatically detecting unlabeled images using an existing initial detection model, a batch of initial annotation results can be quickly obtained for subsequent correction, thus reducing the workload of entirely manual annotation. Here, the initial detection model refers to a target detection model pre-trained or pre-built on other datasets. The initial detection model can perform preliminary identification of targets in the original image samples and output the category information and bounding box location parameters for each target. The initial detection model can be a YOLOv8 model pre-trained on a large-scale general-purpose target detection dataset. The original image samples are input into this pre-trained model, and the model automatically outputs the detected target category and the corresponding coarse bounding box coordinates for each image; these output results are the pseudo-labels.

[0095] Because the initial detection model used has good generalization ability on general datasets, it can initially identify common targets in images (such as leaves and pebbles). However, due to the special characteristics of the underwater environment, the detection results output by the initial detection model may have missed detections, false detections, or localization errors. Therefore, it is necessary to correct the pseudo-labels to obtain labeled samples as training data for the model. The correction process includes the following operations: deleting pseudo-labels corresponding to falsely detected targets (false detections refer to target information that does not exist in the pseudo-labels but is incorrectly labeled by the model); supplementing the bounding boxes corresponding to missed targets, that is, adding bounding boxes to extremely small targets (such as stone fragments with an area of ​​less than 16×16 pixels) that the initial detection model failed to detect; adjusting the bounding boxes corresponding to pseudo-labels with positional deviations exceeding a preset range (e.g., the IoU between the bounding box and the edge of the real target is less than 0.5) to closely fit the real edge of the target. For stacked or partially occluded targets, only their visible areas are labeled, and the occluded parts are not inferred. Then, the corrected pseudo-labels are received, and the target category and bounding box location information corresponding to each image in the corrected pseudo-labels are written into a txt text label file with the same name as the image in YOLO format. Each line of content is in the format of "category ID, normalized center x, normalized center y, normalized width, normalized height", thus forming a complete labeled sample.

[0096] Finally, the object detection model is trained based on the labeled samples obtained above to obtain a pre-trained object detection model. During training, the labeled dataset is divided into training and validation sets according to a preset ratio (e.g., 8:2 or 7:3), and corresponding configuration files (such as YAML format files) are created, which record the image paths, category name lists, and training hyperparameters of the training and validation sets. The model is iteratively trained using the training set. In each batch, the classification loss and regression loss between the model's prediction results and the labeled information are calculated, and the model parameters are updated using the backpropagation algorithm. After each training epoch, the model performance is evaluated on the validation set, and the model weights with the best validation metrics are saved as the final pre-trained object detection model.

[0097] In one possible implementation, training the object detection model based on model training data to obtain a pre-trained object detection model includes: performing sample augmentation on the labeled samples of the model training data to obtain augmented samples; inputting the augmented samples into the object detection model for forward computation to obtain prediction results; calculating the loss based on the prediction results and the annotation information in the labeled samples to obtain classification loss and regression loss; wherein, the annotation information includes the object category and bounding box position; updating the parameters of the object detection model based on the classification loss and regression loss; and repeating the forward computation, loss calculation, and parameter update until a preset training termination condition is reached.

[0098] The purpose of augmenting labeled samples in the model training data is to expand the diversity of the training data and improve the model's generalization ability. Specifically, this application employs various data augmentation techniques, including Mosaic augmentation (randomly scaling, cropping, and stitching four images into a new image), MixUp augmentation (linearly mixing two images at a certain ratio), random flipping (horizontal or vertical flipping), and random color jittering (randomly adjusting brightness, contrast, saturation, and hue). For example, for an image of the bottom of a swimming pool, Mosaic augmentation can combine targets under different lighting conditions and turbidity into the same image, allowing the model to learn richer scene variations; random color jittering can simulate real-time fluctuations in underwater lighting. After the above augmentation processing, an augmented sample set is obtained for subsequent training.

[0099] Then, the enhanced samples are input into the object detection model for forward computation to obtain prediction results. The enhanced image samples pass sequentially through the backbone network, neck network, and detection head of the object detection model. The model predicts the category and determines the bounding box position of the objects in the image according to the current parameters, and outputs the category confidence and bounding box coordinate offset for each candidate object. For example, for an image of the bottom of a swimming pool containing leaves and pebbles, the model's forward computation will output multiple predicted boxes, each accompanied by the probability that it belongs to the category of leaves or pebbles, as well as the center coordinates, width, and height of the bounding box.

[0100] Next, loss calculations are performed based on the prediction results and the annotation information in the labeled samples to obtain classification loss and regression loss; the annotation information includes the target category and bounding box location. In this embodiment, the classification loss uses Focal Loss, and its calculation formula is as follows: ,in =2; =0.25. The classification loss function reduces the weight of easily classified background samples, allowing the model to focus on small target samples that are difficult to classify, thus alleviating the problem of extreme imbalance between targets and background in pool bottom images. The regression loss uses CIoU Loss, and its calculation formula is... ,in This is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box. This is the Euclidean distance between the center point of the predicted bounding box and the center point of the ground truth bounding box. Indicates the center coordinates of the prediction box; Represents the center coordinates of the true bounding box; It is the diagonal length of the smallest closure region that simultaneously contains both the predicted bounding box and the ground truth bounding box; A parameter used to measure aspect ratio consistency; These are the preset weighting coefficients. Then, by comparing the prediction results with the labeled information, the classification loss and regression loss values ​​are calculated separately, and the two are added together to obtain the total loss.

[0101] Based on this, the parameters of the object detection model are updated using classification loss and regression loss. Specifically, the AdamW optimizer can be used for parameter optimization, with an initial learning rate set to 0.01 and a cosine annealing learning rate scheduling strategy employed. This allows the learning rate to decay periodically according to a cosine function during training, helping to escape local optima and accelerate convergence. The batch size for each training batch can be set to 32, meaning that 32 augmented samples are processed simultaneously for each forward computation and loss calculation. Based on the calculated total loss, the gradients of the parameters of each layer of the model are calculated using the backpropagation algorithm, and then the AdamW optimizer is used to update the parameters to reduce the difference between the predicted results and the true labels. The above forward computation, loss calculation, and parameter update steps are repeated until the preset training termination condition is reached. In this embodiment, the preset training termination condition can be set to 300 training epochs. Each training round iterates through the entire training set once, and after each round, the model performance is evaluated on the validation set. Metrics such as mAP@0.5 (mean accuracy when the intersection-over-union threshold is 0.5) are calculated. The weights of the best-performing model on the validation set are saved as the final pre-trained object detection model. If the validation set metrics fail to improve for several consecutive rounds during training, training can be terminated early to prevent overfitting.

[0102] In the embodiments of this application, experimental results on a self-built pool bottom dataset demonstrate that the target detection model proposed in this application (also known as the YOLO26-p2 model) achieves significantly better detection performance than existing YOLO series models. Specifically, the mean average precision (mAP) of YOLOv5s is 84.6%, YOLOv8n is 86.3%, and YOLOv11n is 88.1%, while the mAP of YOLO26-p2 in this application reaches 92.8%, representing an improvement of 4.7 percentage points compared to YOLOv11n, 6.5 percentage points compared to YOLOv8n, and 8.2 percentage points compared to YOLOv5s. Particularly for extremely small stone targets, the average precision (AP) of this application reaches 80.3%, while YOLOv11n is only 72.5%, YOLOv8n is 69.4%, and YOLOv5s is only 56.2%. In terms of model complexity, YOLOv5s has 7.2M parameters and 16.5 GFLOPs of computation; YOLOv8n has 3.2M parameters and 8.9 GFLOPs of computation; YOLOv11n has approximately 4.0M parameters and approximately 10.0 GFLOPs of computation; and the YOLO26-p2 in this application has 4.3M parameters and 10.2 GFLOPs of computation, which is roughly equivalent to YOLOv11n, but with significantly better detection accuracy. This performance improvement is mainly due to two aspects: First, this application adds a P2 high-resolution feature fusion path in the neck network and sets up a dedicated tiny target detection head, enabling the 160×160 resolution feature map to fully preserve the detailed information of tiny targets at the bottom of the pool, effectively solving the problem of detail loss of small targets in deep feature maps; second, Focal Loss is used as the classification loss function during training, making the model more focused on small target samples that are difficult to classify, alleviating the problem of severe imbalance between targets and background at the bottom of the pool. Therefore, this application significantly improves the detection accuracy of extremely small underwater targets while maintaining the lightweight nature of the model, verifying the effectiveness and superiority of the proposed technical solution.

[0103] The above describes an underwater target detection method provided by the embodiments of this application. The following will describe the apparatus for performing the above underwater target detection method.

[0104] Please refer to 3. Figure 3 This is a schematic diagram of the structure of an underwater target detection device provided in an embodiment of this application. Figure 3 As shown, the underwater target detection device 300 includes:

[0105] The feature extraction unit 301 is used to input the image to be detected into the pre-trained target detection model, and extract features from the image to be detected through the backbone network of the target detection model to obtain multiple feature maps of different resolutions.

[0106] The feature fusion unit 302 is used to fuse multiple feature maps through the neck network of the target detection model to obtain fused feature maps at least three scales; wherein, the scale represents the downsampling factor of the fused feature map relative to the image to be detected, and the fused feature maps of different scales correspond to different target detection ranges.

[0107] The detection output unit 303 is used to perform category prediction and bounding box position determination on the fused feature map through the detection head of the target detection model to obtain the detection result of the underwater target.

[0108] In one possible implementation, the feature extraction unit 301 is specifically used to: perform an initial convolutional transformation on the image to be detected to obtain an initial feature map; and perform multi-level convolutional processing and downsampling processing on the initial feature map through a backbone network to output at least four-stage feature maps; wherein the at least four-stage feature maps include a first-stage feature map, a second-stage feature map, a third-stage feature map, and a fourth-stage feature map; the resolution of the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map decreases sequentially.

[0109] In one possible implementation, the feature fusion unit 302 is specifically configured to: perform a first upsampling on the fourth-stage feature map, and concatenate and fuse the first upsampling result with the third-stage feature map to obtain a first intermediate fused feature map; perform a second upsampling on the first intermediate fused feature map, and concatenate and fuse the second upsampling result with the second-stage feature map to obtain a second intermediate fused feature map; perform a third upsampling on the second intermediate fused feature map, and concatenate and fuse the third upsampling result with the first-stage feature map to obtain a third intermediate fused feature map; perform a convolution transformation on the first-stage feature map, and concatenate and fuse it with the third intermediate fused feature map to obtain a first-scale fused feature map; perform a first downsampling on the first-scale fused feature map, and concatenate and fuse the first downsampling result with the second intermediate fused feature map to obtain a second-scale fused feature map; perform a second downsampling on the second-scale fused feature map, and concatenate and fuse the second downsampling result with the first intermediate fused feature map to obtain a third-scale fused feature map; wherein the resolution of the first-scale fused feature map, the resolution of the second-scale fused feature map, and the resolution of the third-scale fused feature map decrease sequentially.

[0110] In one possible implementation, the detection output unit 303 is specifically configured to: set multiple detection branches in the detection head of the target detection model; wherein each detection branch is associated with a fusion feature. Figure 1 A one-to-one connection is established; for each fused feature map, a first convolution process is performed on the fused feature map through the classification prediction sub-branch corresponding to the fused feature map in each detection branch to predict the target category of the candidate target in the fused feature map, thereby obtaining the target category result; for each fused feature map, a second convolution process is performed on the fused feature map through the bounding box determination sub-branch corresponding to the fused feature map in each detection branch to determine the bounding box position parameters of the candidate target in the fused feature map, thereby obtaining the target bounding box result; based on the target category result and the target bounding box result, the detection result of the underwater target is generated.

[0111] In one possible implementation, the detection output unit 303 is specifically used for: acquiring annotation information from multiple labeled samples in the model training data; performing cluster analysis on the size of the annotation boxes of the multiple labeled information to obtain a set of candidate anchor boxes corresponding to different scales; dividing the set of candidate anchor boxes to obtain multiple sets of anchor boxes corresponding to different scales; and assigning the anchor boxes of the corresponding scales to the detection branches corresponding to each fusion feature map according to the multiple sets of anchor boxes, so that each detection branch after assignment determines the bounding box position parameters of the candidate targets in the fusion feature map corresponding to each detection branch based on the assigned anchor boxes.

[0112] In one possible implementation, the device further includes: a model training unit; the model training unit is specifically configured to: acquire raw videos under various underwater environmental conditions, and extract image frames from the raw videos as raw image samples; use an initial detection model to infer the raw image samples to generate pseudo-labels containing target categories and bounding box locations; correct the pseudo-labels to obtain labeled samples as model training data; and train the target detection model based on the model training data to obtain the pre-trained target detection model.

[0113] In one possible implementation, the model training unit is specifically configured to: perform sample augmentation processing on the labeled samples of the model training data to obtain augmented samples; input the augmented samples into the object detection model for forward computation to obtain prediction results; perform loss calculation based on the prediction results and the annotation information in the labeled samples to obtain classification loss and regression loss; wherein, the annotation information includes target category and bounding box position; update the parameters of the object detection model based on the classification loss and the regression loss; and repeatedly perform forward computation, loss calculation and parameter update until a preset training termination condition is reached.

[0114] This application also provides an electronic device in its embodiments. (See reference...) Figure 4 The diagram illustrates a structural schematic of an electronic device suitable for implementing the underwater target detection method in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as mobile phones, laptops, personal digital assistants (PDAs), tablet computers (PADs), desktop computers, etc. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0115] like Figure 4 As shown, the electronic device may include a processing unit (e.g., a central processing unit (CPU), graphics processing unit (GPU), etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. When the electronic device is powered on, RAM 403 also stores various programs and data required for the operation of the electronic device. The processing unit 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0116] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, memory cards, hard drives, etc.; and communication devices 409. Communication device 409 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0117] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the underwater target detection methods provided in this application.

[0118] This application also provides a computer-readable storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the underwater target detection methods provided in this application.

[0119] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause an electronic device (which may be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in the various embodiments of this application.

[0121] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0122] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for detecting underwater targets, characterized in that, include: The image to be detected is input into a pre-trained target detection model, and the backbone network of the target detection model is used to extract features from the image to be detected, resulting in multiple feature maps of different resolutions. The neck network of the target detection model fuses multiple feature maps to obtain fused feature maps at least three scales; wherein, the scale represents the downsampling factor of the fused feature map relative to the image to be detected, and the fused feature maps of different scales correspond to different target detection ranges; The detection head of the target detection model performs category prediction and bounding box position determination on the fused feature map to obtain the detection result of the underwater target.

2. The method according to claim 1, characterized in that, The process of extracting features from the image to be detected using the backbone network of the target detection model yields multiple feature maps of different resolutions, including: The image to be detected is subjected to an initial convolutional transformation to obtain an initial feature map; The initial feature map is subjected to multi-level convolution and downsampling processing through the backbone network to output at least four-stage feature maps; wherein the at least four-stage feature maps include a first-stage feature map, a second-stage feature map, a third-stage feature map, and a fourth-stage feature map; the resolution of the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map decreases sequentially.

3. The method according to claim 1, characterized in that, The multiple feature maps with different resolutions include a first-stage feature map, a second-stage feature map, a third-stage feature map, and a fourth-stage feature map; the resolutions of the first-stage feature map, the second-stage feature map, the third-stage feature map, and the fourth-stage feature map decrease sequentially. The feature fusion of multiple feature maps using the neck network of the target detection model to obtain fused feature maps at least three scales includes: The fourth-stage feature map is upsampled for the first time, and the first upsampling result is concatenated and fused with the third-stage feature map to obtain the first intermediate fused feature map; The first intermediate fused feature map is upsampled a second time, and the result of the second upsampling is concatenated and fused with the second stage feature map to obtain the second intermediate fused feature map; The second intermediate fusion feature map is upsampled a third time, and the result of the third upsampling is concatenated and fused with the first stage feature map to obtain the third intermediate fusion feature map; After performing a convolutional transformation on the first stage feature map, it is concatenated and fused with the third intermediate fusion feature map to obtain the first scale fusion feature map; The first scale fusion feature map is downsampled for the first time, and the first downsampling result is concatenated and fused with the second intermediate fusion feature map to obtain the second scale fusion feature map; The second-scale fused feature map is downsampled a second time, and the second downsampling result is spliced ​​and fused with the first intermediate fused feature map to obtain the third-scale fused feature map; wherein the resolution of the first-scale fused feature map, the resolution of the second-scale fused feature map, and the resolution of the third-scale fused feature map decrease sequentially.

4. The method according to claim 1, characterized in that, The step of performing category prediction and bounding box location determination on the fused feature map using the detection head of the target detection model to obtain the detection result of the underwater target includes: The detection head of the target detection model is provided with multiple detection branches; wherein each detection branch is connected to each fused feature map in a one-to-one correspondence. For each of the fused feature maps, a first convolution process is performed on the fused feature map through the classification prediction sub-branch corresponding to the fused feature map in each detection branch, so as to predict the target category of the candidate target in the fused feature map and obtain the target category result; For each fused feature map, a sub-branch is determined by the bounding box corresponding to the fused feature map in each detection branch, and a second convolution process is performed on the fused feature map to determine the bounding box position parameters of the candidate targets in the fused feature map, thereby obtaining the target bounding box result; Based on the target category results and the target bounding box results, the detection results of the underwater target are generated.

5. The method according to claim 1, characterized in that, Before performing category prediction and bounding box location determination on the fused feature map using the detection head of the target detection model to obtain the detection result of the underwater target, the method further includes: Obtain annotation information from multiple labeled samples in the model training data; Cluster analysis is performed on the size of the annotation boxes of multiple annotation information to obtain a set of candidate anchor boxes corresponding to different scales; The candidate anchor frame set is divided to obtain multiple sets of anchor frames corresponding to different scales; Based on the multiple sets of anchor boxes, anchor boxes of corresponding scales are assigned to the detection branches corresponding to each fused feature map, so that each detection branch determines the bounding box position parameters of the candidate targets in the fused feature map corresponding to each detection branch based on the assigned anchor boxes.

6. The method according to claim 1, characterized in that, The target detection model is trained through the following steps: Raw videos under various underwater environmental conditions are collected, and image frames are extracted from the raw videos as raw image samples; The initial detection model is used to infer the original image samples to generate pseudo-labels containing the target category and bounding box location; The pseudo-labels are corrected to obtain labeled samples as model training data; The target detection model is trained based on the model training data to obtain the pre-trained target detection model.

7. The method according to claim 6, characterized in that, The step of training the target detection model based on the model training data to obtain the pre-trained target detection model includes: The labeled samples of the model training data are subjected to sample augmentation processing to obtain augmented samples; The enhanced samples are input into the target detection model for forward computation to obtain prediction results; The loss is calculated based on the prediction results and the annotation information in the labeled samples to obtain classification loss and regression loss; wherein, the annotation information includes target category and bounding box location; The parameters of the target detection model are updated based on the classification loss and the regression loss. Repeat the forward computation, loss calculation, and parameter update until the preset training termination condition is met.

8. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the underwater target detection method as described in any one of claims 1 to 7.

9. An underwater robot, characterized in that, include: The electronic device as described in claim 8.

10. A computer storage medium, characterized in that, The computer storage medium carries one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the underwater target detection method as described in any one of claims 1 to 7.

Images