Two-stage underwater target detection method based on artificial intelligence

Abstract

The invention discloses a two-stage underwater target detection method based on artificial intelligence, and the method comprises the steps: firstly collecting target detection image data in an underwater environment, and carrying out the preprocessing and data enhancement; then deep features of the underwater image are extracted by using a convolutional neural network and an enhanced feature map is generated, the enhanced feature map and the original RGB image are fused and aligned, then a foreground target and a background region in a complex underwater scene are preliminarily distinguished by using a compressed and excited convolutional neural network, and a target detection frame is obtained; and a Google neural network is used to carry out fine classification on the candidate targets, pixel-level areas of the targets are further determined in a detection frame, and a multi-target joint loss function is constructed to carry out joint supervision training on the network in each stage. According to the method, high-precision target detection and instance segmentation can be carried out on the underwater image to be detected, the target detection frame and the pixel-level classification result are finally output, and the detection accuracy and robustness in the underwater complex environment are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a two-stage underwater target detection method based on artificial intelligence. Background Technology

[0002] In recent years, with the rapid development of computer vision and artificial intelligence technologies, vision-based underwater environmental perception has gradually become a core technology in underwater robot navigation and operating systems. However, due to the complex optical characteristics of underwater environments, image quality is easily affected by light absorption, scattering, and suspended particles, resulting in underwater images generally suffering from low clarity, poor contrast, and severe color shifts, thus limiting the effectiveness of vision-based target detection and recognition algorithms. Images acquired by underwater robots in actual operations are often affected by both the characteristics of the water medium and lighting conditions. As light propagates in water, red and orange wavelengths are rapidly absorbed, while blue and green wavelengths have relatively stronger penetrating power, resulting in an overall bluish or greenish tint to the image and severe distortion of color information. Simultaneously, the scattering of light by suspended particles in the water introduces a large amount of noise, causing image blurring and reduced contrast. For deep learning-based target detection algorithms, this decline in image quality directly leads to instability in feature extraction, thereby reducing detection accuracy. Furthermore, the significant differences in image characteristics under different water conditions, depths, and lighting conditions result in insufficient generalization ability of algorithms when applied across different scenarios, becoming a key bottleneck restricting the improvement of the visual intelligence level of underwater robots. Therefore, how to effectively enhance images and accurately detect targets in complex and ever-changing underwater environments has become a research hotspot and challenge in the field of underwater robot visual perception. To address these issues, academia and engineering have proposed various underwater image enhancement and target detection methods. Traditional image enhancement algorithms, such as histogram equalization, Retinex enhancement, and median filtering, are effective in improving image brightness and contrast and suppressing noise, but their results often depend on scene characteristics and are difficult to apply universally under different water quality conditions. In recent years, with the development of deep learning technology, target detection algorithms have gradually evolved from two-stage detection (such as Faster R-CNN) to single-stage detection represented by the YOLO series, significantly improving detection speed and real-time performance. In particular, the YOLO series algorithms have demonstrated excellent accuracy and efficiency in target detection in complex scenes. However, the low quality and high noise levels of underwater images still interfere with feature extraction by deep networks, making detection models prone to missed detections and false detections. Therefore, combining traditional image enhancement techniques with deep learning target detection algorithms, improving image quality through front-end enhancement, and then using instance segmentation to achieve accurate target recognition has become an effective way to improve the visual perception performance of underwater robots.

[0003] Existing technologies, such as patent CN111723823B, disclose an underwater target detection method based on third-party transfer learning. This method constructs a cross-domain data model and combines it with an improved deep neural network to achieve underwater target detection. While this method alleviates the problem of insufficient underwater samples to some extent and improves the training efficiency of the detection network, it still has significant limitations. First, this patent does not address the degradation problems commonly found in underwater images, such as color shift, low contrast, and suspended particle noise, resulting in the feature quality of the input network still being affected by underwater optical attenuation, thus reducing the stability of detection. Second, existing methods mainly output target detection boxes and lack pixel-level segmentation and fine boundary extraction capabilities, making them prone to missed detections and false detections for targets with blurred edges, partial occlusion, and small sizes. Furthermore, the generalization ability of this method under different water and lighting conditions depends on the third-party data domain, failing to fundamentally solve the problem of cross-scene feature drift caused by underwater optical differences.

[0004] Existing technologies, such as patent US20240362750A1, propose an underwater image enhancement method based on histogram optimization and a depth model, which improves visual quality by generating an optimized color histogram. However, this technical solution mainly focuses on the overall visual enhancement effect of underwater images, and the enhancement process is not designed for the feature requirements of target detection tasks, such as the local contrast of the target region, the prominence of texture features, and the suppression of suspended particle interference. Therefore, when used in practical underwater target detection tasks, this type of enhancement method may still introduce new feature biases due to insufficient or excessive enhancement. In addition, this technology is not jointly optimized with target detection or instance segmentation networks; its enhancement module and backend detection module work independently, resulting in a disconnect between the enhancement results and the detection task, making it difficult to improve the accuracy and robustness of the detection system as a whole. For real-time perception scenarios in complex aquatic environments, its versatility and efficiency still need further verification.

[0005] Existing technologies still suffer from the following problems: Several common shortcomings exist in practical applications of current underwater image enhancement and target detection technologies. First, most solutions lack a unified optimization framework encompassing image enhancement, feature alignment, target detection, and pixel-level segmentation, resulting in insufficient information transfer between modules and limited overall performance. Second, they lack in-depth modeling of underwater optical degradation mechanisms and fail to fully integrate features from complex environments such as multiple scenes, depths, and lighting conditions for feature enhancement and fusion, leading to insufficient generalization ability. Third, existing detection methods generally lack fine delineation of target boundaries, only outputting detection boxes, which cannot meet the high-precision contour recognition requirements of underwater robots in tasks such as operation, grasping, and positioning. Therefore, a two-stage underwater target detection method is needed that can simultaneously address image enhancement, feature fusion, target detection, and pixel-level precise delineation to solve problems such as insufficient image quality, unstable feature extraction, low detection accuracy, and poor cross-scene adaptability in existing technologies. Summary of the Invention

[0006] The purpose of this invention is to overcome the problems of image quality degradation, unstable feature extraction, inaccurate detection box localization, and insufficient cross-scene generalization ability in existing underwater image enhancement and target detection technologies, and to provide a two-stage underwater target detection method based on artificial intelligence. By introducing multi-scene underwater data preprocessing, fusion and alignment of enhanced features and original images, channel attention enhancement of SE-Net, fine classification and pixel-level contouring of Google-Net, and end-to-end optimization of multi-target joint loss, this invention can effectively improve the visibility and distinguishability of targets in complex underwater environments, achieve high-precision detection and instance-level segmentation of underwater targets, and thus significantly improve the intelligent perception and environmental adaptability of underwater robots in the process of detection, monitoring, and operation.

[0007] To achieve the above objectives, the present invention provides a two-stage underwater target detection method based on artificial intelligence, comprising the following steps:

[0008] S1. Collect underwater target detection data from multiple scenarios and preprocess the underwater target detection data;

[0009] S2. Use a convolutional neural network to extract features from the acquired underwater target detection images and generate enhanced feature maps;

[0010] S3. Fuse and align the enhanced feature map and the original RGB image;

[0011] S4. Use a compressed and activated convolutional neural network to initially distinguish between targets and background in complex underwater scenes and detect target detection boxes;

[0012] S5. Use Google's neural network to perform fine classification of the candidate targets output in step S4, and accurately delineate the pixel region of the target within the target detection box;

[0013] S6. Supervised training of the neural network mentioned in steps S2, S4, and S5 is performed using a multi-objective joint loss function to obtain an underwater instance segmentation model.

[0014] S7. Preprocess the underwater environment data to be detected, use the underwater instance segmentation model to detect targets, and finally output the target detection box and pixel-level classification results.

[0015] Furthermore, the multi-scene underwater target detection data includes target detection RGB images in different underwater environments such as lakes, swimming pools, and oceans, as well as corresponding annotation information. The annotation information includes pixel-level target segmentation annotations and target detection bounding box data.

[0016] Furthermore, the preprocessing of the underwater target detection data in step S1 includes the following steps:

[0017] S11, Data Augmentation: Rotate the image in the RGB image data at a random angle, then perform vertical translation in the horizontal and vertical directions, and then scale the image proportionally.

[0018] S12. Normalization Processing: For underwater target detection image data, random noise is first introduced, and then each pixel is normalized.

[0019] Furthermore, the convolutional neural network mentioned in step S2 is a two-dimensional convolutional neural network that includes multiple layers of convolutional layers, pooling layers, and fully connected layers.

[0020] Furthermore, step S3 specifically includes the following steps:

[0021] S31. Align the enhanced feature map with the original RGB image in terms of size;

[0022] S32. Using convolution operations or attention mechanisms, the important information in the enhanced feature map is fused with low-level features such as color and brightness in the original RGB image;

[0023] S33. Use a deep convolutional neural network or image optimization algorithm to perform local enhancement processing on the fused image.

[0024] Furthermore, the implementation process of step S33 is as follows: First, weight alignment is performed; second, an activation function is used to eliminate nonlinearity; and then the weights are reassigned to output the feature map after feature enhancement.

[0025] Furthermore, step S4 specifically includes the following steps:

[0026] S41. Compression and statistics of channel features: Perform global average pooling on the fused input feature map to obtain the statistics of each channel;

[0027] S42, Channel Weight Generation: The importance of channels is modeled using a two-layer fully connected network to obtain the weight value of each channel;

[0028] S43, Channel recalibration: After obtaining the channel weights, the input features are processed by channel-by-channel weighting.

[0029] S44. Target response map calculation: Perform convolution processing on the feature map to obtain the spatial response map;

[0030] S45. Dynamic Threshold Generation and Candidate Region Determination: A dynamic threshold is generated based on the statistical characteristics of the response map, thereby obtaining candidate regions.

[0031] S46. Candidate Detection Box Filtering: The set of detection boxes generated based on the candidate regions is filtered by intersection-union ratio;

[0032] S47. Channel-weighted box stability assessment: Use channel attention weights to determine the consistency of candidate boxes.

[0033] Furthermore, step S5 specifically includes the following steps:

[0034] S51, Candidate Target Region Cropping and Input Normalization: For each candidate target detection box output in step S4, the candidate target region is cropped, and the cropped image region is scaled to a uniform size through bilinear interpolation and other methods. To eliminate brightness and contrast differences, the pixels are normalized.

[0035] S52. Fine classification and deep feature extraction of candidate targets based on Google neural network: Input the preprocessed candidate regions into Google neural network, and obtain the target category probability distribution through fully connected layers and Softmax function;

[0036] S53. Pixel-level target delineation within the candidate target detection box: Upsample and predict the intermediate feature map at the pixel level. Upsample the feature map using deconvolution, bilinear interpolation, or a fully convolutional network structure to restore it to the candidate region size, thus obtaining pixel-level predicted features. Define the probability of each pixel position belonging to the target class and output the probability value of the target pixel. Generate a binary target mask based on the probability map and calculate the minimum bounding rectangle of all target pixels in the mask to obtain a refined detection box.

[0037] Furthermore, step S6 specifically includes the following steps:

[0038] S61. Construct the feature alignment loss;

[0039] S62. Construct the regression loss for candidate detection boxes;

[0040] S63. Construct the target classification loss;

[0041] S64. Construct pixel-level segmentation loss;

[0042] S65. Construct a multi-objective joint loss as the optimization objective for backpropagation to update all network parameters during training.

[0043] Furthermore, the target classification loss adopts cross-entropy loss, and the pixel-level segmentation loss adopts binary cross-entropy loss.

[0044] By constructing an integrated two-stage underwater target detection system encompassing "enhanced feature extraction—feature fusion and alignment—candidate region detection—refined classification and pixel-level delineation—joint loss training," this invention achieves deep integration of image enhancement, feature learning, and instance segmentation, significantly improving target recognition capabilities in complex underwater environments. Compared to existing technologies, the beneficial effects of this invention are:

[0045] 1. This invention introduces a fusion and alignment mechanism between enhanced features and the original RGB image during the feature extraction stage, which effectively compensates for color shift and contrast reduction in underwater images, making the input features more stable and reliable.

[0046] 2. This invention enhances target-related features through the channel attention mechanism of SE-Net while suppressing background interference caused by suspended particles and uneven illumination, thereby improving the accuracy and recall of candidate detection boxes. Furthermore, this invention utilizes Google-Net for fine-grained classification of candidate regions and achieves pixel-level boundary delineation within the detection boxes, solving the problem of existing methods' difficulty in accurately characterizing target shapes.

[0047] 3. The multi-objective joint loss function proposed in this invention can achieve end-to-end collaborative optimization of enhancement, detection and segmentation networks, which can significantly improve the model's cross-scene generalization ability and overall detection accuracy.

[0048] In summary, this invention can stably achieve high-precision underwater target detection under various water conditions, depths, and lighting conditions, and has better robustness, adaptability, and practical value. Attached Figure Description

[0049] Figure 1 This invention provides a schematic diagram of a two-stage underwater target detection method based on artificial intelligence.

[0050] Figure 2 A schematic diagram of the first-stage target selection SE-CNN provided by the present invention;

[0051] Figure 3 This is a schematic diagram of the structure of Google-Net, which is used for the second-stage pixel-level semantic segmentation and classification provided by the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] This invention provides a two-stage underwater target detection method based on artificial intelligence, aiming to solve the problem of low visibility in underwater images caused by light absorption, scattering, and suspended particles, and to improve the target perception capabilities of underwater robots, unmanned underwater vehicles, and underwater monitoring equipment in multiple water areas and at multiple depths. The method includes seven main steps: underwater multi-scene data acquisition and preprocessing, CNN feature extraction, feature fusion and alignment, SE-Net candidate target detection, Google-Net fine classification and pixel delimitation, multi-target joint loss training, and target detection output during the model inference stage.

[0054] This invention employs a system of "front-end enhancement + mid-layer fusion + channel attention detection + deep classification and segmentation + joint training optimization" to achieve highly robust detection of underwater targets in different scenarios.

[0055] This invention provides a two-stage underwater target detection method based on artificial intelligence, such as... Figure 1 The above includes the following steps:

[0056] Step S1: Collect underwater target detection data from multiple scenarios and preprocess the data. The dataset covers RGB images of targets in different underwater environments such as lakes, swimming pools, and oceans, along with corresponding annotation information, including pixel-level target segmentation annotations and target detection bounding box data. After the raw data collection is completed, data augmentation is performed to generate more data to improve the model's robustness.

[0057] In this invention, to improve the algorithm's cross-scene generalization ability, data from different underwater environments are first collected, including but not limited to: images collected by underwater cameras in nearshore shallow sea areas and deep sea areas; inland environments such as freshwater lakes and rivers; experimental environments such as water tanks and pools; underwater imaging environments in low light at night; and special extreme conditions such as turbid water bodies and water bodies with dense algae.

[0058] All collected image data are accompanied by corresponding annotation files, which include: target detection box location labels; pixel-level target segmentation region labels; target category labels, such as underwater organisms (fish, sea cucumbers, sea turtles, seahorses, etc.), underwater structures (pipes, corals, debris, etc.) or artificial objects (robotic arms, detection equipment, etc.).

[0059] To enhance the model's robustness, various data augmentations were performed on the original images, including geometric augmentation (rotation, flipping, scaling), color perturbation (RGB jittering, color cast simulation), illumination enhancement (contrast stretching, dark light simulation), and noise injection (Gaussian noise, suspended particle simulation). The augmented data significantly improved the model's ability to cope with illumination shifts, viewpoint perturbations, and different water quality environments.

[0060] Furthermore, the preprocessing of the underwater target detection data in step S1 includes the following steps:

[0061] S11, Data Augmentation: Rotate the image in the RGB image data at a random angle, then perform vertical translation in the horizontal and vertical directions, and then scale the image proportionally.

[0062] S12. Normalization Processing: For underwater target detection image data, random noise is first introduced. The pixel after noise enhancement can be represented as: Then normalize each pixel:

[0063] ;

[0064] Where I represents the intensity of each pixel, N( () represents a mean of 0 and a variance of . The normal distribution To incorporate noise, the standard deviation follows a normal distribution, where x represents the input image. Represents the maximum intensity of a pixel. Represents the minimum intensity of a pixel. This represents the pixel intensity after noise enhancement.

[0065] Step S2: Use a convolutional neural network (CNN) to extract features from the acquired underwater target detection images and generate enhanced feature maps.

[0066] In this invention, deep semantic features are extracted from preprocessed underwater images using a deep convolutional neural network (CNN). The network architecture can be VGG, ResNet, DenseNet, or a lightweight MobileNet, chosen flexibly based on the computing power of the specific device.

[0067] For the input image By repeatedly stacking convolutional layers, batch normalization layers, ReLU activation layers, and pooling layers, deeper feature maps are obtained. Output feature map. It can be represented as:

[0068]

[0069] in, These are the parameters for the CNN network. It contains a large amount of texture, edge, and shape information related to the target object.

[0070] The features extracted in this step will be used for subsequent fusion with the original RGB image.

[0071] Furthermore, the specific description of the MRI feature extraction convolutional upgrade network structure design mentioned in step S2 is as follows:

[0072] Specifically, the convolutional neural network mentioned in step S2 is a two-dimensional convolutional neural network including multiple layers of convolutional layers, pooling layers, and fully connected layers. After this stage, the model outputs:

[0073] S2.1: Multi-scale semantic feature maps extracted by the backbone network (denoted as C2, C3, C4, C5), and pyramid features obtained by fusion from the feature pyramid network (denoted as P3, P4, P5);

[0074] S2.2: Gradient and edge feature maps generated by the edge enhancement branch, which includes weight-freezing convolution operators to simulate Sobel / Scharr / Laplacian filtering to highlight target boundaries and fine-grained textures;

[0075] S2.3: High and low frequency decoupling feature maps generated by the frequency domain enhancement branch. The branch performs frequency domain modeling on the input based on discrete cosine or wavelet-style convolutional units, selectively amplifies high-frequency components, and suppresses low-frequency turbid background.

[0076] S2.4: Color enhancement tone output by the color correction and contrast enhancement subnet is used to alleviate the blue-green color cast and low contrast problems caused by underwater imaging. The subnet is a lightweight encoder-decoder structure. Its output is spliced ​​with the original image along the channel dimension and then sent to the backbone.

[0077] S2.5: Channel attention weights and spatial attention maps generated by the attention guidance module are used to weight the above multi-scale features to suppress the water background and highlight the target area;

[0078] S2.6: Feature maps with improved resolution or geometric alignment obtained from the super-resolution and alignment module, used to improve the detectability of small targets at a distance and the accuracy of mask boundary alignment.

[0079] Step S3: Fuse and align the RGB enhanced feature map described in step S2 and the original RGB image.

[0080] Furthermore, step S3 specifically includes the following steps:

[0081] S31. Alignment of Feature Map with Original Image: In this step, the RGB enhanced feature map generated in step S2 is first aligned with the original RGB image in terms of size. Since images whose features are extracted by convolutional neural networks are usually reduced in size, interpolation or upsampling algorithms are needed to restore them to their original size, ensuring spatial consistency between the enhanced feature map and the original image. Common interpolation methods include bilinear interpolation, nearest neighbor interpolation, or convolutional backpropagation. The purpose of alignment is to ensure that the enhanced feature map can be fused with the original image at the same pixel locations, thereby preserving the spatial feature information of the image.

[0082] S32. Feature Enhancement and Information Extraction: The RGB enhanced feature map extracted in step S2 contains rich deep image information, including edges, textures, and local features. In this step, specific convolutional operations or attention mechanisms are used to fuse the important information in the enhanced feature map with low-level features such as color and brightness in the original RGB image. Through weighted fusion or concatenation, the detailed information of the enhanced feature map is introduced into the original image, enhancing the image's recognition ability. The fusion method can use direct addition, concatenation, or other specific convolutional operations.

[0083] S33. Further Optimization After Feature Map and Image Fusion: After fusing the feature map and the original RGB image, further optimization is performed to remove redundant information and enhance the accuracy of object detection. This step involves applying a deep convolutional neural network (CNN) or image optimization algorithm to locally enhance the fused image, highlighting the target region and suppressing background noise. Common optimization methods include channel weighting (e.g., using attention mechanisms) or image enhancement techniques (such as adaptive contrast adjustment) to further improve the image's feature response, thereby providing a clearer and more accurate input image for subsequent object detection and classification.

[0084] Specifically, the implementation process of feature alignment and weight reassignment is as follows:

[0085] First, weight alignment is performed. The specific calculation process is as follows:

[0086] in, This represents the features of the stitched multi-channel underwater image. Represents the RGB channels of the original underwater image. This represents the feature map generated after step S3.

[0087] ,

[0088] in, The value represents the characteristic after linear transformation. Represents a linear transformation matrix. Represents offset displacement.

[0089]

[0090] Furthermore, an activation function is used to eliminate nonlinearity, and its calculation process is as follows, where This represents the feature map output after activation function transformation, where exp represents the exponential operation. The sigmoid activation function representing variable e:

[0091]

[0092] The weights are reassigned again. The product represents the Hadamard product, multiplied point by point. Its calculation process is as follows: The output represents the feature map after feature enhancement:

[0093] .

[0094] Step S4 uses a compressed and excitation neural network (SE-Net) to initially distinguish between targets and background in complex underwater scenes and detect target bounding boxes.

[0095] Furthermore, step S4 specifically includes the following steps:

[0096] Step S41, Compression and Statistics of Channel Features (Squeeze Stage):

[0097] First, the fused input feature map Perform global average pooling to obtain the statistics for each channel:

[0098]

[0099] in, : indicates the position of the c-th channel of the input feature map at pixel location. eigenvalues; These represent the height and width of the feature map, respectively; C represents the total number of channels. : Represents the average characteristic response of the c-th channel, which is related to the overall importance of that channel.

[0100] Step S42, Channel Weight Generation (Excitation Stage):

[0101] The importance of channels is modeled using a two-layer fully connected network, and its mathematical expression is as follows:

[0102]

[0103] in, Represents the channel statistics vector; For channel compression and recovery, there is a learnable weight matrix; Represents the ReLU activation function, used to enhance nonlinearity; This represents the Sigmoid activation function, used to compress weights to... scope; The channel attention weight vector generated for SE-Net, where the weight value of each channel reflects the relative importance of that channel in the current task;

[0104] Step S43, Channel Re-scaling:

[0105] After obtaining the channel weights, the input features are processed by weighting each channel:

[0106]

[0107] in, This represents the weight coefficient of the c-th channel; This represents the weighted c-th channel feature; this step enhances the target-related channel features and suppresses channel features that are significantly affected by underwater noise.

[0108] Step S44, Target Response Map Calculation:

[0109] To identify potential target regions from the enhanced features, the feature map is convolved to obtain a spatial response map:

[0110]

[0111] in, This represents the aforementioned weighted enhanced feature map; This represents the convolution operation; Learnable parameters of the convolution kernel; : Used to indicate the degree of target response at different spatial locations.

[0112] Step S45: Dynamic threshold generation and candidate region determination:

[0113] A dynamic threshold is generated based on the statistical characteristics of the response map, and candidate regions are obtained through the following relationship:

[0114]

[0115] in, The mean of the response plot; The standard deviation of the response plot; Represents the threshold adjustment coefficient;

[0116] Candidate regions are defined as:

[0117]

[0118] in, This represents the set of pixels that are identified as potential target regions.

[0119] Step S46: Candidate detection box selection (Non-maximum suppression, NMS):

[0120] A set of detection boxes generated based on candidate regions Representatives are selected using the Intersection over Union (IoU) ratio:

[0121]

[0122] in, Represents any two candidate boxes; This represents the area of ​​the intersection of the two frames; Represents the area of ​​the union of the two frames, when:

[0123]

[0124] If the two boxes are considered duplicates, the one with the lower score will be suppressed. The parameter τ is the threshold for overlap filtering, which is generally set to 0.3 to 0.5.

[0125] Step S47: Evaluation of box stability based on channel weights:

[0126] To further improve robustness, this step utilizes channel attention weights to determine the consistency of candidate boxes:

[0127]

[0128] in, This indicates the overall channel feature stability of the candidate box. Represents the attention value of the c-th channel, if:

[0129]

[0130] If the bounding box features are unstable, it is considered to be removed. The parameter γ is the stability threshold.

[0131] Step S5: Use Google Neural Networks (Google-Net) to perform fine classification of the candidate targets output from Step S4, and accurately delineate the pixel region of the target within the target detection box.

[0132] Furthermore, step S5 specifically includes the following steps:

[0133] S51. Candidate Target Region Clipping and Input Normalization:

[0134] Each candidate target detection box output in step S4 is denoted as:

[0135]

[0136] in, Represents the k-th candidate object detection box; This represents the coordinates of the top-left corner of the candidate box in the original image. These represent the width and height of the candidate box, respectively. This represents the total number of candidate object detection boxes.

[0137] From the original RGB image Crop out the candidate target area:

[0138]

[0139] in, This represents the original input underwater image; Represents the original image height and width; Represents the corresponding candidate box The image sub-region. To adapt to the input requirements of Google-Net, the cropped image region... Scale to a uniform size using methods such as bilinear interpolation. ,get:

[0140]

[0141] To eliminate differences in brightness and contrast, pixels are normalized:

[0142]

[0143] in, After normalization, the k-th candidate region is located at... Pixel values ​​on color channel c; The mean and standard deviation of the c-th color channel can be obtained from the training set statistics; This represents the three channels: R, G, and B.

[0144] This step completes the cropping, scaling, and normalization of the candidate target regions, preparing them for subsequent input into the Google-Net network.

[0145] S52. Fine-grained classification and deep feature extraction of candidate targets based on Google-Net:

[0146] Preprocessed candidate regions Input the Google-Net network, and denote the Google-Net main feature extraction mapping as:

[0147]

[0148] in, This represents the GoogLeNet feature extraction network, which consists of multiple convolutional layers, Inception modules, pooling layers, etc. This represents the set of all learnable parameters of the GoogLeNet network; The deep feature vector representing the k-th candidate region has dimension D.

[0149] In the classification stage, the probability distribution of the target category is obtained through a fully connected layer and a softmax function. Let C be the number of target categories to be identified in this invention. Then, the category determination output for the k-th candidate box is:

[0150]

[0151]

[0152] in, This represents the unnormalized score (logits) of the k-th candidate box on category c. This represents the classification weight vector and bias term for the corresponding category c; represents the probability that the k-th candidate box belongs to category ccc; C represents the number of target categories, including all types of underwater targets and background.

[0153] Optionally, if the true class labels are denoted as one-hot vectors ,in Then the classification loss at this stage can be expressed as cross-entropy (for subsequent S6 joint training):

[0154]

[0155] The above classification output enables the system to make fine-grained category determinations for each candidate target, while also utilizing deep features. This provides a semantic information basis for subsequent pixel-level segmentation.

[0156] S53. Pixel-level target delineation within the candidate target detection box:

[0157] To accurately delineate the pixel region of the target within the candidate detection box, this invention upsamples and performs pixel-level prediction on the intermediate feature map based on Google-Net. Let the feature map output from a certain layer in the GoogLeNet network be:

[0158]

[0159] in, This represents the intermediate layer feature map corresponding to the k-th candidate region; This represents the height and width of the feature map; The number of channels in the feature map is represented by the number of channels. The feature map is upsampled using deconvolution (transposed convolution), bilinear interpolation, or a fully convolutional network (FCN) structure to restore it to the candidate region size (Hg, Wg), resulting in pixel-level predicted features.

[0160]

[0161] in, The upsampling and decoding function can be composed of deconvolutional layers, interpolation layers, and convolutional layers. This represents the set of learnable parameters used for pixel-level decoding. The number of channels represents the pixel-level prediction; generally, for binary classification (target / background), a value of [value] can be taken as [value]. For each pixel position The probability that it belongs to the target class is defined as:

[0162]

[0163] in, The eigenvector at that location; The weights and biases of the pixel-level classifier; This represents the Sigmoid function, used to output the probability of a target in the range of 0–1;

[0164] Represents the pixel in the k-th candidate box This represents the probability value of the target pixel.

[0165] According to the probability diagram Generate a binary target mask:

[0166]

[0167] in, Representing pixels The classification results are as follows: 1 represents the target and 0 represents the background. This represents the pixel-level segmentation threshold, which can be set based on experience with the validation set.

[0168] To further refine the target boundary, the minimum bounding rectangle of all target pixels in the mask can be calculated as a refined detection box:

[0169]

[0170] in. This represents a refined target detection bounding box obtained based on a pixel-level mask. This represents a geometric operation function that calculates the minimum bounding rectangle of a given set of pixels.

[0171] Optionally, if pixel-level labeled data is available, a pixel-level segmentation loss can be defined (for subsequent S6 joint loss training):

[0172]

[0173] in, Represents the pixels within the k-th candidate box The true label is 1 for the target pixel and 0 for the background pixel.

[0174] Step S6: Supervised training of the neural network mentioned in steps S3, S4, and S5 is performed using a multi-objective joint loss function.

[0175] Specifically, the implementation process is as follows:

[0176] S61, Feature fusion consistency loss (corresponding to step S3)

[0177] To constrain the feature fusion process in step S3, so that the enhanced feature map To maintain consistency with the original RGB image features, this invention sets a feature alignment loss:

[0178]

[0179] in, The feature map represents the enhanced fusion. Represents shallow features extracted from the original RGB image;

[0180] This represents the L2 norm, used to measure the difference between the two. This loss helps enhance the stability of features at the color and texture levels.

[0181] S62, Candidate detection box regression loss (corresponding to step S4)

[0182] When SE-Net outputs candidate object boxes, it constrains the consistency between the position of the regression loss box and the ground truth labeled box:

[0183]

[0184] in, This represents the predicted value of the i-th candidate box; Represents the corresponding ground truth bounding box; N represents the number of positive bounding boxes participating in training;

[0185] This represents a smooth L1 loss, which is less sensitive to large errors and provides more stable training. This loss improves the accuracy of candidate box localization.

[0186] S63, Target Classification Loss (corresponding to step S5, first stage classification)

[0187] When performing fine-grained classification of candidate boxes using Google-Net, cross-entropy loss is employed.

[0188]

[0189] Where C represents the number of target categories; Represents the true category label (one-hot format); This represents the category prediction probability of the Google-Net output.

[0190] This classification loss improves the stability of underwater target category identification.

[0191] S64, Pixel-level segmentation loss (corresponding to step S5, second-stage segmentation):

[0192] To accurately delineate the pixel region of the target within the detection box, a binary cross-entropy loss is used:

[0193]

[0194] in, Representing pixels The predicted probability value of the target; This represents the corresponding actual pixel label; This represents the size of the segmentation mask.

[0195] This loss can improve pixel-level boundary accuracy within the detection frame.

[0196] S65. Multi-objective joint loss function:

[0197] Finally, this invention constructs the following multi-objective joint loss function to achieve joint end-to-end training of the three stages S3, S4, and S5:

[0198]

[0199] in The weighting coefficients representing the various losses are used to balance the importance of different tasks and are set based on the performance on the validation set. This represents the joint loss of multiple objectives, serving as the optimization objective for updating all network parameters during backpropagation during training.

[0200] The joint loss mechanism can promote the coordinated improvement of multiple tasks such as feature fusion, target detection, target classification and pixel-level delineation, thereby improving the detection accuracy and robustness in complex underwater environments.

[0201] Step S7: Preprocess the underwater environment data to be detected, use the underwater instance segmentation model trained in step S6 to perform target detection, and finally output the target detection box and pixel-level classification results.

[0202] This invention collects transverse and longitudinal MRI images and various clinical indicators from patients waiting for their appointments. After preprocessing, these images are input into the three-category auxiliary diagnostic model obtained in step S7, thereby outputting the diagnostic classification results.

[0203] The underwater image to be detected is input into the trained model. After enhanced feature extraction, fusion, candidate detection, classification, and segmentation, the output includes: target detection box; target category; and pixel-level mask (instance-level segmentation). It can be applied to scenarios such as underwater robot navigation and positioning; seabed structure damage detection; marine life monitoring; and autonomous underwater drone operations.

[0204] To verify the effectiveness of this invention, it was validated on public datasets, in a pool environment, and in real sea areas: the detection mAP was improved by 12%-28%, the recall rate for small targets was improved by 20%, and the stability of detection in noisy waters was significantly enhanced; the system can run in real time at 20–40 FP. These results demonstrate that this invention has significant robustness and practical value in complex underwater environments such as high turbidity, low light, and color distortion.

[0205] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A two-stage underwater target detection method based on artificial intelligence, characterized in that, Includes the following steps: S1. Collect underwater target detection data from multiple scenarios and preprocess the underwater target detection data; S2. Use a convolutional neural network to extract features from the acquired underwater target detection images and generate enhanced feature maps; S3. Fuse and align the enhanced feature map and the original RGB image; S4. Use a compressed and activated convolutional neural network to initially distinguish between targets and background in complex underwater scenes and detect target detection boxes; S5. Use Google's neural network to perform fine classification of the candidate targets output in step S4, and accurately delineate the pixel region of the target within the target detection box; S6. Supervised training of the neural network mentioned in steps S2, S4, and S5 is performed using a multi-objective joint loss function to obtain an underwater instance segmentation model. S7. Preprocess the underwater environment data to be detected, use the underwater instance segmentation model to detect targets, and finally output the target detection box and pixel-level classification results.

2. The two-stage underwater target detection method based on artificial intelligence according to claim 1, characterized in that, The multi-scene underwater target detection data includes target detection RGB images in different underwater environments and corresponding annotation information. The different underwater environments include lakes, swimming pools, and oceans. The annotation information includes pixel-level target segmentation annotations and target detection bounding box data.

3. The two-stage underwater target detection method based on artificial intelligence according to claim 1, characterized in that, Step S1 involves preprocessing the underwater target detection data, which includes the following steps: S11, Data Augmentation: Rotate the image in the RGB image data at a random angle, then perform vertical translation in the horizontal and vertical directions, and then scale the image proportionally. S12. Normalization Processing: For underwater target detection image data, random noise is first introduced, and then each pixel is normalized.

4. The two-stage underwater target detection method based on artificial intelligence according to claim 1, characterized in that, The convolutional neural network mentioned in step S2 is a two-dimensional convolutional neural network that includes multiple layers of convolutional layers, pooling layers, and fully connected layers.

5. The two-stage underwater target detection method based on artificial intelligence according to claim 1, characterized in that, Step S3 specifically includes the following steps: S31. Align the enhanced feature map with the original RGB image in terms of size; S32. Using convolution operations or attention mechanisms, the important information in the enhanced feature map is fused with the low-level features in the original RGB image, whereby the low-level features include color features and brightness features. S33. Use a deep convolutional neural network or image optimization algorithm to perform local enhancement processing on the fused image.

6. The two-stage underwater target detection method based on artificial intelligence according to claim 5, characterized in that, The implementation process of step S33 is as follows: First, weight alignment is performed; second, an activation function is used to eliminate nonlinearity; and then the weights are reassigned to output the feature map after feature enhancement.

7. The two-stage underwater target detection method based on artificial intelligence according to claim 1, characterized in that, Step S4 specifically includes the following steps: S41. Compression and statistics of channel features: Perform global average pooling on the fused input feature map to obtain the statistics of each channel; S42, Channel Weight Generation: The importance of channels is modeled using a two-layer fully connected network to obtain the weight value of each channel; S43, Channel recalibration: After obtaining the channel weights, the input features are processed by channel-by-channel weighting. S44. Target response map calculation: Perform convolution processing on the feature map to obtain the spatial response map; S45. Dynamic Threshold Generation and Candidate Region Determination: A dynamic threshold is generated based on the statistical characteristics of the response map, thereby obtaining candidate regions. S46. Candidate Detection Box Filtering: The set of detection boxes generated based on the candidate regions is filtered by intersection-union ratio; S47. Channel-weighted box stability assessment: Use channel attention weights to determine the consistency of candidate boxes.

8. The two-stage underwater target detection method based on artificial intelligence according to claim 1, characterized in that, Step S5 specifically includes the following steps: S51, Candidate Target Region Cropping and Input Normalization: For each candidate target detection box output in step S4, the candidate target region is cropped, and the cropped image region is scaled to a uniform size using bilinear interpolation. To eliminate brightness and contrast differences, the pixels are normalized. S52. Fine classification and deep feature extraction of candidate targets based on Google neural network: Input the preprocessed candidate regions into Google neural network, and obtain the target category probability distribution through fully connected layers and Softmax function; S53. Pixel-level target delineation within the candidate target detection box: Upsample and predict the intermediate feature map at the pixel level. Upsample the feature map using deconvolution, bilinear interpolation, or a fully convolutional network structure to restore it to the candidate region size, thus obtaining pixel-level predicted features. Define the probability of each pixel position belonging to the target class and output the probability value of the target pixel. Generate a binary target mask based on the probability map and calculate the minimum bounding rectangle of all target pixels in the mask to obtain a refined detection box.

9. The two-stage underwater target detection method based on artificial intelligence according to claim 1, characterized in that, Step S6 specifically includes the following steps: S61. Construct the feature alignment loss; S62. Construct the regression loss for candidate detection boxes; S63. Construct the target classification loss; S64. Construct pixel-level segmentation loss; S65. Construct a multi-objective joint loss as the optimization objective for backpropagation to update all network parameters during training.

10. The two-stage underwater target detection method based on artificial intelligence according to claim 9, characterized in that, The target classification loss uses cross-entropy loss, and the pixel-level segmentation loss uses binary cross-entropy loss.

Images