A method, system, and device for identifying anomalous targets in underwater images.

Abstract

This invention discloses a method, system, and device for identifying abnormal targets in underwater images. The invention includes using a pre-set abnormal object detection module to detect abnormal targets in the original underwater image within the object bounding box of a feature-fused image, generating preliminary detection results. Based on the depth information of the abnormal targets in the feature-fused image, a pre-set spatial distribution model is used to perform false positive suppression processing on the preliminary detection results, generating the final abnormal target detection result. This solves the technical problem of existing technologies lacking contextual information during enhancement, making it difficult to accurately detect abnormal targets in underwater images. By integrating image enhancement and object detection, this invention fuses features from the original image, enhanced image, and encoder, adapting to features at different scales, fully utilizing contextual information and predicted object depth information, reducing the impact of noise and scattering on the detection results, minimizing false positives, and improving the reliability of the detection results.

Description

Technical Field

[0001] This invention relates to the field of underwater image abnormal target recognition technology, and in particular to an underwater image abnormal target recognition method, system and device. Background Technology

[0002] Underwater image enhancement and anomaly detection are crucial technologies in fields such as underwater robotics, submersible cable inspection, and marine biological research. In underwater environments, due to the absorption and scattering properties of water, images are often affected by blurring and color distortion, reducing the accuracy of anomaly detection. Traditional image enhancement methods and single-network structures cannot achieve ideal detection performance in these complex environments.

[0003] Therefore, existing technologies typically employ Generative Adversarial Networks (GANs) for image enhancement and separate Convolutional Neural Networks (CNNs) for object detection. However, these methods have certain limitations, such as the potential loss of useful information during enhancement, resulting in a lack of contextual information, low detection accuracy, and difficulty in accurately detecting anomalous targets in underwater images. Summary of the Invention

[0004] This invention provides a method, system, and device for identifying abnormal targets in underwater images, which solves the technical problem that existing technologies may lose useful information during the enhancement process, resulting in a lack of contextual information, low detection accuracy, and difficulty in accurately detecting abnormal targets in underwater images.

[0005] The first aspect of this invention provides a method for identifying abnormal targets in underwater images, comprising:

[0006] In response to a received request for underwater image anomaly target identification, the original underwater image corresponding to the request is obtained;

[0007] The original underwater image is enhanced to generate an enhanced underwater image;

[0008] The original underwater image and the enhanced underwater image are convolved and their features are fused to generate a feature-fused image.

[0009] An abnormal object detection module is used to detect abnormal targets in the original underwater image within the object bounding box of the feature fusion image, and a preliminary detection result is generated.

[0010] Based on the depth information of abnormal targets in the feature fusion image, the preliminary detection results are processed to suppress false positives using a preset spatial distribution model, thereby generating abnormal target detection results.

[0011] Optionally, the step of enhancing the original underwater image to generate an enhanced underwater image includes:

[0012] The original underwater image is input into a preset initial image enhancement module;

[0013] The discriminator of the initial image enhancement module extracts features from the original underwater image to generate an updated underwater image;

[0014] The updated underwater image is used to train the initial image enhancement module to generate the target image enhancement module.

[0015] The generator of the target image enhancement module extracts features from the original underwater image to generate an enhanced underwater image.

[0016] Optionally, the discriminator includes multiple convolutional layers, downsampling layers, activation layers, normalization layers, pooling layers, and a first fully connected layer; the step of extracting features from the original underwater image through the discriminator of the initial image enhancement module to generate an updated underwater image includes:

[0017] The features of the original underwater image are extracted through multiple convolutional layers of the discriminator of the initial image enhancement module to generate a first feature image, which is then input into the downsampling layer.

[0018] The features of the first feature image are extracted through the downsampling layer, a second feature image is generated, and then input into the activation layer.

[0019] The second feature image is non-linearly transformed by the activation layer to generate a third feature image, which is then input into the normalization layer.

[0020] The third feature image is normalized by the normalization layer to generate a fourth feature image, which is then input into the pooling layer.

[0021] The pooling layer is used to perform pooling processing on the fourth feature image to generate a fifth feature image, which is then input into the first fully connected layer.

[0022] An updated underwater image is generated by connecting the features of the fifth feature image through the first fully connected layer.

[0023] Optionally, the generator includes an encoder and a decoder; the step of extracting features from the original underwater image through the generator of the target image enhancement module to generate an enhanced underwater image includes:

[0024] The features of the original underwater image are extracted by the convolutional layer corresponding to the encoder of the target image enhancement module, and a first enhanced feature image is generated and input into the activation layer of the encoder.

[0025] The first enhanced feature image is nonlinearly transformed through the activation layer of the encoder to generate a second enhanced feature image, which is then input into the normalization layer of the encoder.

[0026] The second enhanced feature image is normalized through the normalization layer of the encoder to generate a third enhanced feature image, which is then input into the skip connection and the convolutional layer of the decoder.

[0027] The third enhanced feature image is convolved by the convolutional layer of the decoder to generate a fourth enhanced feature image, which is then input into the transposed convolutional layer of the decoder.

[0028] The fourth enhanced feature image is upsampled by the transposed convolutional layer of the decoder to generate a fifth enhanced feature image, which is then input into the activation layer of the decoder.

[0029] The fifth enhanced feature image is nonlinearly transformed through the activation layer of the decoder to generate a sixth enhanced feature image, which is then input into the normalization layer of the decoder.

[0030] The sixth enhanced feature image is normalized through the normalization layer of the decoder to generate a seventh enhanced feature image, which is then input into the skip layer connection.

[0031] The second and seventh enhanced feature images are stitched together using the skip-layer connection to generate an enhanced underwater image.

[0032] Optionally, the step of performing convolution processing on the original underwater image and the enhanced underwater image respectively, and then performing feature fusion to generate a feature-fused image, includes:

[0033] The original underwater image and the enhanced underwater image are convolved by an adaptive dilated convolution module to generate a first convolution feature image and a second convolution feature image, which are then input into a dynamic feature fusion module.

[0034] The dynamic feature fusion module fuses the first convolutional feature image and the second convolutional feature image to generate a feature fused image.

[0035] Optionally, the step of fusing the first convolutional feature image and the second convolutional feature image through the dynamic feature fusion module to generate a feature fused image includes:

[0036] The dynamic feature fusion module performs mean pooling operations on the first convolutional feature of the first convolutional feature image, the second convolutional feature of the second convolutional feature image, and the third convolutional feature through multiple feature channels to generate the first pooling feature, the second pooling feature, and the third pooling feature, respectively.

[0037] The first pooling feature, the second pooling feature, and the third pooling feature are concatenated to generate an initial feature vector, which is then input into the second fully connected layer for connection. An updated feature vector is then generated and input into the softmax layer.

[0038] The updated feature vector is generated by performing a softmax operation on the softmax layer.

[0039] Multiple target vectors are generated by multiplying the value of the update vector by the number of feature channels of the feature corresponding to the update vector;

[0040] All the target vectors are concatenated to generate a feature fusion image.

[0041] Optionally, the step of using a preset abnormal object detection module to detect abnormal targets in the original underwater image within the object bounding box of the feature fusion image and generating preliminary detection results includes:

[0042] The feature-fused image is input into a preset region proposal network to generate multiple object bounding boxes;

[0043] The region proposal network predicts the object property of the feature fusion image within the object's bounding box, scores the object property of the feature fusion image, and generates an object property score.

[0044] Determine whether the objectivity score is greater than or equal to the score threshold;

[0045] If so, extract the object bounding box corresponding to the object quality score and the coordinates of the object bounding box corresponding to the object bounding box.

[0046] The object bounding boxes corresponding to the object property scores are optimized using a re-optimization module to generate optimized object bounding boxes;

[0047] The optimized object bounding box is used to detect the feature fusion image, generating region detection results;

[0048] The original underwater image is input into the abnormal object detection module;

[0049] The feature extractor of the abnormal object detection module extracts features from the original underwater image to generate a first abnormal object feature image.

[0050] The first abnormal object feature image is stitched together with the feature fusion image to generate a second abnormal object feature image, which is then input into the convolutional network.

[0051] The convolutional network is used to extract features from the feature image of the second abnormal object to generate attention detection results.

[0052] The attention detection result is multiplied by the confidence channel corresponding to the region detection result to generate a feature image of the target abnormal object;

[0053] Anomaly detection is performed on the target anomaly feature image based on the optimized object bounding box on the target anomaly feature image, generating preliminary detection results.

[0054] Optionally, the step of performing false positive suppression processing on the preliminary detection results using a preset spatial distribution model to generate abnormal target detection results includes:

[0055] Depth information of abnormal targets is detected from the feature fusion image using a regression method;

[0056] Based on the depth information of the abnormal target and the image pixel data within the optimized object bounding box on the feature fusion image, a preset spatial distribution model is constructed.

[0057] Analyze the spatial distribution of the abnormal targets in the preliminary detection results;

[0058] Determine whether the spatial distribution is greater than or equal to the distribution threshold of the preset spatial distribution model;

[0059] If not, the spatial distribution is identified as a false positive and the spatial distribution is removed.

[0060] If so, the spatial distribution of the abnormal target is determined, and multiple spatial distributions are combined to generate an abnormal target detection result.

[0061] A second aspect of the present invention provides an underwater image anomaly target recognition system, comprising:

[0062] The raw underwater image module is used to respond to a received request for underwater image abnormal target identification and to obtain the raw underwater image corresponding to the request for underwater image abnormal target identification.

[0063] An underwater image enhancement module is used to enhance the original underwater image to generate an enhanced underwater image;

[0064] The feature fusion image module is used to perform convolution processing on the original underwater image and the enhanced underwater image respectively, and to perform feature fusion to generate a feature fusion image;

[0065] The preliminary detection result module is used to perform abnormal target detection on the original underwater image within the object bounding box of the feature fusion image using a preset abnormal object detection module, and generate preliminary detection results.

[0066] The abnormal target detection result module is used to generate abnormal target detection results by performing false positive suppression processing on the preliminary detection results based on the depth information of abnormal targets in the feature fusion image and through a preset spatial distribution model.

[0067] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the underwater image abnormal target recognition method as described in any of the preceding claims.

[0068] As can be seen from the above technical solutions, the present invention has the following advantages:

[0069] This invention addresses the problem of existing technologies losing useful information during enhancement, resulting in low detection accuracy and difficulty in accurately detecting abnormal targets in underwater images. This is achieved by responding to a received request for abnormal target identification in an underwater image; enhancing the original underwater image to generate an enhanced underwater image; convolving both the original and enhanced underwater images and fusing their features to generate a fused feature image; using a pre-defined abnormal object detection module to detect abnormal targets in the original underwater image within the bounding boxes of the fused feature image, generating preliminary detection results; and applying a pre-defined spatial distribution model to the preliminary detection results based on the depth information of the abnormal targets in the fused feature image to suppress false positives, thus generating the final abnormal target detection result. This solution overcomes the technical problem that existing technologies may lose useful information during enhancement, leading to a lack of contextual information, low detection accuracy, and difficulty in accurately detecting abnormal targets in underwater images.

[0070] This invention integrates image enhancement and object detection, fusing features from the original image, enhanced image, and encoder. It can adapt to features at different scales, fully utilizes contextual information and predicted object depth information, reduces the impact of noise and scattering on detection results, reduces false positives, and improves the reliability of detection results. Attached Figure Description

[0071] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1This is a flowchart of the steps of an underwater image abnormal target recognition method provided in Embodiment 1 of the present invention;

[0073] Figure 2 This is a flowchart of the steps of an underwater image abnormal target recognition method provided in Embodiment 2 of the present invention;

[0074] Figure 3 This is a schematic diagram of the module operation flow of an underwater image abnormal target recognition method provided in Embodiment 2 of the present invention;

[0075] Figure 4 This is a structural block diagram of an underwater image abnormal target recognition system provided in Embodiment 3 of the present invention. Detailed Implementation

[0076] This invention provides a method, system, and device for identifying abnormal targets in underwater images, which addresses the technical problem that existing technologies may lose useful information during the enhancement process, resulting in a lack of contextual information, low detection accuracy, and difficulty in accurately detecting abnormal targets in underwater images.

[0077] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0078] Please see Figure 1 , Figure 1 This is a flowchart illustrating the steps of an underwater image abnormal target recognition method provided in Embodiment 1 of the present invention.

[0079] The present invention provides a method for identifying abnormal targets in underwater images, comprising the following steps:

[0080] Step 101: Respond to the received underwater image abnormal target identification request and obtain the original underwater image corresponding to the underwater image abnormal target identification request.

[0081] It should be noted that the request for underwater image anomaly target identification refers to the request to identify anomalies in underwater images captured by an underwater robot.

[0082] In practice, when a request for underwater image abnormality target identification is received, the original underwater image captured by the underwater robot is obtained to facilitate subsequent image enhancement or other image processing.

[0083] Step 102: Enhance the original underwater image to generate an enhanced underwater image.

[0084] In practice, enhancing the original underwater image includes, but is not limited to, extracting features from the original underwater image and performing activation and normalization processing on the image after feature extraction, thereby obtaining an enhanced underwater image.

[0085] Step 103: Perform convolution processing on the original underwater image and the enhanced underwater image respectively, and perform feature fusion to generate a feature fused image.

[0086] It should be noted that a feature fusion image refers to the feature fusion image obtained after fusing features from two feature images.

[0087] In practice, two adaptive dilated convolution modules are used to perform convolution processing on the original underwater image and the enhanced image respectively, generating two updated images. A dynamic feature fusion module is then used to fuse the features of the two updated images to obtain a feature-fused image.

[0088] Step 104: Use the preset abnormal object detection module to perform abnormal target detection on the original underwater image within the object bounding box of the feature fusion image, and generate preliminary detection results.

[0089] It should be noted that the preset anomalous object detection module refers to a context-aware anomalous object detection module. Specifically, it is an attention mechanism guided by predefined underwater object templates, focusing on areas more likely to contain anomalous objects against the background of a diving cable. The aforementioned templates represent the expected appearance of marine life, debris, and cable damage. Using fused features from a dynamic feature fusion module and attention-guided contextual information, anomalous targets are detected within refined region suggestions.

[0090] In practice, the depth information extracted from the feature fusion image and the image pixel data within the object bounding box are used to construct a preset abnormal object detection module. This module then performs abnormal target detection on the original underwater image within the object bounding box to obtain preliminary detection results.

[0091] Step 105: Based on the depth information of abnormal targets in the feature fusion image, perform false positive suppression processing on the preliminary detection results through a preset spatial distribution model to generate abnormal target detection results.

[0092] It should be noted that the abnormal target detection result refers to the detection result obtained after detecting abnormal targets in the underwater image. This detection result includes the depth information of the detected abnormal objects and their surrounding underwater environment.

[0093] False positive suppression refers to using a predefined spatial distribution model tailored to the underwater environment to reduce false positives in underwater images, which helps improve the overall accuracy of anomaly detection.

[0094] In practice, for the bounding box (object boundary box) of the preliminary detection result, the image pixel data inside it and the depth information of the abnormal target in the feature fusion image are cropped and normalized to obtain the distribution probability of the abnormal target in the spatial distribution model. The distribution probability corresponding to the spatial distribution of the abnormal target in the preliminary detection result is extracted. If the distribution probability is less than the distribution probability threshold of the abnormal target in the spatial distribution model, it is identified as a false positive, the spatial distribution is removed, and the detection result corresponding to the remaining spatial distribution is determined as the abnormal target detection result.

[0095] Please see Figure 2-3 , Figure 2 This is a flowchart illustrating the steps of an underwater image abnormal target recognition method provided in Embodiment 2 of the present invention.

[0096] The present invention provides a method for identifying abnormal targets in underwater images, comprising the following steps:

[0097] Step 201: Respond to the received underwater image abnormal target identification request and obtain the original underwater image corresponding to the underwater image abnormal target identification request.

[0098] In this embodiment of the invention, the specific implementation process of step 201 is similar to that of step 101, and will not be repeated here.

[0099] Step 202: Input the original underwater image into the preset initial image enhancement module.

[0100] It should be noted that the preset initial image enhancement module includes a discriminator and a generator. The generator adopts an encoder-decoder architecture with skip connections between corresponding layers.

[0101] In practice, the original underwater image is input into the initial image enhancement module for image enhancement processing.

[0102] Step 203: Extract features from the original underwater image using the discriminator of the initial image enhancement module to generate an updated underwater image.

[0103] Optionally, the discriminator includes multiple convolutional layers, downsampling layers, activation layers, normalization layers, pooling layers, and a first fully connected layer; step 203 includes the following steps S11-S16:

[0104] S11. Extract features from the original underwater image through multiple convolutional layers of the discriminator in the initial image enhancement module, generate the first feature image, and input it into the downsampling layer;

[0105] S12. Extract the features of the first feature image through the downsampling layer, generate the second feature image, and input it into the activation layer;

[0106] S13. The second feature image is non-linearly transformed through the activation layer to generate the third feature image, which is then input into the normalization layer.

[0107] S14. Normalize the third feature image through the normalization layer to generate the fourth feature image and input it into the pooling layer.

[0108] S15. The fourth feature image is pooled through a pooling layer to generate the fifth feature image, which is then input into the first fully connected layer.

[0109] S16. Connect the features of the fifth feature image through the first fully connected layer to generate an updated underwater image.

[0110] It should be noted that the discriminator is a convolutional neural network, consisting of multiple convolutional layers, downsampling layers (including pooling or convolutional layers with a stride greater than 1), activation layers, normalization layers, and a final pooling layer (reducing the spatial size to 1×1), one or more fully connected layers (reducing the feature channels from C to 1), and a sigmoid layer, outputting a single numerical confidence score. This is used to distinguish between real and generated images during network training, guiding the generator to produce enhanced images with quality and appearance similar to the clearer images. Specifically, the first and second fully connected layers in the above steps and in step S42 below are both fully connected layers.

[0111] In specific implementation, in the above steps, the first feature image refers to the feature image obtained after the original underwater image is processed by multiple convolutional layers of the discriminator; the second feature image refers to the feature image obtained after the first feature image is processed by a downsampling layer; the third feature image refers to the feature image obtained after the second feature image is processed by a non-linear transformation layer; the fourth feature image refers to the feature image obtained after the third feature image is processed by a normalization layer; the fifth feature image refers to the feature image obtained after the fourth feature image is processed by a pooling layer; and updating the underwater image refers to the feature image obtained after the fifth feature image is processed by a fully connected layer.

[0112] Step 204: Train the initial image enhancement module using updated underwater images to generate the target image enhancement module.

[0113] It should be noted that the target image enhancement module is formed by training a discriminator and then guiding the generator to generate an enhanced image with quality and appearance similar to the clear image.

[0114] In practice, the initial image enhancement module is trained by updating underwater images. The loss function for training the initial image enhancement module includes adversarial loss L. g Content loss L c and downstream task losses L f When the generator is G, the discriminator is D, and the original underwater image is x0, the ground truth value of its corresponding enhanced image is x. gt The specific formula is as follows

[0115] L g =-log(D(x) gt ))-log(1-D(G(x0))

[0116] L c =||G(x0)-x gt ||

[0117] Specifically, the downstream task loss is obtained by backpropagation of the loss from the detection results of the entire neural network framework. In the early stages of the overall training phase, the image enhancement module is trained separately using data from the degraded original underwater image and the enhanced underwater image. In the mid-to-late stages, it participates in the joint training of the entire framework to optimize the features output by its encoder. Specifically, refer to... Figure 3 ,by Figure 3 The loss from the object detection module and the loss from the depth estimation module are obtained through backpropagation. The specific formula is as follows:

[0118] L f =BP(L det :{Θ RPN ,Θ REF})+αBP(L dep :Θ DE )

[0119] In the formula, L det The target detection loss consists of two parts: the region proposal network loss and the boundary re-optimization loss. RPN For the parameters of the region proposal network, Θ REF L is the parameter of the boundary refinement network. dep The loss for depth estimation is typically the 2-norm between the predicted depth and the ground truth depth, or another loss function (specifically, the distance from each point in the depth image to the camera, i.e., D in RGBD, not referring to underwater depth), Θ DE The parameters are for depth estimation networks. BP stands for backpropagation, which is the conventional backpropagation of gradients using the chain rule.

[0120] In the early stages of network training, the image augmentation module is trained separately, i.e., only the adversarial loss L is used. G and content loss L c The specific formula is as follows:

[0121] L enh_early =L g +λL c

[0122] In the formula, L enh_early L is the loss function for the target image enhancement module in the early stages of training. g To combat the losses, L c This results in content loss.

[0123] The specific formula for participating in joint training in the mid-to-late stages is as follows:

[0124] L enh =L g +λ1L c +λ2L f

[0125] In the formula, L enh Let L be the loss function of the target image enhancement module in the later stages of training. g To combat the losses, L c For content loss, L f This results in losses for downstream tasks.

[0126] Step 205: Extract features from the original underwater image using the generator of the target image enhancement module to generate an enhanced underwater image.

[0127] Optionally, the generator includes an encoder and a decoder; step 205 includes the following steps S21-S28:

[0128] S21. Extract the features of the original underwater image through the convolutional layer corresponding to the encoder of the target image enhancement module, generate the first enhanced feature image, and input it into the activation layer of the encoder.

[0129] S22. The first enhanced feature image is processed by nonlinear transformation through the activation layer of the encoder to generate a second enhanced feature image, which is then input into the normalization layer of the encoder.

[0130] S23. The second enhanced feature image is normalized through the encoder's normalization layer to generate the third enhanced feature image, which is then input into the jump connection and the decoder's convolutional layer respectively.

[0131] S24. The third enhanced feature image is convolved through the convolutional layer of the decoder to generate the fourth enhanced feature image, which is then input into the transposed convolutional layer of the decoder.

[0132] S25. The fourth enhanced feature image is upsampled through the transposed convolutional layer of the decoder to generate the fifth enhanced feature image and input into the activation layer of the decoder.

[0133] S26. The fifth enhanced feature image is processed by nonlinear transformation through the activation layer of the decoder to generate the sixth enhanced feature image and input into the normalization layer of the decoder.

[0134] S27. The sixth enhanced feature image is normalized through the normalization layer of the decoder to generate the seventh enhanced feature image and input into the skip connection.

[0135] S28. The second enhanced feature image and the seventh enhanced feature image are stitched together by skip-layer connection to generate an enhanced underwater image.

[0136] It should be noted that the encoder consists of a series of convolutional layers with a stride of 1 or greater. The first convolutional layer can use, but is not limited to, a 7×7 kernel, and the number of output channels includes, but is not limited to, 128 or 256. Subsequent convolutional layers use 1×1 or 3×3 kernels. When the stride is 1, the feature space size remains unchanged. When the stride is 2 or higher (denoted as s), the feature space size shrinks to 1 / s, which is called downsampling. The number of channels increases with each feature downsampling by (but is not limited to) s or 2s times the previous feature size. During non-downsampling, the number of feature channels remains unchanged. Each convolutional layer is followed by an activation layer, which can use, but is not limited to, ReLU, Leaky ReLU, etc. Each activation layer is followed by a normalization layer, including but not limited to batch normalization, layer normalization, group normalization, or instance normalization. Let the convolutional layer be C, the activation layer be R, and the normalization layer be B. Then, for a certain feature x in a certain layer... i (x0 is the original underwater image) In this regard:

[0137] x i+1 =B i+1 (R i+1 (C i+1 (x i )))

[0138] In practice, the decoder consists of a series of convolutional layers, transposed convolutional layers, activation layers, and normalization layers. The stride of each convolutional layer is 1. The upsampling rate of the transposed convolutional layer corresponds mirror-to-downsampling rate of the encoder. For example, if the encoder contains three downsampling convolutional layers with downsampling rates of 1 / 4, 1 / 2, and 1 / 2, the decoder contains three transposed convolutional layers with upsampling rates of 2, 2, and 4. Each convolutional or transposed convolutional layer in the decoder is followed by an activation layer and a normalization layer.

[0139] Skip-layer connections involve taking intermediate features from the convolutional layers in the encoder and concatenating them on the corresponding scale of the decoder along the feature channels (see Unet).

[0140] In specific implementation, the first enhanced feature image refers to the feature image generated after the original underwater image undergoes convolutional feature extraction processing through the convolutional layer corresponding to the encoder; the second enhanced feature image refers to the feature image generated after the first enhanced feature image undergoes nonlinear transformation processing through the activation layer of the encoder; the third enhanced feature image refers to the feature image generated after the second enhanced feature image undergoes normalization processing through the normalization layer of the encoder; the fourth enhanced feature image refers to the feature image generated after the third enhanced feature image undergoes convolution processing through the convolutional layer of the decoder; the fifth enhanced feature image refers to the feature image generated after the fourth enhanced feature image undergoes upsampling processing through the transposed convolutional layer of the decoder; the sixth enhanced feature image refers to the feature image generated after the fifth enhanced feature image undergoes nonlinear transformation processing through the activation layer of the decoder; the seventh enhanced feature image refers to the feature image generated after the sixth enhanced feature image undergoes normalization processing through the normalization layer of the decoder; and the enhanced underwater image refers to the underwater image generated by stitching together the second and seventh enhanced feature images through a skip-layer connection.

[0141] Step 206: Perform convolution processing on the original underwater image and the enhanced underwater image respectively, and perform feature fusion to generate a feature fused image.

[0142] Optionally, step 206 includes the following steps S31-S32:

[0143] S31. The original underwater image and the enhanced underwater image are convolved by the adaptive dilated convolution module to generate the first convolution feature image and the second convolution feature image respectively, and then input into the dynamic feature fusion module.

[0144] S32. The first convolutional feature image and the second convolutional feature image are fused using the dynamic feature fusion module to generate a feature fused image.

[0145] It should be noted that in the above steps, adaptively dilated convolutional modules with different dilation rates are used to extract features from the original underwater image and the generated enhanced underwater image. Since the size of the object of interest in the image varies considerably depending on factors such as the object's size, camera distance, and viewing angle, a variable receptive field is needed to improve object detection performance. Adaptively dilated convolutional modules expand the receptive field of the convolution without increasing the number of parameters, enabling the network to capture contextual information at different scales. However, manually set dilation rates in adaptively dilated convolutional modules are difficult to adapt to the complex variations in real-world large datasets.

[0146] Therefore, in this embodiment of the invention, a certain convolutional layer is denoted as having C convolutional kernels, including c0 ordinary convolutional kernels and c1 convolutional kernels with variable dilation rates. Taking a certain step in network training as an example, forward propagation is performed using the dilation rates of the c1 dilated convolutional kernels from the previous step to calculate the loss function.

[0147] The dilation rates of some of the c1 dilated convolutional kernels are adjusted heuristically or in a random direction several times. Each time, the loss function is calculated, and the minimum loss function is taken as the optimization direction to find a better dilation rate setting. As the overall framework trains, once the loss function decreases below a set threshold, dilation rate optimization is stopped, and the configuration is fixed. The fixed configuration is then used to continue subsequent training, and object detection is performed during the inference phase. This allows the network to adaptively obtain the optimal receptive field size based on the data distribution. This adaptive dilated convolutional module helps the network capture more meaningful features in anomalous object detection tasks.

[0148] In practical implementation, after the two adaptive dilated convolutional modules extract features from the original underwater image and the enhanced underwater image respectively, the generated first convolutional feature F orig Second convolution feature F enh The features F extracted by the encoder part of the generator in the target image enhancement module. enc All data are input into the dynamic feature fusion module for feature fusion processing to generate a feature fused image.

[0149] Optionally, step S32 includes the following steps S41-S45:

[0150] S41. The first convolutional feature of the first convolutional feature image, the second convolutional feature of the second convolutional feature image, and the third convolutional feature are subjected to mean pooling operation through the multi-channel feature fusion module to generate the first pooling feature, the second pooling feature, and the third pooling feature, respectively.

[0151] S42. Concatenate the first pooling feature, the second pooling feature, and the third pooling feature to generate an initial feature vector and input it into the second fully connected layer for connection, generate an updated feature vector, and input it into the softmax layer.

[0152] S43. The updated feature vector is generated by performing a softmax operation on the softmax layer.

[0153] S44. Multiply the value of the update vector by the number of feature channels of the feature corresponding to the update vector to generate multiple target vectors;

[0154] S45. All target vectors are concatenated to generate a feature fusion image.

[0155] It should be noted that the three feature maps obtained from the encoder and adaptive dilated convolution module of the target image enhancement module's generator are used as input to the dynamic feature fusion module. These features are extracted at different levels of the two networks to capture multi-scale information. A series of attention-based gating mechanisms are employed to selectively combine the feature maps from the three sources. These gating mechanisms learn to weigh the importance of features from each source based on their relevance to the task.

[0156] In practical implementation, the first convolutional features F generated from the original underwater image and the enhanced underwater image are respectively... orig Second convolution feature F enh The features F extracted by the encoder part of the generator in the target image enhancement module. enc That is, the left, middle and right feature channels of the dynamic feature fusion module are respectively input to F orig F enh and F enc Let the number of channels in the three characteristic channels be C. orig C enh and C enc For F respectively orig F enh and F enc Mean pooling is performed in the spatial dimension to generate first, second, and third pooling features. These features are then reduced to a 1×1 spatial size and concatenated to obtain a feature of length C. orig +C enh +C enc The vector (i.e., the initial feature vector) is input into one or more fully connected layers, resulting in a vector of length n (i.e., the updated feature vector). Performing a softmax operation on this vector yields vector A (i.e., the updated vector), where vector A represents n gating mechanisms, where n = 1 + n. D Let n be the number of non-repeating dilation rates in the adaptively dilated convolutional module (for example, in an adaptively dilated convolutional module containing 32 convolutional kernels with a dilation rate of 1 (equivalent to ordinary convolutional kernels), 32 convolutional kernels with a dilation rate of 2, 16 convolutional kernels with a dilation rate of 4, and 16 convolutional kernels with a dilation rate of 8, then n) D =4, i.e., num({1,2,4,8})), where 1 corresponds to F. enc After obtaining vector A, the value of vector A is multiplied by the corresponding channel of the feature to generate multiple target vectors, which are then concatenated along the channel dimension.

[0157] F merge =concatenate({A[0]·F enc ,A[DR=1]·F orig [DR=1],A[DR=1]·Fen [DR=1],A[DR=2]·F orig [DR=2],A[DR=2]·F enh [DR=2],…})

[0158] In the formula, F merge For feature fusion, DR expansion rate, F...[] indicates indexing operation according to the conditions in [], and concatenate indicates concatenation operation in feature channels.

[0159] In practice, the output of the dynamic feature fusion module is a set of fused features, which includes information from the enhanced image and the original image, as well as information from the encoder of the target image enhancement module. The features with different receptive fields are fused after a gating mechanism is added, so that the subsequent network can obtain adaptive and better fused features and has adaptive attention to contextual information at different scales.

[0160] Step 207: Use a preset abnormal object detection module to perform abnormal target detection on the original underwater image within the object bounding box of the feature fusion image, and generate preliminary detection results.

[0161] Optionally, step 207 includes the following steps S51-S512:

[0162] S51. Input the feature fusion image into a preset region proposal network to generate multiple object bounding boxes;

[0163] S52. Predict the object property of the feature fusion image within the object bounding box using a region proposal network, score the object property of the feature fusion image, and generate an object property score.

[0164] S53. Determine whether the object-type score is greater than or equal to the score threshold.

[0165] S54. If so, extract the object bounding box corresponding to the object score and the object bounding box coordinates corresponding to the object bounding box.

[0166] S55. Use the re-optimization module to optimize the bounding boxes of objects corresponding to the object property scores, and generate optimized object bounding boxes.

[0167] S56. Use optimized object bounding boxes to detect the feature fusion image and generate region detection results;

[0168] S57. Input the original underwater image into the abnormal object detection module;

[0169] S58. Extract features from the original underwater image using the feature extractor of the abnormal object detection module to generate the first abnormal object feature image;

[0170] S59. The first abnormal object feature image is stitched together with the feature fusion image to generate the second abnormal object feature image and then input into the convolutional network.

[0171] S510. Extract features from the feature image of the second abnormal object through a convolutional network and generate attention detection results;

[0172] S511. Multiply the attention detection result with the confidence channel corresponding to the region detection result to generate a feature image of the target abnormal object;

[0173] S512. Perform abnormal object detection on the target abnormal object feature image according to the optimized object bounding box on the target abnormal object feature image, and generate preliminary detection results.

[0174] It should be noted that the pre-defined region proposal network generates region proposals by sliding a small network across the feature fusion image from the dynamic feature fusion module, predicting an object score and bounding box coordinates for each region proposal. The optimization module independently refines the bounding box coordinates of the region proposals using the fused features from the dynamic feature fusion module. This improves the localization of detected objects and helps the network focus on the most relevant regions in the anomalous object detection task.

[0175] In practice, the feature fusion image is input into a region proposal network (RPN). The RPN predicts anomalous targets based on the feature fusion and defines bounding boxes for the predicted anomalous targets. The RPN then predicts the object quality of the feature fusion image within the bounding boxes and scores the object quality to generate an object quality score. When the object quality score is greater than or equal to a preset threshold, the bounding box corresponding to the object quality score is extracted.

[0176] In practice, the object bounding box is refined using a re-optimization module, which generates an optimized object bounding box. The optimized object bounding box is then used to detect the feature fusion image, generating region detection results.

[0177] In practical implementation, let I be a set of original underwater images containing common underwater anomalies. tplt If the image size is h×w and the number of images is N, then I tplt The size is N×h×w, and it is input into a feature extractor E in the form of a convolutional neural network. t0 Control E t0 The downsampling rate will reduce its spatial size to approximately the same as F. merge The same, we get F tplt (That is, the first abnormal object feature image), F merge (Feature fusion image) and F tplt(That is, the first anomalous object feature image) is concatenated (i.e., feature channel splicing) to obtain the second anomalous object feature image, which is then input into another convolutional neural network E. t1 This yields an attention result A in the form of a heatmap. T . A T Multiplying the result by the confidence channel in the region detection result yields the feature image of the target anomalous object (the output of the region proposal and re-optimization module is the same as the output of various existing object detection methods, which is h). ′ ×w ′ ×(1+4+C), h ′ ×w′ represents the size of the original underwater image after downsampling by each module, 1 represents the confidence level, 4 represents the optimized object bounding box, and C represents the object category classification result, multiplied by the channel with "1".

[0178] In specific implementation, according to A T The target anomalous object feature image and the bounding box on the target anomalous object feature image are obtained by multiplying the confidence channel in the region detection results of the region proposal and re-optimization module. The information on the target anomalous object feature image within the bounding box is used to detect anomalous objects, thereby obtaining preliminary detection results.

[0179] Specifically, introducing common template attention can further improve the accuracy and generalization ability of the network in anomaly detection.

[0180] Step 208: Based on the depth information of abnormal targets in the feature fusion image, perform false positive suppression processing on the preliminary detection results through a preset spatial distribution model to generate abnormal target detection results.

[0181] Optionally, step 208 includes the following steps S61-S66:

[0182] S61. Detect depth information of abnormal targets from feature fusion images using regression methods;

[0183] S62. Based on the depth information and feature fusion of abnormal targets, optimize the image pixel data within the bounding box of the object in the image and construct a preset spatial distribution model;

[0184] S63. Analyze the spatial distribution of abnormal targets in the preliminary detection results;

[0185] S64. Determine whether the spatial distribution is greater than or equal to the distribution threshold of the preset spatial distribution model;

[0186] S65. If not, then the spatial distribution is identified as a false positive and the spatial distribution is removed.

[0187] S66. If so, determine the spatial distribution of the abnormal target and combine multiple spatial distributions to generate the abnormal target detection result.

[0188] It should be noted that in the above steps, the depth of the detected anomalous targets is estimated using fused features from the dynamic feature fusion module. Depth estimation employs a regression-based method, using a neural network similar to a fully convolutional network (FCN) to extract the depth of each output point and sample the object depth from the predicted anomalous target bounding box. The estimated depth information provides additional contextual information to the false positive suppression module based on rejection sampling, helping it to better distinguish between true and false positives. Simultaneously, depth estimation can provide a loss function for optimizing the network parameters of preceding modules. Furthermore, the depth information of the detected anomalous targets and their surrounding underwater environment is itself information of interest, providing users with greater awareness of the scene.

[0189] In specific implementation, the preset spatial distribution setting method is as follows: the internal RGB and depth data of the abnormal target instance are clipped by the optimized object bounding box, the preset spatial distribution model is constructed in the normalized RGBD space, and the probability distribution (that is, the spatial distribution) is obtained.

[0190] The bounding boxes (i.e., spatial distributions) of predicted anomalous targets in the preliminary detection results are analyzed. Their internal RGB and depth data are cropped, normalized, and their probabilities are sampled from the distribution. If the probability is less than a set threshold (distribution threshold), it is considered a false positive. The spatial distributions corresponding to false positives are removed, thus determining the spatial distribution of anomalous targets. Finally, by combining these spatial distributions, the anomalous target detection results are generated.

[0191] This invention responds to a received request for underwater image anomaly target identification by acquiring the original underwater image corresponding to the request; enhancing the original underwater image to generate an enhanced underwater image; performing convolution processing on both the original and enhanced underwater images, and then fusing features to generate a feature fusion image; using a pre-defined anomaly detection module, detecting anomalies in the original underwater image within the object bounding boxes of the feature fusion image to generate preliminary detection results; and based on the depth information of the anomalies in the feature fusion image, using a pre-defined spatial distribution model to suppress false positives in the preliminary detection results, generating the final anomaly target detection result. This solves the technical problem of existing technologies that may lose useful information during the enhancement process, resulting in a lack of contextual information, low detection accuracy, and difficulty in accurately detecting anomalies in underwater images.

[0192] This invention integrates image enhancement and object detection, fusing features from the original image, enhanced image, and encoder. It can adapt to features at different scales, fully utilizes contextual information and predicted object depth information, reduces the impact of noise and scattering on detection results, reduces false positives, and improves the reliability of detection results.

[0193] Please see Figure 4 , Figure 4 This is a structural block diagram of an underwater image abnormal target recognition system provided in Embodiment 3 of the present invention.

[0194] This invention provides an underwater image anomaly target recognition system, comprising:

[0195] The raw underwater image module 401 is used to respond to the received underwater image abnormal target identification request and obtain the raw underwater image corresponding to the underwater image abnormal target identification request.

[0196] The underwater image enhancement module 402 is used to enhance the original underwater image to generate an enhanced underwater image.

[0197] The feature fusion image module 403 is used to perform convolution processing on the original underwater image and the enhanced underwater image respectively, and to perform feature fusion to generate a feature fusion image;

[0198] The preliminary detection result module 404 is used to perform abnormal target detection on the original underwater image within the object bounding box of the feature fusion image using a preset abnormal object detection module, and generate preliminary detection results.

[0199] The abnormal target detection result module 405 is used to generate abnormal target detection results by performing false positive suppression processing on the preliminary detection results based on the depth information of abnormal targets in the feature fusion image and through a preset spatial distribution model.

[0200] Optionally, the underwater imaging enhancement module 402 includes:

[0201] The initial image enhancement submodule is used to input the original underwater image into the preset initial image enhancement module;

[0202] The underwater image update submodule is used to extract features from the original underwater image through the discriminator of the initial image enhancement module and generate an updated underwater image.

[0203] The target image enhancement submodule is used to train the initial image enhancement module with updated underwater images to generate the target image enhancement module;

[0204] The underwater image enhancement submodule is used to extract features from the original underwater image through the generator of the target image enhancement module to generate an enhanced underwater image.

[0205] Optionally, the discriminator includes multiple convolutional layers, downsampling layers, activation layers, normalization layers, pooling layers, and a first fully connected layer; the underwater image update submodule includes:

[0206] The downsampling layer submodule is used to extract features of the original underwater image through multiple convolutional layers of the discriminator of the initial image enhancement module, generate a first feature image, and input it into the downsampling layer.

[0207] The activation submodule is used to extract features from the first feature image through the downsampling layer, generate a second feature image, and input it into the activation layer.

[0208] The input normalization layer submodule is used to perform nonlinear transformation processing on the second feature image through the activation layer to generate the third feature image and input it into the normalization layer;

[0209] The pooling layer submodule is used to normalize the third feature image through the normalization layer, generate the fourth feature image, and input it into the pooling layer.

[0210] The first fully connected layer submodule is used to perform pooling processing on the fourth feature image through the pooling layer to generate the fifth feature image and input it into the first fully connected layer;

[0211] The module for generating updated underwater images is used to connect the features of the fifth feature image through the first fully connected layer to generate updated underwater images.

[0212] Optionally, the generator includes an encoder and a decoder; the underwater image enhancement submodule includes:

[0213] The first encoder submodule is used to extract features of the original underwater image through the convolutional layer corresponding to the encoder of the target image enhancement module, generate a first enhanced feature image, and input it into the activation layer of the encoder.

[0214] The second encoder submodule is used to perform nonlinear transformation processing on the first enhanced feature image through the activation layer of the encoder to generate a second enhanced feature image and input it into the normalization layer of the encoder.

[0215] The convolutional layer submodule is used to normalize the second enhanced feature image through the encoder's normalization layer, generate the third enhanced feature image, and input them into the convolutional layers of the skip connection and decoder respectively.

[0216] The device's convolutional layer submodule is used to convolve the third enhanced feature image through the decoder's convolutional layer to generate a fourth enhanced feature image, which is then input into the decoder's transposed convolutional layer.

[0217] The activation layer submodule is used to upsample the fourth enhanced feature image through the transposed convolutional layer of the decoder, generate the fifth enhanced feature image, and input it into the activation layer of the decoder.

[0218] The normalization layer submodule is used to perform non-linear transformation processing on the fifth enhanced feature image through the activation layer of the decoder to generate the sixth enhanced feature image and input it into the normalization layer of the decoder.

[0219] The skip-layer connection submodule is used to normalize the sixth enhanced feature image through the normalization layer of the decoder, generate the seventh enhanced feature image, and input it into the skip-layer connection.

[0220] The submodule for generating enhanced underwater images is used to stitch together the second and seventh enhanced feature images through skip connections to generate enhanced underwater images.

[0221] Optionally, the feature fusion image module 403 includes:

[0222] The dynamic fusion submodule is used to perform convolution processing on the original underwater image and the enhanced underwater image respectively through the adaptive dilated convolution module, and generate the first convolution feature image and the second convolution feature image respectively, which are then input into the dynamic feature fusion module.

[0223] The feature fusion image submodule is used to fuse the first convolutional feature image and the second convolutional feature image through the dynamic feature fusion module to generate a feature fusion image.

[0224] Optionally, the feature fusion image submodule includes:

[0225] The third pooling feature submodule is used to perform mean pooling operations on the first convolutional feature of the first convolutional feature image, the second convolutional feature of the second convolutional feature image, and the third convolutional feature through the multiple feature channels of the dynamic feature fusion module, respectively, to generate the first pooling feature, the second pooling feature, and the third pooling feature.

[0226] The softmax layer submodule is used to concatenate the first pooling feature, the second pooling feature, and the third pooling feature to generate an initial feature vector, which is then fed into the second fully connected layer for connection, generating an updated feature vector, which is then fed into the softmax layer.

[0227] The update vector submodule is used to perform a softmax operation on the update feature vector through a softmax layer to generate an update vector.

[0228] The target vector submodule is used to generate multiple target vectors by multiplying the value of the update vector by the number of feature channels of the feature corresponding to the update vector.

[0229] The feature fusion image submodule is used to concatenate all target vectors to generate a feature fusion image.

[0230] Optionally, the preliminary detection results module 404 includes:

[0231] The object bounding box submodule is used to input the feature fusion image into a preset region proposal network to generate multiple object bounding boxes.

[0232] The object property score submodule is used to predict the object property of the feature fusion image within the object bounding box through the region proposal network, and to score the object property of the feature fusion image to generate an object property score.

[0233] The score threshold submodule is used to determine whether the object's score is greater than or equal to the score threshold.

[0234] The Object Bounding Box Coordinates submodule is used to extract the object bounding box corresponding to the object nature score and the object bounding box coordinates corresponding to the object bounding box if the condition is met.

[0235] The Optimize Object Bounding Box submodule is used to optimize the object bounding boxes corresponding to the object property scores using the Re-optimization module, and generate optimized object bounding boxes.

[0236] The region detection results submodule is used to detect the feature fusion image using optimized object bounding boxes and generate region detection results.

[0237] The abnormal object detection module submodule is used to input the raw underwater image into the abnormal object detection module;

[0238] The first anomalous object feature image submodule is used to extract features from the original underwater image through the feature extractor of the anomalous object detection module and generate the first anomalous object feature image.

[0239] The convolutional network submodule is used to stitch the first abnormal object feature image with the feature fusion image to generate a second abnormal object feature image and input it into the convolutional network.

[0240] The attention detection result submodule is used to extract features from the feature image of the second abnormal object through a convolutional network and generate attention detection results.

[0241] The target anomalous object feature image submodule is used to multiply the attention detection result with the confidence channel corresponding to the region detection result to generate the target anomalous object feature image;

[0242] The preliminary detection results submodule is used to perform abnormal object detection on the target abnormal object feature image according to the optimized object bounding box on the target abnormal object feature image, and generate preliminary detection results.

[0243] Optionally, the abnormal target detection result module 405 includes:

[0244] The depth information submodule is used to detect the depth information of abnormal targets from the feature fusion image using a regression method;

[0245] A submodule is constructed to build a preset spatial distribution model based on the image pixel data within the object bounding box in the image, which is optimized by fusing depth information and features of the abnormal target.

[0246] The analysis submodule is used to analyze the spatial distribution of abnormal targets in the preliminary detection results;

[0247] The distribution threshold submodule is used to determine whether the spatial distribution is greater than or equal to the distribution threshold of the preset spatial distribution model;

[0248] The spatial distribution removal submodule is used to identify spatial distributions as false positives and remove them if the condition is not met.

[0249] The abnormal target detection result submodule is used to determine the spatial distribution of abnormal targets if the condition is met, and to generate abnormal target detection results by combining multiple spatial distributions.

[0250] Embodiment 4 of the present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the underwater image abnormal target recognition system method as described in any of the above embodiments.

[0251] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0252] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0253] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0254] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0255] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0256] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying abnormal targets in underwater images, characterized in that, include: In response to a received request for underwater image anomaly target identification, the original underwater image corresponding to the request is obtained; The original underwater image is enhanced to generate an enhanced underwater image; The original underwater image and the enhanced underwater image are convolved and their features are fused to generate a feature-fused image. An abnormal object detection module is used to detect abnormal targets in the original underwater image within the object bounding box of the feature fusion image, and a preliminary detection result is generated. Based on the depth information of abnormal targets in the feature fusion image, the preliminary detection results are processed to suppress false positives using a preset spatial distribution model to generate abnormal target detection results. The step of using a preset abnormal object detection module to detect abnormal targets in the original underwater image within the object bounding box of the feature fusion image and generating preliminary detection results includes: The feature-fused image is input into a preset region proposal network to generate multiple object bounding boxes; The region proposal network predicts the object property of the feature fusion image within the object's bounding box, scores the object property of the feature fusion image, and generates an object property score. Determine whether the objectivity score is greater than or equal to the score threshold; If so, extract the object bounding box corresponding to the object quality score and the coordinates of the object bounding box corresponding to the object bounding box. The object bounding boxes corresponding to the object property scores are optimized using a re-optimization module to generate optimized object bounding boxes; The optimized object bounding box is used to detect the feature fusion image, generating region detection results; The original underwater image is input into the abnormal object detection module; The feature extractor of the abnormal object detection module extracts features from the original underwater image to generate a first abnormal object feature image. The first abnormal object feature image is stitched together with the feature fusion image to generate a second abnormal object feature image, which is then input into the convolutional network. The convolutional network is used to extract features from the feature image of the second abnormal object to generate attention detection results. The attention detection result is multiplied by the confidence channel corresponding to the region detection result to generate a feature image of the target abnormal object; Anomaly detection is performed on the target anomaly feature image based on the optimized object bounding box on the target anomaly feature image, generating preliminary detection results.

2. The underwater image abnormal target recognition method according to claim 1, characterized in that, The step of enhancing the original underwater image to generate an enhanced underwater image includes: The original underwater image is input into a preset initial image enhancement module; The discriminator of the initial image enhancement module extracts features from the original underwater image to generate an updated underwater image; The updated underwater image is used to train the initial image enhancement module to generate the target image enhancement module. The generator of the target image enhancement module extracts features from the original underwater image to generate an enhanced underwater image.

3. The underwater image abnormal target recognition method according to claim 2, characterized in that, The discriminator includes multiple convolutional layers, downsampling layers, activation layers, normalization layers, pooling layers, and a first fully connected layer; the step of extracting features from the original underwater image through the discriminator of the initial image enhancement module to generate an updated underwater image includes: The features of the original underwater image are extracted through multiple convolutional layers of the discriminator of the initial image enhancement module to generate a first feature image, which is then input into the downsampling layer. The features of the first feature image are extracted through the downsampling layer, a second feature image is generated, and then input into the activation layer. The second feature image is non-linearly transformed by the activation layer to generate a third feature image, which is then input into the normalization layer. The third feature image is normalized by the normalization layer to generate a fourth feature image, which is then input into the pooling layer. The pooling layer is used to perform pooling processing on the fourth feature image to generate a fifth feature image, which is then input into the first fully connected layer. An updated underwater image is generated by connecting the features of the fifth feature image through the first fully connected layer.

4. The underwater image abnormal target recognition method according to claim 2, characterized in that, The generator includes an encoder and a decoder; the step of extracting features from the original underwater image through the generator of the target image enhancement module to generate an enhanced underwater image includes: The features of the original underwater image are extracted by the convolutional layer corresponding to the encoder of the target image enhancement module, and a first enhanced feature image is generated and input into the activation layer of the encoder. The first enhanced feature image is nonlinearly transformed through the activation layer of the encoder to generate a second enhanced feature image, which is then input into the normalization layer of the encoder. The second enhanced feature image is normalized through the normalization layer of the encoder to generate a third enhanced feature image, which is then input into the skip connection and the convolutional layer of the decoder. The third enhanced feature image is convolved by the convolutional layer of the decoder to generate a fourth enhanced feature image, which is then input into the transposed convolutional layer of the decoder. The fourth enhanced feature image is upsampled by the transposed convolutional layer of the decoder to generate a fifth enhanced feature image, which is then input into the activation layer of the decoder. The fifth enhanced feature image is nonlinearly transformed through the activation layer of the decoder to generate a sixth enhanced feature image, which is then input into the normalization layer of the decoder. The sixth enhanced feature image is normalized through the normalization layer of the decoder to generate a seventh enhanced feature image, which is then input into the skip layer connection. The second and seventh enhanced feature images are stitched together using the skip-layer connection to generate an enhanced underwater image.

5. The underwater image abnormal target recognition method according to claim 1, characterized in that, The steps of performing convolution processing on the original underwater image and the enhanced underwater image respectively, and then performing feature fusion to generate a feature-fused image, include: The original underwater image and the enhanced underwater image are convolved by an adaptive dilated convolution module to generate a first convolution feature image and a second convolution feature image, which are then input into a dynamic feature fusion module. The dynamic feature fusion module fuses the first convolutional feature image and the second convolutional feature image to generate a feature fused image.

6. The underwater image abnormal target recognition method according to claim 5, characterized in that, The step of fusing the first convolutional feature image and the second convolutional feature image through the dynamic feature fusion module to generate a feature fused image includes: The dynamic feature fusion module performs mean pooling operations on the first convolutional feature of the first convolutional feature image, the second convolutional feature of the second convolutional feature image, and the third convolutional feature through multiple feature channels to generate the first pooling feature, the second pooling feature, and the third pooling feature, respectively. The first pooling feature, the second pooling feature, and the third pooling feature are concatenated to generate an initial feature vector, which is then input into the second fully connected layer for connection. An updated feature vector is then generated and input into the softmax layer. The updated feature vector is generated by performing a softmax operation on the softmax layer. Multiple target vectors are generated by multiplying the value of the update vector by the number of feature channels of the feature corresponding to the update vector; All the target vectors are concatenated to generate a feature fusion image.

7. The underwater image abnormal target recognition method according to claim 1, characterized in that, The step of performing false positive suppression processing on the preliminary detection results using a preset spatial distribution model to generate abnormal target detection results includes: Depth information of abnormal targets is detected from the feature fusion image using a regression method; Based on the depth information of the abnormal target and the image pixel data within the optimized object bounding box on the feature fusion image, a preset spatial distribution model is constructed. Analyze the spatial distribution of the abnormal targets in the preliminary detection results; Determine whether the spatial distribution is greater than or equal to the distribution threshold of the preset spatial distribution model; If not, the spatial distribution is identified as a false positive and the spatial distribution is removed. If so, the spatial distribution of the abnormal target is determined, and multiple spatial distributions are combined to generate an abnormal target detection result.

8. An underwater image anomaly target recognition system, characterized in that, include: The raw underwater image module is used to respond to a received request for underwater image abnormal target identification and to obtain the raw underwater image corresponding to the request for underwater image abnormal target identification. An underwater image enhancement module is used to enhance the original underwater image to generate an enhanced underwater image; The feature fusion image module is used to perform convolution processing on the original underwater image and the enhanced underwater image respectively, and to perform feature fusion to generate a feature fusion image; The preliminary detection result module is used to perform abnormal target detection on the original underwater image within the object bounding box of the feature fusion image using a preset abnormal object detection module, and generate preliminary detection results. The abnormal target detection result module is used to perform false positive suppression processing on the preliminary detection result based on the depth information of abnormal targets in the feature fusion image and through a preset spatial distribution model to generate abnormal target detection results; The preliminary detection results module includes: The object bounding box submodule is used to input the feature fusion image into a preset region proposal network to generate multiple object bounding boxes. The object property score submodule is used to predict the object property of the feature fusion image within the object bounding box through the region proposal network, and to score the object property of the feature fusion image to generate an object property score. The score threshold submodule is used to determine whether the objectivity score is greater than or equal to the score threshold; The object bounding box coordinates submodule is used to extract the object bounding box corresponding to the object property score and the object bounding box coordinates corresponding to the object bounding box if the object property score is true. The optimized object bounding box submodule is used to optimize the object bounding box corresponding to the object property score using the re-optimization module, thereby generating an optimized object bounding box. The region detection result submodule is used to detect the feature fusion image using the optimized object bounding box and generate region detection results; An abnormal object detection module submodule is used to input the original underwater image into the abnormal object detection module; The first anomalous object feature image submodule is used to extract features from the original underwater image through the feature extractor of the anomalous object detection module to generate the first anomalous object feature image. The convolutional network submodule is used to stitch the first abnormal object feature image with the feature fusion image to generate a second abnormal object feature image and input it into the convolutional network. The attention detection result submodule is used to extract features from the feature image of the second abnormal object through the convolutional network and generate attention detection results. The target anomalous object feature image submodule is used to multiply the attention detection result with the confidence channel corresponding to the region detection result to generate a target anomalous object feature image; The preliminary detection result submodule is used to perform abnormal object detection on the target abnormal object feature image according to the optimized object bounding box on the target abnormal object feature image, and generate preliminary detection results.

9. An electronic device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the underwater image abnormal target recognition method as described in any one of claims 1-7.

Images