A method for underwater salient object detection based on multimodal task-driven enhancement

Abstract

The present invention provides an underwater salient object detection method based on multimodal task-driven enhancement, comprising: establishing a medium transmission branch through a convolution module guided by a generalized prior, convolving the original image through the medium transmission branch to obtain a medium transmission map; calculating the global background light of the original image through a quadtree algorithm to obtain an underwater enhancement map; calculating the ambient light in the original image based on a generalized dark channel prior algorithm to obtain a medium transmission map; calculating the mean square error loss for the medium transmission map and the medium transmission map, guiding the medium transmission branch through the mean square error loss; performing cross-modal feature fusion and multi-stream feature extraction on the underwater enhancement map and the medium transmission map to obtain a saliency map. The present invention can improve the adaptability to specific underwater environments, and ensure that the enhanced underwater image is directly applicable to USOD tasks, thereby improving the accuracy and robustness of detection.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to an underwater salient object detection method based on multimodal task-driven enhancement. Background Art

[0002] In order to deal with the inherent problems of underwater images such as blur and color cast, many underwater image enhancement methods have been developed. Some techniques pre-process the input image using image enhancement methods to provide enhanced images for subsequent detection methods. However, these methods may be subject to two limitations: the enhanced image is usually not what the target detector wants. Experiments show that in some underwater environments, traditional image enhancement methods that pre-process the input image may degrade the performance of the detector because these traditional methods ignore potential information that helps with target detection and sometimes even produce distorted images that mislead the detector; additional datasets are required. Image enhancement methods that are jointly trained with detectors usually require additional paired image enhancement datasets to support the training of the enhancement module. However, some detection tasks cannot access these additional datasets to use these methods.

[0003] Although there are currently a variety of salient object detection (RGBD-SOD) networks that utilize RGBD information, their application in underwater environments still faces major challenges, especially in the acquisition and utilization of depth information. Most existing methods only use RGB images for salient object detection, and the few methods that consider depth information use depth information as a preprocessing step instead of integrating it into the detection process for collaborative training, which limits the performance and adaptability of the model in complex underwater environments. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides an underwater salient object detection method based on multimodal task-driven enhancement.

[0005] The present invention provides an underwater salient object detection method based on multimodal task-driven enhancement, comprising:

[0006] S1: establishing a medium transmission branch through a generalized prior-guided convolution module, and convolving the original image through the medium transmission branch to obtain a medium transmission map;

[0007] S2: Calculate the global background light of the original image by using a quadtree algorithm to obtain an underwater enhanced image;

[0008] S3: Calculate the ambient light in the original image based on the generalized dark channel prior algorithm to obtain the medium transmission map;

[0009] S4: calculating a mean square error loss for the medium transmission map and the medium transmission map, and guiding the medium transmission branch by the mean square error loss;

[0010] S5: performing cross-modal feature fusion and multi-stream feature extraction on the underwater enhancement image and the medium transmission image to obtain a saliency map.

[0011] According to the underwater salient object detection method based on multimodal task-driven enhancement provided by the present invention, step S2 further includes:

[0012] S21: Calculate the global background light of the original image through the quadtree algorithm;

[0013] S22: restoring the original image through the medium transmission image and the global background light to obtain an underwater enhanced image.

[0014] According to the underwater salient object detection method based on multimodal task-driven enhancement provided by the present invention, step S21 further includes:

[0015] S221: Divide the original image into four primary rectangular blocks;

[0016] S222: for each first-level rectangular block, respectively calculate the pixel value mean and pixel value variance of the current first-level rectangular block, and obtain the statistical difference of the pixel value mean and pixel value variance of each first-level rectangular block;

[0017] S223: dividing the first-level rectangular blocks with the largest statistical difference among all the first-level rectangular blocks into four second-level rectangular blocks, calculating the statistical difference of each second-level rectangular block, and selecting the second-level rectangular blocks with the largest statistical difference for further equal division until the size of the equally divided rectangular blocks is smaller than a preset threshold, thereby obtaining a plurality of secondary rectangular blocks;

[0018] S224: Select a maximum pixel value rectangular block with extremely large pixel values ​​from the multiple secondary rectangular blocks, and use the maximum pixel value of the maximum pixel value rectangular block as the global background light of the original image.

[0019] According to the underwater salient object detection method based on multimodal task-driven enhancement provided by the present invention, the expression of the ambient light calculated in step S3 is:

[0020]

[0021] in, The ambient light calculated by the generalized dark channel prior algorithm is is the pixel index in the original image, Pixels in the original image Background light at It is the set of pixel positions before 0.1% of the maximum pixel value in the original image.

[0022] According to a method for underwater salient object detection based on multimodal task-driven enhancement provided by the present invention, the expression of the medium transmission graph in step S3 is:

[0023]

[0024] in, To estimate the transmission diagram of the obtained medium, is the color channel, Pixel The local area centered on For the first The pixel points in the local area centered on the pixel index, is the pixel point in the original image Background light.

[0025] According to the underwater salient object detection method based on multimodal task-driven enhancement provided by the present invention, the expression of the mean square error loss in step S4 is:

[0026]

[0027] in, is the mean square error loss calculated for the medium transmission map and the medium transmission map, is the total number of pixels, is the pixel coordinate, Transmission map for the medium at pixel points The value at For the medium transmission map in pixels The value at .

[0028] According to a method for underwater salient object detection based on multimodal task-driven enhancement provided by the present invention, in step S5, the step of performing cross-modal feature fusion of the underwater enhancement image and the medium transmission image further includes:

[0029] S51: mapping the underwater enhancement image and the medium transmission image into a first feature image and a second feature image respectively through convolution, wherein the first feature image corresponds to the underwater enhancement image, and the second feature image corresponds to the medium transmission image;

[0030] S52: Aggregating different domain features of the first feature map and the second feature map through one-dimensional channel attention and two-dimensional spatial attention to obtain a first enhanced feature corresponding to the first feature map and a second enhanced feature corresponding to the second feature map;

[0031] S53: performing element-wise multiplication and maximization on the first enhanced feature and the second enhanced feature in sequence to obtain a fusion feature.

[0032] According to the underwater salient object detection method based on multimodal task-driven enhancement provided by the present invention, the expression of the fusion feature in step S53 is:

[0033]

[0034]

[0035]

[0036]

[0037]

[0038] in, is the output fusion feature, is a convolutional layer with a convolution kernel size of 3, For the connection operation, is the first feature map, is the second feature map, is the one-dimensional channel attention, is the two-dimensional spatial attention, is the first enhanced feature, is the second enhanced feature, To take the maximum value operation in the feature map, is element-wise multiplication, The features obtained by maximizing the first enhancement feature and the second enhancement feature, The feature is obtained by multiplying the first enhanced feature and the second enhanced feature element by element.

[0039] According to the underwater salient object detection method based on multimodal task-driven enhancement provided by the present invention, the input for multi-stream feature extraction in step S5 includes the fused feature, the underwater enhancement image, and the medium transmission image.

[0040] According to the underwater salient object detection method based on multimodal task-driven enhancement provided by the present invention, the loss expression of the saliency map in step S5 is:

[0041]

[0042]

[0043]

[0044] in, is the loss corresponding to the saliency map, is the mean square error loss calculated for the medium transmission map and the medium transmission map, is the segmentation loss for outputting the segmentation results of salient objects, is the loss corresponding to the underwater enhancement map, is the binary cross entropy loss, is the intersection-and-join loss, is the pixel space loss, Assume loss for the grey world.

[0045] The present invention provides an underwater salient object detection method based on multimodal task-driven enhancement, and proposes a MES-Net (Multimodal Enhance Salient Net) model, which improves the existing technology from the perspective of multimodal information extraction. Compared with the traditional RGB-USOD network, MES-Net improves the accuracy and robustness of underwater salient target detection by combining underwater image enhancement technology and medium transmission information. Different from other USOD models that consider image enhancement, MES-Net does not simply use enhanced images as input, but adopts a generalized dark channel prior method to obtain additional medium transmission information from a single RGB image and trains it collaboratively, so that the model can supervise the medium transmission information generated by the medium transmission branch without the need for additional images, and at the same time, underwater image enhancement and saliency detection can be jointly optimized in the same network architecture.

[0046] This integration method not only enhances the adaptability of the model to the specific underwater environment, but also ensures that the enhanced underwater images are directly applicable to the USOD task, thereby improving the accuracy and robustness of detection. Through this innovative network structure and training method, MES-Net significantly improves the performance of salient object detection in complex underwater environments.

[0047] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 It is a flow chart of a method for underwater salient object detection based on multimodal task-driven enhancement provided by the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0051] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0052] Combine the following Figure 1 Embodiments of the present invention are described.

[0053] The present invention provides an underwater salient object detection method based on multimodal task-driven enhancement, comprising:

[0054] S1: establishing a medium transmission branch through a generalized prior-guided convolution module, and convolving the original image through the medium transmission branch to obtain a medium transmission map;

[0055] The impact of image quality degradation under adverse weather conditions can be described by a physical model of image formation, which represents degraded images and is widely used in image dehazing and underwater image restoration algorithms.

[0056] According to the model, the key to restoring the original image is to accurately estimate the medium transmission map and the global background light. Therefore, the present invention establishes an underwater enhancement module in step S1, and divides the established underwater enhancement module into a medium transmission branch for step S1 and a global background light branch for step S2. The above-mentioned medium transmission map and global background light are obtained through the designed medium transmission branch and global illumination branch.

[0057] The medium transmission branch uses a deep learning network to estimate the transmittance by analyzing the local and global features of the input image and generalization prior guidance, while the global illumination branch determines the intensity of the background light through a quadtree subdivision method.

[0058] In this stage, the original image is convolved based on the guided filtering method to obtain a medium transmission map. Specifically, the ResNet network is first selected as the backbone network. Based on the multi-head attention mechanism, the original image is convolved through multiple batch normalization layers and multiple convolution layers of the backbone network to obtain a convolution map. The convolution map is smoothed by the guided filtering method to obtain the medium transmission map.

[0059] Furthermore, the medium transmission map required for underwater image enhancement is generated by a convolutional module guided by a generalized prior. The module is called the medium transmission branch. The backbone network of the medium transmission branch consists of three residual convolutional layers embedded with large kernel convolutions. The branch is supervised by the generalized dark prior method and the loss feedback of the subsequent SOD network.

[0060] First, an efficient medium transmission branch backbone network is constructed by embedding the first three layers of ResNet50 with large kernel convolution. ResNet, with its residual connection characteristics, can effectively avoid the gradient vanishing problem in deep networks, which makes the network more stable during training.

[0061] Each encoder contains multiple residual blocks. In each residual block, not only batch normalization (BN) is used to optimize the training process, but also the ReLU activation function is used to enhance the nonlinear processing capability of the model. The size of the large kernel convolution is set to 32*32, which not only enhances the network's spatial understanding of the input image, but also introduces more shape biases to the network through a wider receptive field.

[0062] Furthermore, by using 1×1 convolution layers and BN layers before and after the large kernel convolution, the multi-head self-attention mechanism in the Vision Transformer (ViT) is imitated to improve the network's adaptability and discrimination to different features.

[0063] In the stage of estimating the medium transmission map, because the medium transmission map in the underwater environment is often accompanied by blur and noise, guided filtering is used as a post-processing step to further refine the medium transmission map. Guided filtering is an edge-preserving filter that can maintain edge information while smoothing the image, which is particularly important for transmission map processing.

[0064] S2: Calculate the global background light of the original image by using a quadtree algorithm to obtain an underwater enhanced image;

[0065] Wherein, step S2 further comprises:

[0066] S21: Calculate the global background light of the original image through the quadtree algorithm;

[0067] Wherein, step S21 further includes:

[0068] S221: Divide the original image into four primary rectangular blocks;

[0069] S222: for each first-level rectangular block, respectively calculate the pixel value mean and pixel value variance of the current first-level rectangular block, and obtain the statistical difference of the pixel value mean and pixel value variance of each first-level rectangular block;

[0070] S223: dividing the first-level rectangular blocks with the largest statistical difference among all the first-level rectangular blocks into four second-level rectangular blocks, calculating the statistical difference of each second-level rectangular block, and selecting the second-level rectangular blocks with the largest statistical difference for further equal division until the size of the equally divided rectangular blocks is smaller than a preset threshold, thereby obtaining a plurality of secondary rectangular blocks;

[0071] S224: Select a maximum pixel value rectangular block with extremely large pixel values ​​from the multiple secondary rectangular blocks, and use the maximum pixel value of the maximum pixel value rectangular block as the global background light of the original image.

[0072] S22: restoring the original image through the medium transmission image and the global background light to obtain an underwater enhanced image.

[0073] The expression of the underwater enhancement map in step S22 is:

[0074]

[0075] in, is the original underwater image, To obtain the underwater enhanced image, is the transmission map corresponding to the underwater enhancement map, is the global background light corresponding to the underwater image.

[0076] In the global illumination branch, a quadtree subdivision-based method is used to calculate the global background light. The specific process is as follows: first, the input image is divided into four equal-sized rectangular blocks. For each block, the mean and variance of its pixel values ​​are calculated, and the difference between these two statistics (mean-variance) is used as an evaluation indicator.

[0077] Among these four blocks, the block with the largest (mean-variance) value is selected for further subdivision, and similarly, the block is divided into four smaller rectangular blocks again. This process continues recursively, each time selecting the block with the largest (mean-variance) value for subdivision until the size of the block drops to a preset threshold.

[0078] In the final iteration, the block with the maximum pixel value is selected from these small blocks, and its pixel maximum value is used as the estimated value of the global background light. This quadtree-based recursive subdivision method not only improves the accuracy of global illumination estimation, but also enhances the robustness of the method, making it more adaptable to different lighting conditions and environmental variations.

[0079] After processing from steps S1 to S2, the medium transmission map and the global background light are substituted into the image formation model, and a restored image with significantly improved quality can be calculated and reconstructed. This method not only significantly enhances the visual quality of underwater images, but also improves the detection accuracy of the subsequent SOD network by combining the collaborative training of joint learning with the underwater salient object segmentation task, so that it exhibits higher recognition ability and better performance in complex underwater environments.

[0080] S3: Calculate the ambient light in the original image based on the generalized dark channel prior algorithm to obtain the medium transmission map;

[0081] In this stage, it is necessary to effectively supervise the enhanced image generated by the underwater image enhancement module without the need for additional images to ensure that it conforms to the actual physical process of underwater imaging. Therefore, it is very important to effectively guide the medium transmission map generated by the medium transmission branch. In this stage, the estimated transmission map of the generalized dark channel prior method is adopted, and the corresponding loss function is designed to improve the accuracy of medium transmission and the overall performance of the network.

[0082] Since the corresponding real medium transmission map of the input underwater image is not available in practice, it becomes difficult to train the medium transmission branch to generate the medium transmission map. Although the traditional dark channel prior algorithm (DCP) provides a solution, the calculated transmittance often fails to generate accurate results. Because red light is attenuated more than other wavelengths underwater, the RGB channel-based DCP ultimately only considers the red channel, resulting in unreliable transmission estimation.

[0083] To address this problem, we use a method based on the generalized dark channel prior (GDCP) algorithm to obtain the medium transmission map. This method uses an adaptive ambient light estimation algorithm to improve the adaptability to underwater images, thereby effectively calculating the medium transmission map. The GDCP method is optimized based on the traditional dark channel prior, and increases the adaptability to specific underwater lighting and color distribution. The specific formula for calculating ambient light is as follows.

[0084] The expression of the ambient light calculated in step S3 is:

[0085]

[0086] in, The ambient light calculated by the generalized dark channel prior algorithm is is the pixel index in the original image, Pixels in the original image Background light at It is the set of pixel positions before 0.1% of the maximum pixel value in the original image.

[0087] The expression of the medium transmission graph in step S3 is:

[0088]

[0089] in, To estimate the transmission diagram of the obtained medium, is the color channel, Pixel The local area centered on For the first The pixel points in the local area centered on the pixel index, is the pixel point in the original image Background light.

[0090] As for the estimation of the transmission map, median filtering and linear stretching are applied to refine the estimation of the transmission map. The final formula for estimating the medium transmission map is as above. As shown in the formula, the medium transmission estimation is related to the uniform background light, so the uniform background light is estimated based on the depth-dependent color change in the present invention.

[0091] S4: calculating a mean square error loss for the medium transmission map and the medium transmission map, and guiding the medium transmission branch by the mean square error loss;

[0092] After step S3, the media transmission map generated by the media transmission branch is compared with the transmission map generated by the generalized dark channel prior to obtain the mean square error (MSE) loss, and the media transmission branch is guided by the convolution module guided by the mean square error loss obtained in step S4, that is, the generalized prior.

[0093] The expression of the mean square error loss in step S4 is:

[0094]

[0095] in, is the mean square error loss calculated for the medium transmission map and the medium transmission map, is the total number of pixels, is the pixel coordinate, Transmission map for the medium at pixel points The value at For the medium transmission map in pixels The value at .

[0096] S5: performing cross-modal feature fusion and multi-stream feature extraction on the underwater enhancement image and the medium transmission image to obtain a saliency map.

[0097] In the feature fusion stage, we feed the enhanced underwater image and medium transmission map into a two-stream feature extraction network to generate hierarchical features. U-Net with skip connections has shown its effectiveness in pixel-level segmentation tasks. Therefore, the two-stream feature extraction network connects features between the encoder and decoder through a U-Net-like architecture to preserve valuable geometric priors in each decoder.

[0098] After the underwater image enhancement process, the medium transmission information T is fed into a two-stream feature extraction network together with the enhanced underwater image to generate hierarchical features. Generally speaking, the features extracted from the enhanced underwater image contain rich semantic and texture information; meanwhile, the features of the medium transmission channel contain more discriminative scene layout clues, which are complementary to the features of the RGB features.

[0099] Previous studies have demonstrated the effectiveness of learning from two heterogeneous modalities for RGB-D SOD. Color images provide rich information on visual appearance, while depth maps contain more spatial priors. Both modalities help to regulate homogeneous semantic information. Therefore, the goal of multimodal learning is to effectively fuse diverse information from different modalities. To this end, we propose a robust multimodal fusion scheme that uses a cross-attention cue complementary module to mitigate ambiguity in domain-specific features, thereby making full use of cross-domain cues for multimodal and multi-level reasoning. It consists of two parts, namely cross-modal feature enhancement and adaptive feature fusion.

[0100] Among them, in step S5, the step of performing cross-modal feature fusion of the underwater enhancement image and the medium transmission image further includes:

[0101] S51: mapping the underwater enhancement image and the medium transmission image into a first feature image and a second feature image respectively through convolution, wherein the first feature image corresponds to the underwater enhancement image, and the second feature image corresponds to the medium transmission image;

[0102] S52: Aggregating different domain features of the first feature map and the second feature map through one-dimensional channel attention and two-dimensional spatial attention to obtain a first enhanced feature corresponding to the first feature map and a second enhanced feature corresponding to the second feature map;

[0103] S53: performing element-wise multiplication and maximization on the first enhanced feature and the second enhanced feature in sequence to obtain a fusion feature.

[0104] Furthermore, the above-mentioned convolutional module guided by the generalization prior is trained collaboratively with the subsequent feature fusion network.

[0105] The expression of the fusion feature in step S53 is:

[0106]

[0107]

[0108]

[0109]

[0110]

[0111] in, is the output fusion feature, is a convolutional layer with a convolution kernel size of 3, For the connection operation, is the first feature map, is the second feature map, is the one-dimensional channel attention, is the two-dimensional spatial attention, is the first enhanced feature, is the second enhanced feature, To take the maximum value operation in the feature map, is element-wise multiplication, The features obtained by maximizing the first enhancement feature and the second enhancement feature, The feature is obtained by multiplying the first enhanced feature and the second enhanced feature element by element.

[0112] Specifically, assuming that there are two paired multimodal features, namely the input underwater enhancement map and the medium transmission map, we first map the input to the feature map through 1*1 convolution and 3*3 convolution. After obtaining the lightweight representation, the next step is to aggregate features from different domains. To this end, the present invention proposes a cross-attention fusion scheme. Specifically, from each input feature map, we learn one-dimensional channel attention to determine what information needs to be involved, and two-dimensional spatial attention to determine the part that needs attention, so that the feature map of one modality can be used to enhance the other modality. In addition, in order to retain the original information of each modality, the present invention adds residual connections to combine the enhanced features with their original features. Therefore, the present invention can obtain the cross-enhanced feature representation of the two modalities as shown in the above formula. Through the dual attention learned from each feature map, the present invention realizes cross-domain interaction, which can reduce the ambiguity in domain-specific features.

[0113] There are many strategies for fusing features from different modalities, including element-wise multiplication and maximization. In order to benefit from the advantages of different strategies, we use element-wise multiplication and maximization, and then concatenate the results. After the proposed cross-attention cue complementary module, we can obtain cross-modal fusion features, which, together with the original features extracted from the enhanced underwater image stream and the medium transmission stream, are further fed to three independent decoders. Finally, the predictions of the three decoders are added together to generate the final saliency map SMap and accept the true image supervision.

[0114] The input for multi-stream feature extraction in step S5 includes the fusion feature, the underwater enhancement map, and the medium transmission map.

[0115] The loss expression of the saliency map in step S5 is:

[0116]

[0117]

[0118]

[0119] in, is the loss corresponding to the saliency map, is the mean square error loss calculated for the medium transmission map and the medium transmission map, is the segmentation loss for outputting the segmentation results of salient objects, is the loss corresponding to the underwater enhancement map, is the binary cross entropy loss, is the intersection-and-join loss, is the pixel space loss, Assume loss for the grey world.

[0120] Furthermore, the model proposed in this invention is named MES-Net, and its loss is composed of the segmentation loss for outputting the segmentation results of salient objects , Image enhancement loss for underwater image enhancement module And the generalization prior loss for the medium transmission branch Composition, segmentation loss A mixed loss function containing two losses is used to comprehensively evaluate the gap between the result and the true value. The two losses included are binary cross entropy (BCE) loss and Intersection over Union (IoU) loss , The effectiveness of this hybrid loss has also been verified in many salient object segmentation tasks.

[0121] For the design of the loss function, we considered the importance of three levels: SOD, medium transmission information, and underwater image enhancement. Therefore, the three types of losses are added as the final loss function to jointly optimize the entire network. The underwater image enhancement module receives the back propagation of the three losses at the same time, thereby achieving the effect of task-driven underwater image enhancement, ensuring that the generated enhanced underwater images are directly applicable to USOD tasks, which helps to improve USOD performance.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for underwater salient object detection based on multimodal task-driven enhancement, characterized in that: include: S1: establishing a medium transmission branch through a generalized prior-guided convolution module, and convolving the original image through the medium transmission branch to obtain a medium transmission map; The backbone network of the medium transmission branch consists of three residual convolution layers embedded with large kernel convolutions. The branch is supervised by the generalized dark prior method, and the size of the large kernel convolution is set to 32*32. S2: Calculate the global background light of the original image by using a quadtree algorithm to obtain an underwater enhanced image; S21: Calculate the global background light of the original image through the quadtree algorithm; S22: restoring the original image through the medium transmission image and the global background light to obtain an underwater enhanced image; S3: Calculate the ambient light in the original image based on the generalized dark channel prior algorithm to obtain the medium transmission map; S4: calculating a mean square error loss for the medium transmission map and the medium transmission map, and guiding the medium transmission branch by the mean square error loss; S5: performing cross-modal feature fusion and multi-stream feature extraction on the underwater enhancement image and the medium transmission image to obtain a saliency map; The input of multi-stream feature extraction includes the fusion feature, the underwater enhancement map, and the medium transmission map. The cross-modal fused features, together with the original features extracted from the enhanced underwater image stream and the medium transmission stream, are further fed into three independent decoders. Finally, the predictions of the three decoders are added together to generate the final saliency map SMap and accept the ground-truth image supervision.

2. The underwater salient object detection method based on multimodal task-driven enhancement according to claim 1, characterized in that: Step S21 further includes: S221: Divide the original image into four primary rectangular blocks; S222: for each first-level rectangular block, respectively calculate the pixel value mean and pixel value variance of the current first-level rectangular block, and obtain the statistical difference of the pixel value mean and pixel value variance of each first-level rectangular block; S223: dividing the first-level rectangular blocks with the largest statistical difference among all the first-level rectangular blocks into four second-level rectangular blocks, calculating the statistical difference of each second-level rectangular block, and selecting the second-level rectangular blocks with the largest statistical difference for further equal division until the size of the equally divided rectangular blocks is smaller than a preset threshold, thereby obtaining a plurality of secondary rectangular blocks; S224: Select a maximum pixel value rectangular block with extremely large pixel values ​​from the multiple secondary rectangular blocks, and use the maximum pixel value of the maximum pixel value rectangular block as the global background light of the original image.

3. The underwater salient object detection method based on multimodal task-driven enhancement according to claim 1, characterized in that: The expression of the ambient light calculated in step S3 is: in, The ambient light calculated by the generalized dark channel prior algorithm is is the pixel index in the original image, Pixels in the original image Background light at It is the set of pixel positions before 0.1% of the maximum pixel value in the original image.

4. The underwater salient object detection method based on multimodal task-driven enhancement according to claim 3 is characterized in that: The expression of the medium transmission graph in step S3 is: in, To estimate the transmission diagram of the obtained medium, is the color channel, Pixel The local area centered on Pixel The pixel points in the local area centered at is the pixel point in the original image Background light.

5. The underwater salient object detection method based on multimodal task-driven enhancement according to claim 1, characterized in that: The expression of the mean square error loss in step S4 is: in, is the mean square error loss calculated for the medium transmission map and the medium transmission map, is the total number of pixels, is the pixel coordinate, Transmission map for the medium at pixel points The value at For the medium transmission map in pixels The value at .

6. The underwater salient object detection method based on multimodal task-driven enhancement according to claim 1, characterized in that: In step S5, the step of performing cross-modal feature fusion of the underwater enhancement image and the medium transmission image further includes: S51: mapping the underwater enhancement image and the medium transmission image into a first feature image and a second feature image respectively through convolution, wherein the first feature image corresponds to the underwater enhancement image, and the second feature image corresponds to the medium transmission image; S52: Aggregating different domain features of the first feature map and the second feature map through one-dimensional channel attention and two-dimensional spatial attention to obtain a first enhanced feature corresponding to the first feature map and a second enhanced feature corresponding to the second feature map; S53: performing element-wise multiplication and maximization on the first enhanced feature and the second enhanced feature in sequence to obtain a fusion feature.

7. The underwater salient object detection method based on multimodal task-driven enhancement according to claim 6, characterized in that: The expression of the fusion feature in step S53 is: in, is the output fusion feature, is a convolutional layer with a convolution kernel size of 3, For the connection operation, is the first feature map, is the second feature map, is the one-dimensional channel attention, is the two-dimensional spatial attention, is the first enhanced feature, is the second enhanced feature, To take the maximum value operation in the feature map, is element-wise multiplication, The features obtained by maximizing the first enhancement feature and the second enhancement feature, The feature is obtained by element-wise multiplication of the first enhanced feature and the second enhanced feature.

8. The underwater salient object detection method based on multimodal task-driven enhancement according to claim 1, characterized in that: The loss of the saliency map in step S5 is expressed as: in, is the loss corresponding to the saliency map, is the mean square error loss calculated for the medium transmission map and the medium transmission map, is the segmentation loss for outputting the segmentation results of salient objects, is the loss corresponding to the underwater enhancement map, is the binary cross entropy loss, is the intersection-and-join loss, is the pixel space loss, Assume loss for the grey world.

Images