A method for underwater salient object detection based on secondary degradation
By using media transmission branch and quadtree algorithm to enhance image in underwater target detection, combined with multi-flow feature extraction, the image blur and color offset problems in underwater environment are solved, the detection accuracy and robustness are improved, and the detection accuracy and robustness are achieved. Efficient object detection in complex environments.
Patent Information
- Application Number
- CN202411052999.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-02
AI Technical Summary
Existing underwater target detection methods have problems such as image blurring and color shift in underwater environments. Traditional image enhancement methods may reduce the performance of the detector, and most methods fail to effectively utilize depth information, limiting the model in complex underwater environments. performance and adaptability.
The target detection method of underwater significant object based on secondary degradation is used to convolve the original image through medium transmission branching and guiding filtering. The global background light is calculated in combination with the quad-tree algorithm to enhance the underwater image, and multi-stream Feature extraction and collaborative training are generated to generate detection networks for object detection.
Improve the accuracy and robustness of underwater target detection, so that the model can show higher recognition ability and adaptability in complex underwater environments, ensuring that enhanced underwater images are directly suitable for salient target detection tasks, and improving detection accuracy.
Smart Images

Figure CN118570625B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection, and in particular to a method for detecting underwater salient objects based on secondary degradation. Background Art
[0002] In order to deal with the inherent problems of underwater images in underwater target detection, such as blur and color cast, many underwater image enhancement methods have been developed. Some works use image enhancement methods to preprocess the input images and provide enhanced images for subsequent detection methods. However, these methods may be subject to two limitations: the enhanced images are usually not what the target detector wants. Experiments show that in underwater environments, traditional image enhancement methods that preprocess the input images may degrade the performance of the detector because these traditional methods ignore potential information that helps target detection and sometimes even produce distorted images that mislead the detector. Additional datasets are required. Image enhancement methods that are jointly trained with detectors usually require additional paired image enhancement datasets to support the training of the enhancement module. However, some detection tasks cannot access these additional datasets to use these methods.
[0003] Although there are currently a variety of salient object detection (RGBD-SOD) networks that utilize RGBD information, their application in underwater environments still faces major challenges, especially in the acquisition and utilization of depth information. Most existing methods only use RGB images for salient object detection, and the few methods that consider depth information use depth information as a preprocessing step instead of integrating it into the detection process for collaborative training, which limits the performance and adaptability of the model in complex underwater environments. Summary of the invention
[0004] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a method for underwater salient object detection based on secondary degradation.
[0005] The present invention provides a method for underwater salient object detection based on secondary degradation, comprising:
[0006] S1: convolve the original image based on the medium transmission branch and the guided filtering method, and calculate the global background light of the original image through the quadtree algorithm to obtain an underwater enhanced image;
[0007] S2: performing secondary degradation unsupervised learning on the underwater enhanced image to obtain a secondary enhanced image;
[0008] S3: performing multi-stream feature extraction on the secondary enhancement image and the secondary transmission image corresponding to the secondary enhancement image to obtain a detection network;
[0009] S4: Perform underwater salient object detection on the image to be detected through the detection network.
[0010] According to the underwater salient object detection method based on secondary degradation provided by the present invention, step S1 further comprises:
[0011] S11: Based on the deep learning network, a medium transmission branch is established to convolve the original image, and the original image is refined by the guided filtering method to obtain the medium transmission map;
[0012] S12: Calculate the global background light of the original image by using a quadtree algorithm;
[0013] S13: restoring the original image through the medium transmission image and the global background light to obtain an underwater enhanced image.
[0014] According to the underwater salient object detection method based on secondary degradation provided by the present invention, step S11 further includes:
[0015] S111: Select ResNet network as the backbone network;
[0016] S112: Based on a multi-head attention mechanism, convolve the original image through multiple batch normalization layers and multiple convolution layers of the backbone network to obtain a convolution graph;
[0017] S113: Smoothing the convolution image by guided filtering to obtain a medium transmission image.
[0018] According to the underwater salient object detection method based on secondary degradation provided by the present invention, step S12 further includes:
[0019] S121: Divide the original image into four primary rectangular blocks;
[0020] S122: for each first-level rectangular block, respectively calculate the pixel value mean and pixel value variance of the current first-level rectangular block, and obtain the statistical difference of the pixel value mean and pixel value variance of each first-level rectangular block;
[0021] S123: dividing the first-level rectangular blocks with the largest statistical difference among all the first-level rectangular blocks into four second-level rectangular blocks, calculating the statistical difference of each second-level rectangular block, and selecting the second-level rectangular blocks with the largest statistical difference for further equal division until the size of the equally divided rectangular blocks is smaller than a preset threshold, thereby obtaining a plurality of secondary rectangular blocks;
[0022] S124: Selecting a maximum pixel value rectangular block with extremely large pixel values from the multiple secondary rectangular blocks, and using the maximum pixel value of the maximum pixel value rectangular block as the global background light of the original image.
[0023] According to the underwater salient object detection method based on secondary degradation provided by the present invention, step S2 further comprises:
[0024] S21: Perform image enhancement on the original image to obtain an enhanced image;
[0025] S22: performing image quality degradation on the original image to obtain a secondary degradation image, and performing image enhancement on the secondary degradation image to obtain a secondary enhancement image;
[0026] S23: performing unsupervised learning on the primary enhancement graph through the secondary enhancement graph.
[0027] According to a method for underwater salient object detection based on secondary degradation provided by the present invention, the expression of the primary enhancement image in step S21 is:
[0028]
[0029] in, is the original image, is an enhanced graph, is the transmission image corresponding to the underwater enhancement image, Global background light for underwater enhancement images;
[0030] The expression of the secondary degradation graph in step S22 is:
[0031]
[0032] in, is a secondary degradation graph, is the hyperparameter used for secondary degradation;
[0033] The expression of the secondary enhancement map in step S22 is:
[0034]
[0035] in, is the secondary enhancement graph, It is the transmission map corresponding to the secondary enhancement map.
[0036] According to a method for underwater salient object detection based on secondary degradation provided by the present invention, in step S2, the expression for the loss of performing secondary degradation on the underwater enhanced image is:
[0037]
[0038] in, is the transmission graph loss during the secondary degradation process, is the hyperparameter used for secondary degradation, is the pixel point of the transmission image in the secondary degradation process, is a transmission graph corresponding to an enhanced graph, Secondary transmission graph corresponding to the secondary enhancement graph.
[0039] According to a method for underwater salient object detection based on secondary degradation provided by the present invention, in step S3, feature extraction is performed on the secondary enhanced image and the secondary transmission image corresponding to the secondary enhanced image, and the extracted features are collaboratively trained by the same framework to complete multi-source fusion learning to obtain fusion features, and the expression of the fusion features is:
[0040]
[0041] in, The fusion features are obtained by extracting dual-stream features based on the secondary enhancement image and the secondary transmission image. is the first input feature map corresponding to the secondary enhancement map, is the second input feature map corresponding to the secondary transmission map, is the weight used for feature fusion.
[0042] According to a method for underwater salient object detection based on secondary degradation provided by the present invention, the loss function of the detection network in step S4 is expressed as:
[0043]
[0044]
[0045]
[0046] in, To detect the overall loss function corresponding to the network, is the segmentation loss, is the binary cross entropy loss, is the intersection-and-join loss, is the image enhancement loss, is the pixel space loss, Assume loss for the grey world.
[0047] The present invention provides a method for underwater salient object detection based on secondary degradation, proposes an MSH-Net (MultiSourceHydro Net) model, improves the existing model from the perspective of multi-source joint learning, and compared with the traditional RGB-USOD network, MSH-Net improves the accuracy and robustness of underwater salient target detection by combining underwater image enhancement technology and medium transmission information. Different from other USOD models that consider image enhancement, MSH-Net does not simply use the enhanced image as input, but adopts a secondary degradation method to obtain secondary transmission information from a single RGB image and conducts collaborative training, so that the model can supervise the medium transmission information generated by the medium transmission branch without the need for additional images, and at the same time, the enhancement and saliency detection of the underwater image can be jointly optimized in the same network architecture.
[0048] The invention provides a method for underwater salient object detection based on secondary degradation, which not only enhances the adaptability of the model to the specific underwater environment, but also ensures that the enhanced underwater images are directly applicable to the USOD task, thereby improving the accuracy and robustness of detection. Through this innovative network structure and training method, MSH-Net significantly improves the performance of salient target detection in complex underwater environments.
[0049] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0051] Figure 1 This is a flow chart of a method for underwater salient object detection based on secondary degradation provided by the present invention. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.
[0053] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0054] Combine the following Figure 1 Embodiments of the present invention are described.
[0055] The present invention provides a method for underwater salient object detection based on secondary degradation, comprising:
[0056] S1: convolve the original image based on the medium transmission branch and the guided filtering method, and calculate the global background light of the original image through the quadtree algorithm to obtain an underwater enhanced image;
[0057] The impact of image quality degradation under adverse weather conditions can be described by a physical model of image formation, which represents degraded images and is widely used in image dehazing and underwater image restoration algorithms.
[0058] According to the model, the key to restoring the original image is to accurately estimate the medium transmission map and the global background light. Therefore, the present invention establishes an underwater enhancement module in step S1, and divides the established underwater enhancement module into a medium transmission branch and a global background light branch. The above-mentioned medium transmission map and global background light are obtained through the designed medium transmission branch and global illumination branch.
[0059] The medium transmission branch uses a deep learning network to estimate the transmittance by analyzing the local and global features of the input image and generalization prior guidance, while the global illumination branch determines the intensity of the background light through a quadtree subdivision method.
[0060] Wherein, step S1 further comprises:
[0061] S11: Based on the deep learning network, a medium transmission branch is established to convolve the original image, and the original image is refined by the guided filtering method to obtain the medium transmission map;
[0062] Wherein, step S11 further comprises:
[0063] S111: Select ResNet network as the backbone network;
[0064] S112: Based on a multi-head attention mechanism, convolve the original image through multiple batch normalization layers and multiple convolution layers of the backbone network to obtain a convolution graph;
[0065] S113: Smoothing the convolution image by guided filtering to obtain a medium transmission image.
[0066] Furthermore, the medium transmission map required for underwater image enhancement is generated by a convolutional module guided by a generalized prior. The module is called the medium transmission branch. The backbone network of the medium transmission branch consists of three residual convolutional layers embedded with large kernel convolutions. The branch is supervised by the generalized dark prior method and the loss feedback of the subsequent SOD network.
[0067] First, an efficient medium transmission branch backbone network is constructed by embedding the first three layers of ResNet50 with large kernel convolution. ResNet, with its residual connection characteristics, can effectively avoid the gradient vanishing problem in deep networks, which makes the network more stable during training.
[0068] Each encoder contains multiple residual blocks. In each residual block, not only batch normalization (BN) is used to optimize the training process, but also the ReLU activation function is used to enhance the nonlinear processing capability of the model. The size of the large kernel convolution is set to 32*32, which not only enhances the network's spatial understanding of the input image, but also introduces more shape biases to the network through a wider receptive field.
[0069] Furthermore, by using 1×1 convolution layers and BN layers before and after the large kernel convolution, the multi-head self-attention mechanism in the Vision Transformer (ViT) is imitated to improve the network's adaptability and discrimination to different features.
[0070] In the stage of estimating the medium transmission map, because the medium transmission map in the underwater environment is often accompanied by blur and noise, guided filtering is used as a post-processing step to further refine the medium transmission map. Guided filtering is an edge-preserving filter that can maintain edge information while smoothing the image, which is particularly important for transmission map processing.
[0071] S12: Calculate the global background light of the original image by using a quadtree algorithm;
[0072] Wherein, step S12 further comprises:
[0073] S121: Divide the original image into four primary rectangular blocks;
[0074] S122: for each first-level rectangular block, respectively calculate the pixel value mean and pixel value variance of the current first-level rectangular block, and obtain the statistical difference of the pixel value mean and pixel value variance of each first-level rectangular block;
[0075] S123: dividing the first-level rectangular blocks with the largest statistical difference among all the first-level rectangular blocks into four second-level rectangular blocks, calculating the statistical difference of each second-level rectangular block, and selecting the second-level rectangular blocks with the largest statistical difference for further equal division until the size of the equally divided rectangular blocks is smaller than a preset threshold, thereby obtaining a plurality of secondary rectangular blocks;
[0076] S124: Selecting a maximum pixel value rectangular block with extremely large pixel values from the multiple secondary rectangular blocks, and using the maximum pixel value of the maximum pixel value rectangular block as the global background light of the original image.
[0077] In the global illumination branch, a quadtree subdivision-based method is used to calculate the global background light. The specific process is as follows: first, the input image is divided into four equal-sized rectangular blocks. For each block, the mean and variance of its pixel values are calculated, and the difference between these two statistics (mean-variance) is used as an evaluation indicator.
[0078] Among these four blocks, the block with the largest (mean-variance) value is selected for further subdivision, and similarly, the block is divided into four smaller rectangular blocks again. This process continues recursively, each time selecting the block with the largest (mean-variance) value for subdivision until the size of the block drops to a preset threshold.
[0079] In the final iteration, the block with the maximum pixel value is selected from these small blocks, and its pixel maximum value is used as the estimated value of the global background light. This quadtree-based recursive subdivision method not only improves the accuracy of global illumination estimation, but also enhances the robustness of the method, making it more adaptable to different lighting conditions and environmental variations.
[0080] S13: restoring the original image through the medium transmission image and the global background light to obtain an underwater enhanced image.
[0081] After processing steps S11 to S12, these parameters are substituted into the image formation model to calculate and reconstruct a restored image with significantly improved quality. This method not only significantly enhances the visual quality of underwater images, but also improves the detection accuracy of the subsequent SOD network by combining the collaborative training of joint learning with the underwater salient object segmentation task, so that it exhibits higher recognition ability and better performance in complex underwater environments.
[0082] S2: performing secondary degradation unsupervised learning on the underwater enhanced image to obtain a secondary enhanced image;
[0083] In order to effectively supervise the enhanced images generated by the underwater image enhancement module without the need for additional images and ensure that its generation conforms to the actual physical process of underwater imaging, it is necessary to design a suitable unsupervised learning method that conforms to the actual physical process of underwater imaging. Therefore, in this stage, a controllable secondary degradation unsupervised learning strategy is designed for the underwater image enhancement module, so that the underwater image enhancement module can be trained together with the detector without any additional data set.
[0084] Wherein, step S2 further comprises:
[0085] S21: Perform image enhancement on the original image to obtain an enhanced image;
[0086] The expression of the primary enhancement graph in step S21 is:
[0087]
[0088] in, is the original image, is an enhanced graph, is the transmission image corresponding to the underwater enhancement image, Global background light for underwater enhancement images;
[0089] S22: performing image quality degradation on the original image to obtain a secondary degradation image, and performing image enhancement on the secondary degradation image to obtain a secondary enhancement image;
[0090] The expression of the secondary degradation graph in step S22 is:
[0091]
[0092] in, is a secondary degradation graph, is the hyperparameter used for secondary degradation;
[0093] The expression of the secondary enhancement map in step S22 is:
[0094]
[0095] in, is the secondary enhancement graph, It is the transmission map corresponding to the secondary enhancement map.
[0096] In the above formula, theoretically, whether it is the first enhanced image or the second enhanced image, it should be an undegraded image. The input to the underwater image enhancement module should be ,Will The input to the underwater image enhancement module should be , It is a hyperparameter that controls the degree of degradation of the secondary enhancement image. Its value ranges from 0 to 1. Its value is randomly selected between 0.5 and 0.9 in each round of training to improve the robustness of the unsupervised method.
[0097] In addition, from the above formula, we can know that Calculate , and use the underwater image enhancement module with shared parameters to obtain The transmission diagram and the corresponding global background light, theoretically, according to The transmission diagram obtained should be consistent with Similarly, based on such physical properties, we can design a secondary transmission graph loss for unsupervised learning, that is, by minimizing The transmission diagram obtained is The similarity gap between them is used to optimize the underwater image enhancement module, and then the transmission map loss can be obtained as follows.
[0098] Wherein, in step S2, the expression of the loss of secondary degradation of the underwater enhancement image is:
[0099]
[0100] in, is the transmission graph loss during the secondary degradation process, is the hyperparameter used for secondary degradation, is the pixel point of the transmission image in the secondary degradation process, is a transmission graph corresponding to an enhanced graph, Secondary transmission graph corresponding to the secondary enhancement graph.
[0101] S23: performing unsupervised learning on the primary enhancement graph through the secondary enhancement graph.
[0102] During the training process, the above transfer graph loss It plays a key role in ensuring that the underwater image enhancement module follows the laws of physics for unsupervised learning.
[0103] In the secondary degradation multi-source joint learning, "secondary degradation" refers to the secondary degradation unsupervised learning method used by the underwater image enhancement module, ensuring that the underwater image enhancement module follows physical principles for unsupervised learning.
[0104] S3: performing multi-stream feature extraction on the secondary enhancement image and the secondary transmission image corresponding to the secondary enhancement image to obtain a detection network;
[0105] In the feature fusion stage, the secondary enhanced image and the secondary transfer map are respectively input into the two-stream feature extraction network to generate multi-modal features.
[0106] U-Net with skip connections has shown its effectiveness in pixel-level segmentation tasks, so the two-stream feature extraction network connects features between the encoder and decoder through a U-Net-like architecture to preserve valuable geometric priors in each decoder.
[0107] Subsequently, a feature fusion strategy is adopted, that is, an attention fusion module (AFM) is established for feature fusion, aiming to integrate valuable clues from RGB features and media transmission into cross-modal fusion features.
[0108] Three decoding branches are designed in the attention fusion module to process the features of enhanced underwater image, medium transmittance and fusion level respectively, and the corresponding outputs are added to generate the final saliency map SMap.
[0109] After the underwater image enhancement process, the secondary enhanced image and secondary transmission Figure 1 They are fed into a two-stream feature extraction network to generate hierarchical features. Generally speaking, the features extracted from enhanced underwater images contain rich semantic and texture information. At the same time, the features of the medium transmission channel contain more discriminative scene layout clues, which are complementary to the features of RGB features. In order to integrate cross-modal information, the present invention designs a fusion strategy called Attention Fusion Module (AFM).
[0110] The proposed attention fusion module (AFM) aims to mine and combine the most discriminative channels from medium transmission and enhanced underwater image features and generate more informative features. Specifically, for two input feature maps X and Y from different branches, the AFM operation is implemented using an attention weight generator M, which is expressed as follows.
[0111] Among them, in step S3, feature extraction is performed on the secondary enhancement image and the secondary transmission image corresponding to the secondary enhancement image, and the extracted features are collaboratively trained by the same framework to complete multi-source fusion learning to obtain fusion features. The expression of the fusion feature is:
[0112]
[0113] in, The fusion features are obtained by extracting dual-stream features based on the secondary enhancement image and the secondary transmission image. is the first input feature map corresponding to the secondary enhancement map, is the second input feature map corresponding to the secondary transmission map, is the weight used for feature fusion.
[0114] In the above formula, is a real number in the range of 0 to 1. This feature allows the network to input features and The key to this weight parameter is that by changing the size of the spatial pool, attention can be achieved at multiple scales. In the selection of weights, point-wise convolution (PWConv) is used as the local channel context aggregator, and then the local channel context and the global feature context are accumulated, and then calculated after being processed by the sigma function.
[0115] Multi-source joint learning refers to the secondary enhanced image and secondary transfer map generated by the underwater image enhancement module being input into the dual-stream feature extraction network respectively to generate multi-modal features, and both are optimized by joint training. The advantage of this method is that by inputting the enhanced image and the transfer map containing depth information into the dual-stream network, multi-modal features can be generated to enhance the accuracy of salient object detection. At the same time, both are jointly trained, so that the effective back propagation from subsequent tasks to image enhancement and transfer map calculation guides the calculation effect of image enhancement and transfer map.
[0116] S4: Perform underwater salient object detection on the image to be detected through the detection network.
[0117] After steps S1 to S3, cross-modal fusion features can be obtained, which, together with the original features extracted from the enhanced underwater image stream and the medium transmission stream, are further fed into three independent decoders. Finally, the predictions of the three decoders are added together to generate the final saliency map SMap and accept the real image supervision.
[0118] The loss function of the detection network in step S4 is expressed as:
[0119]
[0120]
[0121]
[0122] in, To detect the overall loss function corresponding to the network, is the segmentation loss, is the binary cross entropy loss, is the intersection-and-join loss, is the image enhancement loss, is the pixel space loss, Assume loss for the grey world.
[0123] Furthermore, the loss of MSH-Net is composed of the segmentation loss for outputting the salient object segmentation results , Image enhancement loss for underwater image enhancement module And the secondary transfer graph loss for unsupervised learning of secondary degradation Composition, segmentation loss A mixed loss function containing two losses is used to comprehensively evaluate the gap between the result and the true value. The two losses included are binary cross entropy (BCE) loss and Intersection over Union (IoU) loss , The effectiveness of this hybrid loss has also been verified in many salient object segmentation tasks.
[0124] It should be noted that the pixel space loss It is used to constrain the enhanced image to the image pixel space to avoid pixel value overflow and underflow, while the gray world assumption loss It is a loss proposed based on the grayscale world assumption of natural image statistics, which can reduce the color cast in the restored image.
[0125] For the design of the loss function, the present invention considers the importance of three levels: SOD, secondary transmission map and underwater image enhancement. Therefore, the three types of losses are added as the final loss function to jointly optimize the entire network, in which the underwater image enhancement module receives the back propagation of the three losses at the same time, thereby achieving the effect of multi-source joint learning of underwater image enhancement, ensuring that the generated enhanced underwater image is directly applicable to the USOD task, which helps to improve the performance of USOD.
[0126] This paper provides a method for underwater salient object detection based on secondary degradation, and proposes an innovative network model named MSH-Net, which aims to solve the problem of salient object detection in underwater images. The core design of MSH-Net includes two stages: underwater image enhancement stage and three-stream feature fusion stage.
[0127] In the underwater image enhancement stage, in order to calculate and reconstruct a restored image with significantly improved quality, the medium transmission branch and the global illumination branch are first used to obtain the medium transmission map and the global background light. Then, secondary degradation unsupervised learning is performed on this basis to obtain a secondary enhanced image and a secondary transmission map. The medium transmission branch uses a deep learning network to estimate the transmittance by analyzing the local and global features of the input image and generalized prior guidance; the global illumination branch determines the intensity of the background light through the quadtree subdivision method.
[0128] In the feature fusion stage, the secondary enhanced image and the secondary transmission map are respectively input into the two-stream feature extraction network to generate multi-modal features. Each stream adopts the U-Net structure as the backbone network. Subsequently, a feature fusion strategy is adopted: the attention fusion module (AFM) aims to integrate valuable clues from RGB features and media transmission into cross-modal fusion features. In the feature fusion stage, three decoding branches are also designed to process the features of RGB, media transmission and fusion levels respectively, and the corresponding outputs are added to generate the final saliency map SMap.
[0129] MSH-Net improves the accuracy and robustness of underwater salient object detection by combining underwater image enhancement technology and media transmission information. Different from other USOD models that consider image enhancement, MSH-Net does not simply use enhanced images as input, but adopts a secondary degradation method to obtain secondary transmission information from a single RGB image and trains it together, so that the model can supervise the media transmission information generated by the media transmission branch without the need for additional images, and underwater image enhancement and saliency detection can be jointly optimized in the same network architecture.
[0130] This integration method not only enhances the adaptability of the model to the specific underwater environment, but also ensures that the enhanced underwater images are directly applicable to the USOD task, thereby improving the accuracy and robustness of detection. Through this innovative network structure and training method, MSH-Net significantly improves the performance of salient object detection in complex underwater environments.
[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for underwater salient object detection based on secondary degradation, characterized in that: include: S1: Convolve the original image based on the medium transmission branch and the guided filtering method, and calculate the global background light of the original image through the quadtree algorithm to obtain an underwater enhanced image; specifically including: S11: Based on the deep learning network, a medium transmission branch is established to convolve the original image, and the original image is refined by the guided filtering method to obtain the medium transmission map; S12: Calculate the global background light of the original image by using a quadtree algorithm; S13: restoring the original image through the medium transmission image and the global background light to obtain an underwater enhanced image; S2: performing secondary degradation unsupervised learning on the underwater enhanced image to obtain a secondary enhanced image; specifically comprising: S21: Perform image enhancement on the original image to obtain an enhanced image; S22: performing image quality degradation on the original image to obtain a secondary degradation image, and performing image enhancement on the secondary degradation image to obtain a secondary enhancement image; S23: performing unsupervised learning on the primary enhancement graph through the secondary enhancement graph; S3: Perform multi-stream feature extraction on the secondary enhancement graph and the secondary transmission graph corresponding to the secondary enhancement graph to obtain a detection network; wherein, feature extraction is performed on the secondary enhancement graph and the secondary transmission graph corresponding to the secondary enhancement graph, and the extracted features are collaboratively trained by the same framework to complete multi-source fusion learning to obtain fusion features, and the expression of the fusion features is: in, The fusion features are obtained by extracting dual-stream features based on the secondary enhancement image and the secondary transmission image. is the first input feature map corresponding to the secondary enhancement map, is the second input feature map corresponding to the secondary transmission map, is the weight for feature fusion; in the feature fusion stage, the secondary enhancement map and the secondary transmission map are respectively input into the dual-stream feature extraction network to generate multi-modal features. Each stream uses the U-Net structure as the backbone network. Subsequently, the feature fusion strategy is adopted: the valuable clues from the RGB features and media transmission are integrated into the cross-modal fusion features through the attention fusion module. In the feature fusion stage, three decoding branches are also designed to process the features of RGB, media transmission and fusion levels respectively, and the corresponding outputs are added to generate the final saliency map; S4: Perform underwater salient object detection on the image to be detected through the detection network.
2. The underwater salient object detection method based on secondary degradation according to claim 1 is characterized in that: Step S11 further comprises: S111: Select ResNet network as the backbone network; S112: Based on a multi-head attention mechanism, convolve the original image through multiple batch normalization layers and multiple convolution layers of the backbone network to obtain a convolution graph; S113: Smoothing the convolution image by guided filtering to obtain a medium transmission image.
3. The underwater salient object detection method based on secondary degradation according to claim 1 is characterized in that: Step S12 further comprises: S121: Divide the original image into four primary rectangular blocks; S122: for each first-level rectangular block, respectively calculate the pixel value mean and pixel value variance of the current first-level rectangular block, and obtain the statistical difference of the pixel value mean and pixel value variance of each first-level rectangular block; S123: dividing the first-level rectangular blocks with the largest statistical difference among all the first-level rectangular blocks into four second-level rectangular blocks, calculating the statistical difference of each second-level rectangular block, and selecting the second-level rectangular blocks with the largest statistical difference for further equal division until the size of the equally divided rectangular blocks is smaller than a preset threshold, thereby obtaining a plurality of secondary rectangular blocks; S124: Selecting a maximum pixel value rectangular block with extremely large pixel values from the multiple secondary rectangular blocks, and using the maximum pixel value of the maximum pixel value rectangular block as the global background light of the original image.
4. The underwater salient object detection method based on secondary degradation according to claim 1 is characterized in that: The expression of the primary enhancement graph in step S21 is: in, is the original image, is an enhanced graph, is the transmission image corresponding to the underwater enhancement image, Global background light for underwater enhancement images; The expression of the secondary degradation graph in step S22 is: in, is a secondary degradation graph, is the hyperparameter used for secondary degradation; The expression of the secondary enhancement map in step S22 is: in, is the secondary enhancement graph, It is the transmission graph corresponding to the secondary enhancement graph.
5. The underwater salient object detection method based on secondary degradation according to claim 1 is characterized in that: In step S2, the loss of secondary degradation of the underwater enhanced image is expressed as: in, is the transmission graph loss during the secondary degradation process, is the hyperparameter used for secondary degradation, is the pixel point of the transmission image in the secondary degradation process, is a transmission graph corresponding to an enhanced graph, Secondary transmission graph corresponding to the secondary enhancement graph.
6. The underwater salient object detection method based on secondary degradation according to claim 5 is characterized in that: The expression of the loss function of the detection network in step S4 is: in, To detect the overall loss function corresponding to the network, is the segmentation loss, is the binary cross entropy loss, is the intersection-and-join loss, is the image enhancement loss, is the pixel space loss, Assume loss for the grey world.
