Underwater image enhancement method based on multi-scale unwrapping strategy

Abstract

The invention provides an underwater image enhancement method based on a multi-scale unwrapping strategy, and the method comprises the steps: employing a multi-scale unwrapping network, firstly obtaining a transmission image and scene brightness through a scene radiation unwrapping module and a transmission image unwrapping module, and then carrying out the segmentation of the transmission image and the scene brightness through a Jaffe-McGamby underwater imaging model, three unwrapping components of scene brightness and global background light are coupled into an original image, and the unwrapping performance is improved through a self-supervision method.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision, underwater image enhancement, and in particular to an underwater image enhancement method based on a multi-scale disentanglement strategy. Background Art

[0002] In recent years, deep learning methods have become an important research direction for underwater images. These methods rely on a large amount of data for model training to learn the mapping relationship between clear images and blurred images in order to restore underwater images in complex environments. However, existing deep learning methods often ignore the scene features of underwater imaging, which affects the enhancement effect of underwater images.

[0003] The prior art documents related to the present invention include:

[0004] [1] N. Carlevaris-Bianco, A. Mohan, and RMEustice, “Initial results inunderwater single image dehazing,” in Oceans 2010MTS / IEEE Seattle, pp.1–8, 2010.

[0005] [2]MJIslam, Y.Xia, and J.Sattar, "Fast underwater image enhancement for improved visual perception," IEEE Robotics and Automation Letters, vol.5, no.2, pp.3227–3234, 2020.

[0006] [3] L.Peng, C.Zhu, and L.Bian, "U-shape transformer for underwater image enhancement," IEEE Transactions on Image Processing, vol.32, pp.3066–3079, 2023.

[0007] [4]C.Li,C.Guo,W.Ren,R.Cong,J.Hou,S.Kwong,and D.Tao,“An underwaterimage enhancement benchmark dataset and beyond,”IEEE Transactions on ImageProcessing,vol.29,pp.4376–4389,2020.

[0008] [5]K.He,J.Sun,and X.Tang,“Single image haze removal using darkchannel prior,”IEEE Transactions on pattern analysis and machineintelligence,vol.33,no.12,pp.2341–2353,2010.

[0009] [6]P.J.Drews,E.do Nascimento,F.Moraes,S.Botelho,and M.Campos,“Transmission estimation in underwater single images,”in Proceedings oftheIEEE International Conference on Computer Vision(ICCV)Workshops,pp.825–830,2013.

[0010] [7]S.An,L.Xu,Z.Deng,and H.Zhang,“Hfm:A hybrid fusion method forunderwater image enhancement,”Engineering Applications ofArtificialIntelligence,vol.127,2024.

[0011] [8]W.Zhang,Y.Wang,and C.Li,“Underwater image enhancement byattenuated color channel correction and detail preserved contrastenhancement,”IEEE Journal ofOceanic Engineering,vol.47,no.3,pp.718–735,2022.

[0012] [9]A.R.Naik,A.Swarnakar,and K.Mittal,“Shallow-uwnet:Compressed modelfor underwater image enhancement,”in AAAI Conference on ArtificialIntelligence,2021.

[0013]

[10] Z.Fu,W.Wang,Y.Huang,X.Ding,and K.-K.Ma,“Uncertainty inspiredunderwater image enhancement,”in European Conference on Computer Vision,2022.

[0014]

[11] Y.Rao,W.Liu,K.Li,H.Fan,S.Wang,and J.Dong,“Deep color compensationfor generalized underwater image enhancement,”IEEE Transactions on Circuitsand Systems for Video Technology,vol.34,no.4,pp.2577–2590,2023.

[0015]

[12] J.Korhonen and J.You,“Peak signal-to-noise ratio revisited:Issimple beautiful?”in Fourth International Workshop on Quality of MultimediaExperience,2012,pp.37–38.

[0016]

[13] A.Horéand D.Ziou,“Image quality metrics:Psnr vs.ssim,”inInternational Conference on Pattern Recognition,2010,p.2366–2369.

[0017]

[14] M.Yang and A.Sowmya,“An underwater color image quality evaluationmetric,”IEEE Transactions on Image Processing,vol.24,no.12,p.6062–6071,2015.

[0018]

[15] K.Panetta,C.Gao,and S.Agaian,“Human-visual-system-inspiredunderwater image quality measures,”IEEE Journal of Oceanic Engineering,vol.41,no.3,p.541–551,2016.

[0019]

[16] D.Berman,T.treibitz,and S.Avidan,“Non-local image dehazing,”inIEEE / CVF Conference on Computer Vision and Pattern Recognition(CVPR),2016,p.1674–1682.

[0020]

[17] J.Canny,“A computational approach to edge detection,”IEEETransactions on Pattern Analysis and Machine Intelligence,vol.X8,no.6,p.679–698,1986.

[0021]

[18] D.G.Lowe,“Distinctive image features from scale-invariantkeypoints,”International Journal of Computer Vision,vol.60,no.2,p.91–110,2004.

[0022]

[19]

[0023]

[20] T. Lei, Summary of the invention

[0024] In view of the defects and shortcomings of the prior art, the present invention proposes a multi-scale disentanglement network (MD-Net). MD-Net mainly includes a transmission map disentanglement (TMD) module and a scene brightness disentanglement (SRD) module, which are used to obtain the transmission map and scene brightness, respectively. Then the acquired parameters (including the global background light) are coupled into the original image through the underwater imaging model, and the self-supervision method is used to improve the acquisition accuracy of each parameter to approximate the true value. In SRD and TMD, the present invention develops a multi-level fusion module (MFM) and a dual-layer weight estimation unit (DWEU) for the extraction of shallow spatial features and deep semantic features, respectively.

[0025] The present invention specifically adopts the following technical solutions:

[0026] An underwater image enhancement method based on a multi-scale disentanglement strategy: a multi-scale disentanglement network is adopted. First, the transmission map and scene brightness are obtained respectively through the scene radiation disentanglement module and the transmission map disentanglement module. Then, the three disentangled components including the transmission map, scene brightness and global background light are coupled into the original image through the Jaffe-McGlamery underwater imaging model, and the disentanglement performance is improved through a self-supervised method.

[0027] Furthermore, a multi-level fusion module and a double-layer weight estimation unit are provided in the scene radiation unwrapping module and the transmission map unwrapping module. The multi-level fusion module is used to extract spatial features from three layers of different receptive fields, and transmit shallow spatial information to high layers through jump connections; the double-layer weight estimation unit is used to estimate pixel weights and channel weights through one or more iterations to obtain deep semantic features.

[0028] Furthermore, the multi-scale disentanglement network consists of a Gaussian blur block, a transmission map disentanglement module, a scene radiation disentanglement module and a Jaffe-McGlamery underwater imaging model; the Gaussian blur block, the transmission map disentanglement module and the scene radiation disentanglement module respectively disentangle the original image into global background light, transmission map and scene radiation; the Jaffe-McGlamery underwater imaging model reconstructs the original image by three parameters of global background light, transmission map and scene radiation to supervise the disentanglement accuracy of the Gaussian blur block, the transmission map disentanglement module and the scene radiation disentanglement module.

[0029] Furthermore, the transmission map unwrapping module and the scene radiation unwrapping module adopt the same network structure; the process of unwrapping the original image into the transmission map unwrapping module or the scene radiation unwrapping module is: using three different receptive fields of 3×3, 5×5, and 7×7 through a multi-level fusion module to extract shallow spatial features and multi-layer fusion features; then the multi-layer fusion features are input into the global average pooling layer and the double-layer weight estimation unit to obtain down-sampled features and double-layer weight features respectively, the down-sampled features and the double-layer weight features are multiplied to obtain deep semantic features, and finally the deep semantic features corresponding to the three different receptive fields are added, and the number of channels of the transmission map or scene radiation is adjusted using an adaptive convolutional layer to obtain the final output.

[0030] Furthermore, the multi-level fusion module first captures spatial information features at different levels from the input image through three convolutional layers with different receptive field sizes: 3×3, 5×5, and 7×7. Then Convblock-1(Conv1) is used to further capture more detailed shallow information:

[0031]

[0032] Among them, I S Represents shallow spatial features. Each Conv1 contains two convolutional layers, a ReLU activation function and a normalization layer.

[0033] Subsequently, the tensor concatenation operation is performed and feature fusion is performed through Convblock-2 (Conv2) to obtain dynamic weights. In order to assign weights to each branch, tensor splitting is performed on the dynamic weights:

[0034]

[0035] Among them, W1, W2, and W3 correspond to the branch weights after segmentation, spl represents tensor cutting, cat represents tensor concatenation, and Conv2 contains two convolutional layers, a ReLU activation function, and a Sigmoid function. Finally, the multi-layer fusion feature I obtained by fusing W1, W2, and W3 is mf To integrate image feature information of receptive fields of different scales.

[0036] Furthermore, the iterative process of the two-layer weight estimation unit is as follows: first, a convolutional layer with a kernel size of 3×3 is used to obtain the multi-layer fusion feature I mf Then, they are input into the channel estimation unit and the pixel estimation unit respectively to obtain the pixel weight W p and channel weight W c ; The pixel weight estimation is expressed as:

[0037] W p =conv.s(conv.ri(I i )) (3)

[0038] Among them, I i It is the feature extracted from the convolution. conv.ri contains a convolution layer with a kernel size of 3×3 for processing image features, followed by a ReLU activation for nonlinear transformation and a normalization layer for standardizing feature maps. conv.s contains a convolution layer with a kernel size of 3×3 for further processing feature maps, followed by a Sigmoid function to compress the output value into the range of [0, 1];

[0039] The channel weight estimation is expressed as:

[0040] W c =conv.s(conv.r(gap(I i ))) (4)

[0041] Among them, gap represents the global average pooling layer used to extract global features, and conv.r is used to extract and nonlinearly transform channel features;

[0042] Next, the estimated weights are used for I i The pixel weighted allocation and channel weighted allocation are used to obtain I p and I c :

[0043]

[0044] Among them, I p Represents a pixel weighted feature of size 1×H×W, Ic Represents a channel weighted feature of size C×1×1; I p and I c The sum of passes through the convolution layer to obtain the final two-level weight feature I dw :

[0045] I dw =conv(I p +I c ) (6)

[0046] Among them, conv is used to balance the lighting characteristics and color information, I dw The size is C×H×W.

[0047] Furthermore, the number of iterations of the double-layer weight estimation unit is 4 times.

[0048] And, an underwater image enhancement system based on a multi-scale disentanglement strategy, based on a computer system, including: an underwater image enhancement network based on a multi-scale disentanglement strategy, the multi-scale disentanglement network consists of a Gaussian blur block, a transmission map disentanglement module, a scene radiation disentanglement module and a Jaffe-McGlamery underwater imaging model; the Gaussian blur block, the transmission map disentanglement module and the scene radiation disentanglement module respectively disentangle the original image into global background light, transmission map and scene radiation; the Jaffe-McGlamery underwater imaging model reconstructs the original image by three parameters of global background light, transmission map and scene radiation to supervise the disentanglement accuracy of the Gaussian blur block, the transmission map disentanglement module and the scene radiation disentanglement module.

[0049] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the underwater image enhancement method based on the multi-scale disentanglement strategy as described above are implemented.

[0050] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the underwater image enhancement method based on the multi-scale disentanglement strategy as described above.

[0051] Compared with the prior art, the present invention and its preferred embodiment propose a multi-scale disentanglement network (MD-Net) for underwater image enhancement (UIE). In the network, the scene radiation (clear image) and transmission map are first obtained by the scene radiation disentanglement (clear image) (SRD) module and the transmission map disentanglement (TMD) module. Then the three disentanglement parameters (including the global background light obtained by Gaussian blur) are coupled to the original image through the underwater imaging model, and the disentanglement performance of the network is improved by the self-supervision method. Among them, in SRD and TMD, the present invention designs a multi-level fusion module (MFM) and a double-layer weight estimation unit (DWEU) to further improve the network performance from the shallow spatial and deep semantic feature scales. MFM extracts spatial features from three layers with different receptive fields, and transmits shallow spatial information to high layers through jump connections. Then, the present invention uses iterative DWEU to estimate pixel weights and channel weights to obtain deep semantic features. The proposed method is compared with six UIE methods on three real UIE benchmark datasets (EUVP[2], LSUI[3] and UIEB[4]), including four non-data-driven methods (DCP[5], UDCP[6], UIEF[7], ACDC[8]) and three data-driven methods (Shallow-UWnet[9], PUIE-Net

[10] , P2CNet

[11] ). The superiority of the proposed method is demonstrated. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0053] Figure 1 This is a diagram of the network model architecture of an embodiment of the present invention;

[0054] Figure 2 This is a diagram of the MFM network model architecture of an embodiment of the present invention;

[0055] Figure 3 It is a shallow spatial information feature map of an embodiment of the present invention;

[0056] Figure 4 This is a characteristic diagram of MFM operation parameters according to an embodiment of the present invention;

[0057] Figure 5 This is a diagram of the DWEU network model architecture of an embodiment of the present invention;

[0058] Figure 6 Embodiment 1 of the present invention p ,I c and I dw Gray visual comparison chart;

[0059] Figure 7 This is a result diagram of different DWEU iteration times according to an embodiment of the present invention;

[0060] Figure 8 This is a qualitative comparison result diagram of ablation experiments in an embodiment of the present invention;

[0061] Fig. 9 This is a qualitative comparison result diagram of an embodiment of the present invention in a blue scene;

[0062] Fig.10 This is a qualitative comparison result diagram of the green scene of the embodiment of the present invention;

[0063] Fig.11 This is a qualitative comparison result diagram of an embodiment of the present invention in a yellow scene;

[0064] Fig.12 This is a qualitative comparison result diagram of an embodiment of the present invention in a low visibility scenario;

[0065] Fig.13 Graphs showing the experimental results of various underwater applications of embodiments of the present invention: from top to bottom they are underwater depth estimation, underwater edge detection, underwater key point detection, underwater saliency detection and underwater image segmentation. DETAILED DESCRIPTION

[0066] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail with the accompanying drawings as follows:

[0067] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0068] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0069] The embodiment of the present invention proposes an underwater image enhancement method based on a multi-scale disentanglement strategy. Based on a multi-scale disentanglement network (MD-Net), the transmission map and scene brightness are obtained respectively through a scene radiance disentanglement (SRD) module and a transmission map disentanglement (TMD) module. A multi-level fusion module (MFM) and a double-layer weight estimation unit (DWEU) are designed in the SRD and TMD. The MFM module extracts spatial features from three different receptive fields and transmits shallow spatial information to high layers through jump connections. Iterative DWEU is used to estimate pixel weights and channel weights to obtain deep semantic features. Then, the three disentangled components (transmission map, scene brightness and global background light) are coupled into the original image through an underwater imaging model [1], and the disentanglement performance is improved through a self-supervised method.

[0070] 1. Method Overview

[0071] The network architecture for implementing the method of the present invention is as follows Figure 1 As shown in the figure. MD-Net consists of a Gaussian blur (GB) block, a transmission map disentanglement (TMD) module, a scene radiance (clear image) disentanglement (SRD) module and a Jaffe-McGlamery underwater imaging model [1]. The GB, TMD and SRD modules disentangle the original image into global background light, transmission map and scene radiance (clear image) respectively. The Jaffe-McGlamery underwater imaging model [1] reconstructs the original image through the above three parameters to supervise the disentanglement accuracy of GB, TMD and SRD. Since the transmission map and scene radiance have the same dependence on the pixel value and contrast of the original image, MD-Net adopts a similar network structure for TMD and SRD. The steps of untangling the original image into the transmission map and the scene radiance (clear image) are as follows; the multi-level fusion module (MFM) uses different receptive fields (3×3, 5×5, 7×7) to extract shallow spatial features and multi-layer fusion features; then the multi-layer fusion features are input into the global average pooling layer (GAP) and the DWEU group to obtain down-sampled features and two-level weight features, and the two are multiplied to obtain deep semantic features. Finally, the deep semantic features obtained with different receptive fields are added, and the number of channels of the transmission map and the scene radiance (clear image) are adjusted using the adaptive convolution layer to obtain the final output.

[0072] 2Multi-level fusion module

[0073] Receptive fields of different sizes have different perception ranges for images. For example, a small receptive field can capture local detail features, but may need to include broader contextual information. A larger receptive field can capture more global information, but may ignore some detailed features. In order to combine their properties for underwater image enhancement, the structure of the multi-level fusion module (MFM) designed in the present invention is as follows: Figure 2As shown, it is used to obtain multi-level fusion features and shallow spatial features.

[0074] Specifically, MFM first captures different levels of spatial information features from the input image through convolutional layers with different receptive field sizes (3×3, 5×5, 7×7). Then Convblock-1 (Conv1) is used to further capture more detailed shallow information, and the formula is expressed as:

[0075]

[0076] Among them I S It refers to the general term for shallow spatial features. Each Conv1 contains two convolutional layers, a ReLU activation function and a normalization layer. It is worth noting that the convolution block performs standard convolution operations, which effectively reduces the computational cost associated with convolution. All shallow features such as Figure 3 shown.

[0077] Subsequently, the present invention performs a tensor concatenation operation and performs feature fusion through Convblock-2 (Conv2) to obtain dynamic weights. In order to assign weights to each branch, the present invention performs tensor segmentation on the dynamic weights:

[0078]

[0079] Among them, W1, W2, and W3 are the branch weights after segmentation, spl represents tensor cutting, cat represents tensor concatenation, and Conv2 contains two convolutional layers, a ReLU activation function, and a Sigmoid function. Figure 4 As shown, and Figure 3 In and Compared with the traditional fusion method, the dynamic weighting of W1, W2 and W3 achieves better results than direct feature fusion. Finally, the multi-layer fusion feature I mf The image feature information of receptive fields of different scales is fully integrated.

[0080] 3 Two-layer weight estimation unit

[0081] In order to understand the more complex relationship between lighting characteristics and color information. After the MFM output, the present invention iteratively applies DWEU, which integrates channel weight estimation and pixel weight estimation. The DWEU structure is as follows Figure 5 As shown, it is used to capture the dual-level weight features.

[0082] Specifically, the present invention first uses a convolutional layer with a kernel size of 3×3 to extract features from the multi-level fusion features. Then, the present invention inputs them into the channel estimation unit and the pixel estimation unit respectively to obtain the pixel weight W p and channel weight W c . The pixel weight estimation can be expressed as:

[0083] W p =conv.s(conv.ri(I i )) (3)

[0084] Among them I i It is the feature extracted from the convolution. conv.ri contains a convolution layer with a convolution kernel size of 3×3 for processing image features, followed by a ReLU activation for nonlinear transformation and a normalization layer for standardizing feature maps. conv.s contains a convolution layer with a kernel size of 3×3 for further processing feature maps, followed by a Sigmoid function to compress the output value into the range of [0, 1]. The channel weight formula can be expressed as:

[0085] W c =conv.s(conv.r(gap(I i ))) (4)

[0086] Where gap represents the global average pooling layer used to extract global features, and conv.r is used to extract and nonlinearly transform channel features. Next, the estimated weights are used for I i The pixel weighted allocation and channel weighted allocation are used to obtain I p and I c :

[0087]

[0088] Among them I p Represents a pixel weighted feature of size 1×H×W, I c represents a channel weighted feature of size C×1×1. p and I c The sum of passes through the convolution layer to obtain the final two-level weight feature I dw :

[0089] I dw =conv(I p +I c ) (6)

[0090] Where conv is used to balance lighting features and color information, I dw The size is C×H×W. Figure 6 Show I p ,Ic and I dw Gray visual contrast.

[0091] Based on the above operations, one iteration of DWEU has been completed. Figure 7 A visual comparison of DWEU after 4 iterations is shown. Figure 4 (h) Compared with Figure 7 (a) shows better color restoration and low-light enhancement, confirming that the construction of a two-layer weight estimation unit can significantly improve visibility and color restoration. It is worth noting that four iterations are sufficient to produce satisfactory underwater color and image visibility.

[0092] In summary, an embodiment of the present invention proposes a multi-scale disentanglement network (MD-Net) for underwater image enhancement. In the network, the scene radiation (clear image) and the transmission map are first obtained through the scene radiation disentanglement (clear image) (SRD) module and the transmission map disentanglement (TMD) module. Then the three disentanglement parameters (including the global background light obtained by Gaussian blur) are coupled to the original image through the underwater imaging model, and the disentanglement performance of the network is improved by the self-supervision method. Among them, in SRD and TMD, the present invention designs a multi-level fusion module (MFM) and a double-layer weight estimation unit (DWEU) to further improve the network performance from the shallow spatial and deep semantic feature scales. MFM extracts spatial features from three layers with different receptive fields, and transmits shallow spatial information to high layers through jump connections. Then, the present invention uses iterative DWEU to estimate pixel weights and channel weights to obtain deep semantic features. The proposed method is compared with six UIE methods on three real UIE benchmark datasets (EUVP[2], LSUI[3] and UIEB[4]), including four non-data-driven methods (DCP[5], UDCP[6], UIEF[7], ACDC[8]) and three data-driven methods (Shallow-UWnet[9], PUIE-Net

[10] , P2CNet

[11] ). A large number of experiments demonstrate the superiority of the proposed method.

[0093] The effects of the embodiments of the present invention are further described and verified by experiments and test examples as follows:

[0094] 1 Experimental data and evaluation

[0095] The performance evaluation of the present invention is based on real underwater image enhancement (UIE) benchmark datasets, including EUVP[2], LSUI[3] and UIEB[4] datasets. Each dataset is divided into a training set and a test set. The training set contains 11,206 images of the EUVP[2] dataset, 3,794 images of the LSUI dataset, and 800 images of the UIEB dataset. The test set uses the remaining 760 images (Test-760) of the EUVP[2] dataset, the remaining 485 images (Test-485) of the LSUI[3] dataset, and the remaining 90 images (Test-90) of the UIEB[4] dataset.

[0096] This embodiment uses four metrics to measure the performance of different methods on Test-485 and Test-90. These metrics include full-reference and no-reference metrics, namely Peak Signal-to-Noise Ratio (PSNR)

[12] , Structural Similarity Index (SSIM)

[13] , Underwater Color Image Quality Evaluation (UCIQE)

[14] , and Underwater Image Quality Measure (UIQM)

[15] . Higher PSNR

[12] and SSIM

[13] scores indicate that the enhanced image is closer to the reference image in terms of content and structure; higher UCIQE

[14] and UIQM

[15] values ​​indicate a better balance in terms of color vividness, sharpness, and contrast.

[0097] 2 Ablation studies

[0098] Table 1 Average PSNR

[12] and SSIM

[13] scores of ablation experiments on Test-90.

[0099] ModelTerms PSNR↑ SSIM↑ Our-settingI 12.4288 0.3290 Our-settingII 21.7992 0.7823 Our 22.2697 0.88161

[0100] This example conducts comprehensive ablation experiments on blue, green, yellow and low visibility degradation scenes, and systematically analyzes the impact of each component in MD-Net. There are three settings: 1) MD-Net without SRD and disentanglement strategy (Our-settingI), 2) MD-Net without disentanglement (Our-settingII), 3) complete method (Our). Table 1 shows the quantitative comparison of ablation experiments. In addition, Figure 8 Shows the visual effects of the ablation experiment.

[0101] 3 Qualitative and quantitative comparison

[0102] This embodiment uses PSNR

[12] , SSIM

[13] , UCIQE

[14] and UIQM

[15] to evaluate the performance of various UIE methods on Test-485 and Test-90. Tables 2 and 3 give the quantitative evaluation results on the LSUI[4] and UIEB[3] datasets, respectively. For PSNR

[12] and SSIM

[13] , the algorithm of the present invention produces four best scores, which shows that the results of the present invention are highly similar to the content and structure of the reference images. For UCIQE

[14] , compared with other existing schemes, the algorithm of the present invention obtains the best score and the second best score, which shows that the proposed method alleviates non-uniform color cast, reduces chromatic aberration blur and enhances contrast. In addition, the algorithm of the present invention locks two second-best UIQM

[15] scores, demonstrating enhanced excellence in color, clarity and contrast. Figures 9 to 12 Qualitative results are shown for various underwater scenes on the EUVP[2], LSUI[3], and UIEB[4] datasets.

[0103] Table 2 Quantitative evaluation results on the LSUI dataset, the best score and the second best score are marked in bold and italics respectively.

[0104]

[0105] Table 3 Quantitative evaluation results on the UIEB dataset, the best score and the second best score are marked in bold and italics respectively.

[0106]

[0107] 4 Complexity and running time

[0108] In order to measure the computational efficiency of the present invention, this embodiment compares FLOPs (G), training parameters (M) and single image processing speed (s). As shown in Tables 4 and 5. Although the FLOPs (G) of the algorithm of the present invention is not as good as other competitors at image sizes of 256×256, 512×512 and 1024×1024, its architecture parameters show excellent performance. At the same time, Table 4 also shows that the algorithm of the present invention achieves good processing speed across different image sizes.

[0109] Table 4 Comparison of FLOPs (G) and training parameters (M). The best score and the second best score are marked in bold and italics respectively.

[0110]

[0111] Table 5. Comparison of running time for a single image. The best and second-best scores are marked in bold and italics, respectively.

[0112] Methods 256×256 512×512 1024×1024 DCP[5] 1.1190 4.6559 19.0440 UDCP[6] 1.0511 4.3603 18.9691 Shallow-UWnet[9] 0.0221 0.0250 0.0337 UIEF[7] 0.5818 1.4439 6.4266 ACDC[8] 0.7574 2.6709 11.4419 PUIE-Net

[10] 0.0044 0.0124 0.0452 P2CNet

[11] 0.0319 0.3184 0.3715 Algorithm of the present invention 0.0153 0.0622 0.2562

[0113] 5 Application Experiment

[0114] This embodiment demonstrates the effectiveness of the algorithm of the present invention through various underwater application tests, including depth estimation, edge detection, key point detection, saliency detection, and image segmentation. For underwater depth estimation, the present invention uses non-local priors

[16] , uses the Canny operator

[17] for underwater edge detection, uses SIFT key point detection

[18] for underwater key point detection, uses BASNet

[19] for underwater saliency detection, and applies a superpixel-based underwater clustering algorithm

[20] for image segmentation. All results are shown in Figure 2. Fig.13 As shown. The algorithm of the present invention detects more key points and captures clearer structures. In addition, the segmentation results of this embodiment have higher consistency and accuracy, and its saliency map contains more clear objects and clear boundaries. These experiments show that the algorithm of the present invention can effectively enhance both underwater image segmentation and saliency detection.

[0115] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0116] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0117] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0119] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive other various forms of underwater image enhancement methods based on multi-scale disentanglement strategies under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.

Claims

1. An underwater image enhancement method based on a multi-scale disentanglement strategy, characterized in that: A multi-scale disentanglement network is adopted. Firstly, the transmission map and scene brightness are obtained respectively through the scene radiation disentanglement module and the transmission map disentanglement module. Then, the three disentangled components including the transmission map, scene brightness and global background light are coupled into the original image through the Jaffe-McGlamery underwater imaging model, and the disentanglement performance is improved through the self-supervision method.

2. The underwater image enhancement method based on the multi-scale disentanglement strategy according to claim 1, characterized in that: A multi-level fusion module and a double-layer weight estimation unit are provided in the scene radiation unwrapping module and the transmission map unwrapping module. The multi-level fusion module is used to extract spatial features from three layers of different receptive fields, and transmit shallow spatial information to high layers through jump connections; the double-layer weight estimation unit is used to estimate pixel weights and channel weights through one or more iterations to obtain deep semantic features.

3. The underwater image enhancement method based on the multi-scale disentanglement strategy according to claim 1, characterized in that: The multi-scale disentanglement network consists of a Gaussian blur block, a transmission map disentanglement module, a scene radiation disentanglement module and a Jaffe-McGlamery underwater imaging model; the Gaussian blur block, the transmission map disentanglement module and the scene radiation disentanglement module respectively disentangle the original image into global background light, transmission map and scene radiation; the Jaffe-McGlamery underwater imaging model reconstructs the original image by three parameters of global background light, transmission map and scene radiation to supervise the disentanglement accuracy of the Gaussian blur block, the transmission map disentanglement module and the scene radiation disentanglement module.

4. The underwater image enhancement method based on the multi-scale disentanglement strategy according to claim 2, characterized in that: The transmission map unwrapping module and the scene radiation unwrapping module adopt the same network structure; the process of unwrapping the original image into the transmission map unwrapping module or the scene radiation unwrapping module is: using three different receptive fields of 3×3, 5×5, and 7×7 through a multi-level fusion module to extract shallow spatial features and multi-layer fusion features; then the multi-layer fusion features are input into the global average pooling layer and the double-layer weight estimation unit to obtain down-sampled features and double-layer weight features respectively, the down-sampled features and the double-layer weight features are multiplied to obtain deep semantic features, and finally the deep semantic features corresponding to the three different receptive fields are added, and the number of transmission map or scene radiation channels is adjusted by an adaptive convolution layer to obtain the final output.

5. The underwater image enhancement method based on the multi-scale disentanglement strategy according to claim 4 is characterized in that: The multi-level fusion module first captures spatial information features at different levels from the input image through three convolutional layers with different receptive field sizes: 3×3, 5×5, and 7×7. Then Convblock-1(Conv1) is used to further capture more detailed shallow information: Among them, I S Represents shallow spatial features. Each Conv1 contains two convolutional layers, a ReLU activation function and a normalization layer. Subsequently, the tensor concatenation operation is performed and feature fusion is performed through Convblock-2 (Conv2) to obtain dynamic weights. In order to assign weights to each branch, tensor splitting is performed on the dynamic weights: Among them, W1, W2, and W3 correspond to the branch weights after segmentation, spl represents tensor cutting, cat represents tensor concatenation, and Conv2 contains two convolutional layers, a ReLU activation function, and a Sigmoid function. Finally, the multi-layer fusion feature I obtained by fusing W1, W2, and W3 is mf To integrate image feature information of receptive fields of different scales.

6. The underwater image enhancement method based on the multi-scale disentanglement strategy according to claim 4, characterized in that: The iterative process of the two-layer weight estimation unit is as follows: first, a convolutional layer with a kernel size of 3×3 is used to obtain the multi-layer fusion feature I mf Then, they are input into the channel estimation unit and the pixel estimation unit respectively to obtain the pixel weight W p and channel weight W c ; The pixel weight estimation is expressed as: W p =conv.s(conv.r.i(I i )) (3) Among them, I i It is the feature extracted from the convolution. conv.ri contains a convolution layer with a convolution kernel size of 3×3, which is used to process image features, followed by a ReLU activation for nonlinear transformation and a normalization layer for standardizing feature maps. conv.s contains a convolution layer with a kernel size of 3×3, which is used to further process the feature map, followed by a Sigmoid function to compress the output value into the range of [0, 1]; The channel weight estimation is expressed as: W c =conv.s(conv.r(.gap(I i ))) (4) Among them, gap represents the global average pooling layer used to extract global features, and conv.r is used to extract and nonlinearly transform channel features; Next, the estimated weights are used for I i The pixel weighted allocation and channel weighted allocation are used to obtain I p and I c : Among them, I p Represents a pixel weighted feature of size 1×H×W, I c Represents a channel weighted feature of size C×1×1; I p and I c The sum of passes through the convolution layer to obtain the final two-level weight feature I dw : I dw =conv(I p +I c ) (6) Among them, conv is used to balance the lighting characteristics and color information, I dw The size is C×H×W.

7. The underwater image enhancement method based on the multi-scale disentanglement strategy according to claim 6, characterized in that: The number of iterations of the double-layer weight estimation unit is 4.

8. An underwater image enhancement system based on a multi-scale disentanglement strategy, characterized in that: Based on a computer system, the invention comprises: an underwater image enhancement network based on a multi-scale disentanglement strategy, wherein the multi-scale disentanglement network consists of a Gaussian blur block, a transmission map disentanglement module, a scene radiation disentanglement module and a Jaffe-McGlamery underwater imaging model; the Gaussian blur block, the transmission map disentanglement module and the scene radiation disentanglement module respectively disentangle the original image into global background light, transmission map and scene radiation; the Jaffe-McGlamery underwater imaging model reconstructs the original image by three parameters of global background light, transmission map and scene radiation to supervise the disentanglement accuracy of the Gaussian blur block, the transmission map disentanglement module and the scene radiation disentanglement module.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the underwater image enhancement method based on the multi-scale disentanglement strategy as described in any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the underwater image enhancement method based on a multi-scale disentanglement strategy as described in any one of claims 1 to 7 are implemented.

Images