A collaborative multi-scale underwater image enhancement method based on internal supervision

Abstract

The application provides a kind of collaborative multiscale underwater image enhancement method based on internal supervision, by integrating four stage images, different scale scene information is provided for network.First, the input image is used to obtain first, second and third degradation images through degradation operation respectively.The initial features of the input image are extracted using the context attention module based on double branch, and then the first degradation features are obtained after down-sampling using the context attention module based on double branch;The features conducive to scene recovery are selected using the adaptive selective internal supervision module on the first degradation image, and integrated into the first degradation features.The second and third degradation stages are repeated twice using the first degradation features.The third degradation features are recovered to integrate the second degradation stage for secondary feature recovery.The operation is repeated twice using the second and first degradation stages.After obtaining the first degradation features, the enhanced image is restored.Based on the multi-stage degradation loss function, the enhancement process of the four stages is constrained.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more particularly to a collaborative multi-scale underwater image enhancement method based on internal supervision. Background Technology

[0002] With continuous population growth and rapid economic development, terrestrial resources are becoming increasingly depleted and unable to meet the ever-growing needs of humanity. In contrast, underwater environments such as oceans, rivers, and lakes possess abundant resource potential. Therefore, the development, surveying, and protection of underwater resources have become a focus of widespread international attention. However, underwater imaging technology is affected by degrading factors such as scattering, absorption, and changes in illumination, leading to decreased image and video quality, loss of detail, and increased noise. Therefore, improving the visual quality and recognizability of underwater images is an urgent task.

[0003] In recent years, underwater image enhancement methods have been widely used in computer vision and image processing. Many scholars at home and abroad have also studied this topic and made corresponding progress. Among them, underwater image enhancement methods can be divided into traditional underwater enhancement methods and deep learning-based underwater image enhancement methods.

[0004] Traditional underwater image enhancement methods include physical model-based underwater image restoration methods and underwater image enhancement methods. Due to the unique underwater optical environment, directly applying traditional image enhancement methods to image enhancement has certain limitations. Existing underwater image enhancement methods can only be applied to specific applications or have limited image enhancement capabilities; the enhanced image will lose some information, and it is difficult to balance the relationship between image contrast and saturation. Physical model-based underwater image restoration methods analyze the imaging process and obtain a clear image through inverse operations on the imaging model, improving image quality from the imaging principle. Existing restoration methods often rely on prior knowledge to estimate transmission maps and atmospheric light; however, without sufficient prior knowledge or with inaccurate prior knowledge, they cannot obtain the correct solution.

[0005] Deep learning-based methods can be categorized into image restoration methods based on model parameter estimation and end-to-end underwater image enhancement methods. Image restoration methods based on model parameter estimation primarily learn the parameters of an underwater image restoration model by training a deep neural network. Typically, the model takes a degraded underwater image as input and infers the restored image through the network. However, this type of method requires prior knowledge or an explicit degradation model for training, and the complexity and uncertainty of the underwater environment may limit the model's generalization ability.

[0006] In end-to-end underwater image enhancement methods, multiple stages of degradation input can provide additional scene information for underwater scene degradation, while avoiding detailed feature loss caused by multiple feature extraction. Summary of the Invention

[0007] To address the technical problems mentioned in the background section, this invention provides a collaborative multi-scale underwater image enhancement method based on internal supervision. This invention primarily integrates image inputs from four stages, providing the network with scene information and relevant features at different scales.

[0008] The technical means employed in this invention are as follows:

[0009] A collaborative multi-scale underwater image enhancement method based on internal supervision includes the following steps:

[0010] S01: Acquire the input underwater image, and perform degradation operations on the input underwater image I to acquire the first-level degradation image I2, the second-level degradation image I3, and the third-level degradation image I4 respectively;

[0011] S02: Input the underwater image I into the initial feature extraction module to extract the initial feature E1. At the same time, input the first-level degraded image I2, the second-level degraded image I3, and the third-level degraded image I4 into the feature extraction module to extract the corresponding first-level degraded image feature G21, second-level degraded image feature G31, and third-level degraded image feature G41, respectively.

[0012] S03: The obtained initial feature E1 is downsampled and then input into the dual-branch context attention module to obtain the first-level degradation feature E2;

[0013] S04: Input the first-level degraded image feature G21 into the adaptive selective intrinsic supervision module for feature selection, obtain the image feature reconstruction feature G22, and then add it together with the first-level degraded feature E2 to obtain the first-level comprehensive feature Z2;

[0014] S05: Perform S03 and S04 respectively on the secondary degraded image I3 and the tertiary degraded image I4 to obtain the corresponding secondary comprehensive features Z3 and tertiary comprehensive features Z4 for feature reconstruction;

[0015] S06: The third-level degraded image features G41 are enhanced by a dual-branch context attention module and selected by an adaptive selective intrinsic supervision module to obtain the third-level degraded image restoration features R4;

[0016] S07: The three-level comprehensive feature Z4 is added to the three-level degraded image restoration feature R4 and input into the dual-branch context attention module for feature restoration to obtain the three-level restoration feature D4;

[0017] S08: The secondary degraded image features G31 are enhanced by a dual-branch context attention module and selected by an adaptive selective intrinsic supervision feature module to obtain secondary degraded image restoration features R3 that are beneficial for feature restoration.

[0018] S09: Upsample the third-level restoration feature D4, superimpose the second-level comprehensive feature Z3 and the second-level degraded image restoration feature R3, and input them into the dual-branch context attention module for feature restoration to obtain the first-level degraded image restoration feature R2;

[0019] S10: Repeat S08 and S09 using the first-level degraded image feature G21, the second-level degraded image restoration feature R3 and the first-level comprehensive feature Z2 to obtain the image restoration feature R1;

[0020] S11: Input the image restoration feature R1 into the dual-branch context attention module and convolutional layer to restore the image and obtain the enhanced image J;

[0021] S12: Repeat the S11 operation using the first-level degraded image restoration feature R2, the second-level degraded image restoration feature R3, and the third-level degraded image restoration feature R4 to obtain the corresponding enhanced images J2, J3, and J4;

[0022] S13: Use a multi-stage degradation loss function to constrain the four-stage enhancement process.

[0023] Furthermore, the degradation operation in S01 includes: a downsampling operation; where I∈R H×W×3 Let R represent the size of the image, H represent the height and width of the image, and W represent the height and width of the image, respectively. I² ∈ R H / 2×W / 2×3 , I3∈R H / 4×W / 4×3 , I4∈R H / 8×W / 8×3 .

[0024] Furthermore, the feature extraction module in S02 includes a 3*3 convolutional layer with 3 input channels and 32 output channels, and a dual-branch context attention module; the dual-branch context attention module is as follows:

[0025] S021: Branch 1 of the dual-branch context attention module, which integrates the feature attention module and the resolution-guided intrinsic attention module. The formula for the integrated feature attention module is as follows:

[0026] f z (x)=Conv(concat(CA(x)×x,PA(x)×x)); (1)

[0027] Where x represents the input feature, f zThe expression represents a comprehensive feature attention operation. Conv indicates a convolutional layer with the same number of channels in the input and output feature maps. CA indicates a channel attention mechanism, PA indicates a spatial attention mechanism, and concat indicates concatenating the feature maps along the channel dimension.

[0028] S022: Based on the comprehensive feature attention module, the resolution-guided intrinsic attention module is formulated as follows:

[0029] f u (x)=Sigmoid(Up(Conv(Down(Conv(x)))))×x; (2)

[0030] Where x represents the input feature, f u This represents the resolution-guided intrinsic attention operation, where Conv represents a convolutional layer with the same number of channels in both the input and output feature maps, Down represents the downsampling operation, and Up represents the upsampling operation, where Sigmoid(x) = 1 / (1+e^(-1 / 2)). -x );

[0031] S023: Based on the comprehensive feature attention module and the resolution-guided intrinsic attention module, the formula for branch one of the dual-branch context attention module is as follows:

[0032] f1(x)=f u (f z (x); (3)

[0033] Where x represents the input feature and f1 represents the branch-one operation;

[0034] S024: In the second branch of the dual-branch context attention module, the context-aware feature extraction module has the following formula:

[0035] f2(x)=Conv(Conv(x)+DConv(x)); (4)

[0036] Where x represents the input feature, f2 represents the context-aware feature extraction operation, and DConv represents the dilated convolution operation;

[0037] S025: Based on the branch one formula and the branch two formula, the formula for the dual-branch context attention module is as follows:

[0038] f b (x)=Conv(Gelu(concat(f1(x),f2(x)))); (5)

[0039] Where x represents the input feature, f bThis represents a context attention operation based on a two-branch architecture, where DConv represents a dilated convolution operation, and Gelu represents the Gelu activation function, as shown in the following formula:

[0040]

[0041] The feature extraction module in S02 obtains initial feature E1 by performing feature extraction on the input underwater image I, as shown in the following formula:

[0042] E1 = f b (Conv(I)); (7)

[0043] G21=f b (Conv(I2)); (8)

[0044] G31=f b (Conv(I3)); (9)

[0045] G41=f b (Conv(I4)); (10)

[0046] Where I represents the input underwater image, I2, I3, and I4 represent the first, second, and third level degradation images, respectively, and f b This represents a context attention operation based on two branches, where E1 represents the initial feature, and G21, G31, and G41 represent the first-level, second-level, and third-level degraded image features, respectively.

[0047] Furthermore, the initial feature E1 in S03 is downsampled and then input into the dual-branch context attention module, as shown in the following formula:

[0048] E2=f b (E1); (11)

[0049] Here, E2 represents the first-level degradation characteristic.

[0050] Furthermore, the formula for the adaptive selective intrinsic supervision module in S04 is as follows:

[0051]

[0052] Where x represents the input feature, r represents the reference feature, and f s This represents an adaptive selective intrinsic supervision operation;

[0053] The first-level degraded image feature G21 is input into the adaptive selective intrinsic supervised feature module for feature selection, as shown in the following formula:

[0054] G22=f s (G21,E2); (13)

[0055] Then, it is superimposed with the first-level degenerate feature E2 to obtain the first-level comprehensive feature Z2, as shown in the following formula:

[0056] Z2 = G22 + E2. (14)

[0057] Further, in step S07, the third-level comprehensive feature Z4, superimposed with the third-level degraded image restoration feature R4, is input into the bi-branch context attention module for feature restoration to obtain the third-level restoration feature D4, as shown in the following formula:

[0058] D4 = f b (Z4+R4). (15)

[0059] Furthermore, in step S08, the secondary degraded image feature G31 is enhanced using a bi-branch contextual attention module and selected using an adaptive selective intrinsic supervision feature module to obtain the secondary degraded image restoration feature R3, which is beneficial for feature restoration. The formula is as follows:

[0060] R3 = f s (f b (G31)). (16)

[0061] Further, in step S09, D4 is upsampled, and the secondary comprehensive feature Z3 and the secondary degraded image restoration feature R3 are superimposed and input into the bi-branch context attention module for feature restoration to obtain the primary degraded image restoration feature R2, as shown in the following formula:

[0062] R2=f b (Up(D4)+Z3+R3). (17)

[0063] Further, in step S11, the image restoration feature R1 is input into a dual-branch context attention module and a convolutional layer to restore the image and obtain the enhanced image J, as shown in the following formula:

[0064] J = Conv(f b (R1)). (18)

[0065] Furthermore, in step S12, the corresponding enhanced images J2, J3, and J4 are obtained by repeating the operation of step S11 using the first-level degraded image restoration feature R2, the second-level degraded image restoration feature R3, and the third-level degraded image restoration feature R4, as shown in the following formula:

[0066] J2 = Conv(f b (R2)); (19)

[0067] J3 = Conv(f b(R3); (20)

[0068] J4 = Conv(f b (R4)). (21)

[0069] Compared with the prior art, the present invention has the following advantages:

[0070] 1. To address the problem that current underwater image enhancement methods often suffer from incomplete scene detail recovery due to the limited use of scale-related features, we propose an internally supervised collaborative multi-scale detail optimization framework for underwater image enhancement.

[0071] 2. This invention designs a dual-branch contextual attention module within multi-scale details and frameworks. Branch one enhances image detail features while fully utilizing contextual semantic features. Branch two employs resolution-guided intrinsic attention, effectively expanding the network's receptive field.

[0072] 3. The adaptive selective intrinsic supervision feature module designed in this invention can effectively select low-resolution feature information and avoid feature redundancy caused by the superposition of multi-scale features.

[0073] 4. This invention designs a multi-degradation loss function, which can effectively constrain and optimize the feature extraction and enhancement process at multiple degradation stages.

[0074] Based on the reasons stated above, this invention can be widely applied in fields such as image processing. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a schematic diagram of the process of the present invention.

[0077] Figure 2 This image shows a comparison of the enhancement effects of this invention with other algorithms on underwater scene images.

[0078] Figure 3 This image shows a comparison of the enhancement effects of this invention with other algorithms on underwater scene images.

[0079] Figure 4 This image shows a comparison of the enhancement effects of this invention with other algorithms on underwater scene images.

[0080] in, Figure 2-1 , Figure 3-1 , Figure 4-1 Input images for underwater use; Figure 2-2 , Figure 3-2 , Figure 4-2 This is the result image after processing using the WaterNet method; Figure 2-3 , Figure 3-3 , Figure 4-3 This is the result image after processing using the UWCNN method; Figure 2-4 , Figure 3-4 , Figure 4-4 This is the result image after processing using the Ucolor method; Figure 2 -5、 Figure 3 -5、 Figure 4 -5 is the result image after processing using the PUIE method; Figure 2 -6、 Figure 3 -6、 Figure 4 -6 is the result image after processing by the algorithm proposed in this paper. Detailed Implementation

[0081] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0082] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of units or elements is not necessarily limited to those explicitly listed, but may include other units or elements not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0083] like Figure 1 As shown, this invention provides a collaborative multi-scale underwater image enhancement method based on internal supervision, comprising the following steps:

[0084] S01: Acquire the input underwater image, and perform degradation operations on the input underwater image I to obtain a first-level degradation image I2, a second-level degradation image I3, and a third-level degradation image I4 respectively; the degradation operation in S01 includes: downsampling operation; where I∈R H×W×3 Let R represent the size of the image, H represent the height and width of the image, and W represent the height and width of the image, respectively. I² ∈ R H / 2×W / 2×3 , I3∈R H / 4×W / 4×3 , I4∈R H / 8×W / 8×3 .

[0085] S02: Input the underwater image I into the initial feature extraction module to extract the initial feature E1. At the same time, input the first-level degraded image I2, the second-level degraded image I3, and the third-level degraded image I4 into the feature extraction module to extract the corresponding first-level degraded image feature G21, second-level degraded image feature G31, and third-level degraded image feature G41, respectively.

[0086] The feature extraction module in S02 includes a 3*3 convolutional layer with 3 input channels and 32 output channels, and a dual-branch context attention module; the dual-branch context attention module is as follows:

[0087] S021: Branch 1 of the dual-branch context attention module, which integrates the feature attention module and the resolution-guided intrinsic attention module. The formula for the integrated feature attention module is as follows:

[0088] f z (x)=Conv(concat(CA(x)×x,PA(x)×x)); (1)

[0089] Where x represents the input feature, f z The expression represents a comprehensive feature attention operation. Conv indicates a convolutional layer with the same number of channels in the input and output feature maps. CA indicates a channel attention mechanism, PA indicates a spatial attention mechanism, and concat indicates concatenating the feature maps along the channel dimension.

[0090] S022: Based on the comprehensive feature attention module, the resolution-guided intrinsic attention module is formulated as follows:

[0091] f u (x)=Sigmoid(Up(Conv(Down(Conv(x)))))×x; (2)

[0092] Where x represents the input feature, f uThis represents the resolution-guided intrinsic attention operation, where Conv represents a convolutional layer with the same number of channels in both the input and output feature maps, Down represents the downsampling operation, and Up represents the upsampling operation, where Sigmoid(x) = 1 / (1+e^(-1 / 2)). -x );

[0093] S023: Based on the comprehensive feature attention module and the resolution-guided intrinsic attention module, the formula for branch one of the dual-branch context attention module is as follows:

[0094] f1(x)=f u (f z (x); (3)

[0095] Where x represents the input feature and f1 represents the branch-one operation;

[0096] S024: In the second branch of the dual-branch context attention module, the context-aware feature extraction module has the following formula:

[0097] f2(x)=Conv(Conv(x)+DConv(x)); (4)

[0098] Where x represents the input feature, f2 represents the context-aware feature extraction operation, and DConv represents the dilated convolution operation;

[0099] S025: Based on the branch one formula and the branch two formula, the formula for the dual-branch context attention module is as follows:

[0100] f b (x)=Conv(Gelu(concat(f1(x),f2(x)))); (5)

[0101] Where x represents the input feature, f b This represents a context attention operation based on a two-branch architecture, where DConv represents a dilated convolution operation, and Gelu represents the Gelu activation function, as shown in the following formula:

[0102]

[0103] The feature extraction module in S02 obtains initial feature E1 by performing feature extraction on the input underwater image I, as shown in the following formula:

[0104] E1 = f b (Conv(I)); (7)

[0105] G21=f b (Conv(I2)); (8)

[0106] G31=fb (Conv(I3)); (9)

[0107] G41=f b (Conv(I4)); (10)

[0108] Where I represents the input underwater image, I2, I3, and I4 represent the first, second, and third level degradation images, respectively, and f b This represents a context attention operation based on two branches, where E1 represents the initial feature, and G21, G31, and G41 represent the first-level, second-level, and third-level degraded image features, respectively.

[0109] S03: The obtained initial feature E1 is downsampled and then input into the dual-branch context attention module to obtain the first-level degenerate feature E2; the formula for downsampling the initial feature E1 in S03 and inputting it into the dual-branch context attention module is as follows:

[0110] E2=f b (E1); (11)

[0111] Here, E2 represents the first-level degradation characteristic.

[0112] S04: The first-level degraded image feature G21 is input into the adaptive selective intrinsic supervision module for feature selection to obtain the image feature reconstruction feature G22, which is then added to the first-level degraded feature E2 to obtain the first-level comprehensive feature Z2. The formula for the adaptive selective intrinsic supervision module in S04 is as follows:

[0113]

[0114] Where x represents the input feature, r represents the reference feature, and f s This represents an adaptive selective intrinsic supervision operation;

[0115] The first-level degraded image feature G21 is input into the adaptive selective intrinsic supervised feature module for feature selection, as shown in the following formula:

[0116] G22=f s (G21,E2); (13)

[0117] Then, it is superimposed with the first-level degenerate feature E2 to obtain the first-level comprehensive feature Z2, as shown in the following formula:

[0118] Z2 = G22 + E2. (14)

[0119] S05: Perform S03 and S04 respectively on the secondary degraded image I3 and the tertiary degraded image I4 to obtain the corresponding secondary comprehensive features Z3 and tertiary comprehensive features Z4 for feature reconstruction;

[0120] S06: The third-level degraded image features G41 are enhanced by a dual-branch context attention module and selected by an adaptive selective intrinsic supervision module to obtain the third-level degraded image restoration features R4;

[0121] S07: The three-level comprehensive feature Z4 is added to the three-level degraded image restoration feature R4 and input into the bi-branch context attention module for feature restoration to obtain the three-level restoration feature D4; in S07, the three-level comprehensive feature Z4 is superimposed with the three-level degraded image restoration feature R4 and input into the bi-branch context attention module for feature restoration to obtain the three-level restoration feature D4, as shown in the following formula:

[0122] D4 = f b (Z4+R4). (15)

[0123] S08: The secondary degraded image features G31 are enhanced using a bi-branch contextual attention module and selected using an adaptive selective intrinsic supervision feature module to obtain secondary degraded image restoration features R3 that are beneficial for feature recovery; the formula for obtaining secondary degraded image restoration features R3 that are beneficial for feature recovery in S08 is as follows:

[0124] R3 = f s (f b (G31)). (16)

[0125] S09: Upsample the third-level restoration feature D4, superimpose the second-level comprehensive feature Z3 and the second-level degraded image restoration feature R3, and input them into the bi-branch context attention module for feature restoration to obtain the first-level degraded image restoration feature R2; In S09, upsample the D4, superimpose the second-level comprehensive feature Z3 and the second-level degraded image restoration feature R3, and input them into the bi-branch context attention module for feature restoration to obtain the first-level degraded image restoration feature R2, as shown in the following formula:

[0126] R2=f b (Up(D4)+Z3+R3). (17)

[0127] S10: Repeat S08 and S09 using the first-level degraded image feature G21, the second-level degraded image restoration feature R3 and the first-level comprehensive feature Z2 to obtain the image restoration feature R1;

[0128] S11: Input the image restoration feature R1 into the dual-based...

[0129] The context attention module and convolutional layer of the branch perform image restoration to obtain the enhanced image J; in S11, the image restoration feature R1 is input into the context attention module and convolutional layer based on the two branches to perform image restoration to obtain the enhanced image J, as shown in the following formula:

[0130] J = Conv(f b (R1)). (18)

[0131] S12: The corresponding enhanced images J2, J3, and J4 are obtained by repeating operation S11 using the first-level degraded image restoration feature R2, the second-level degraded image restoration feature R3, and the third-level degraded image restoration feature R4. The formula for obtaining the enhanced images J2, J3, and J4 in S12 is as follows:

[0132] J2 = Conv(f b (R2)); (19)

[0133] J3 = Conv(f b (R3); (20)

[0134] J4 = Conv(f b (R4)). (21)

[0135] S13: Use a multi-stage degradation loss function to constrain the four-stage enhancement process.

[0136] Example 1

[0137] like Figure 2 As shown, the first image is first processed using various algorithms; Figure 2-1 This is the original underwater image. Figure 2-2 The image shows the result after processing with the WaterNet method. Although it restores the color cast of the underwater image to some extent, the overall image is still dark. Figure 2-3 The image shows the result after processing with the UWCNN method. It can be seen that the image saturation is reduced and the overall image has a purple tint. Figure 2-4 The image shows the result after processing with the Ucolor method. It can be seen that the contrast and saturation of the image have been enhanced, but there is still a problem of insufficient detail enhancement. Figure 2-5 The result image after processing with the PUIE method effectively enhances the image details, but still suffers from low color saturation. Figure 2 -6 shows the result image after processing by the algorithm in this paper. It can be seen that the algorithm can effectively enhance the detail information of the image, and at the same time, it can effectively restore the color cast of the underwater image and enhance the color saturation.

[0138] The second image is then processed using various algorithms; Figure 3-1 This is the original underwater image. Figure 3-2 The image shows the result after processing with the WaterNet method. It can be seen that the overall image has an incorrect yellow tint. Figure 3-3 The image is the result after processing with the UWCNN method. Although the color cast problem of the image is effectively restored, the details in the dark areas of the image are not enhanced. Figure 3-4 The image shows the result after processing with the Ucolor method. It can be seen that details in dark areas have been enhanced to some extent, but the image saturation is low. Figure 3 -5 The result image after processing with the PUIE method shows that the brightness of the image is excessively enhanced, resulting in overexposure. Figure 3 -6 shows the result image after processing by the algorithm in this paper. It can be seen that the algorithm in this paper effectively enhances the details in the dark areas of the image while avoiding overall overexposure.

[0139] Then the third image is processed using various algorithms; Figure 4-1 This is the original underwater image. Figure 4-2 The image shows the result after processing with the WaterNet method. It can be seen that incorrect color information appears in the beach area. Figure 4-3 The image shows the result after processing with the UWCNN method. Compared to the original image, the information of the coral has been enhanced, but the enhancement is limited. Figure 4-4 The image shows the result after processing with the Ucolor method. The turtle's back is enhanced, but the color of the beach is not restored. Figure 2 -5 The result image after processing with the PUIE method shows that the color saturation of the turtle's back is limited compared to the algorithm in this paper. Figure 4 -6 shows the result image after processing by the algorithm in this paper. It can be seen that the algorithm can effectively enhance the detail information of coral and the color saturation of the turtle's back.

[0140] The sequence numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the embodiments of the present invention, the descriptions of each embodiment have different focuses; parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. It should be understood that the disclosed technical content in the several embodiments provided in this application can be implemented in other ways.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0142] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

Claims

1. A collaborative multi-scale underwater image enhancement method based on internal supervision, characterized in that, Includes the following steps: S01: Acquire the input underwater image, and perform degradation operations on the input underwater image I to acquire the first-level degradation image I2, the second-level degradation image I3, and the third-level degradation image I4 respectively; S02: Input the underwater image I into the initial feature extraction module to extract the initial feature E1. At the same time, input the first-level degraded image I2, the second-level degraded image I3, and the third-level degraded image I4 into the feature extraction module to extract the corresponding first-level degraded image feature G21, second-level degraded image feature G31, and third-level degraded image feature G41, respectively. S03: The obtained initial feature E1 is downsampled and then input into the dual-branch context attention module to obtain the first-level degradation feature E2; S04: Input the first-level degraded image feature G21 into the adaptive selective intrinsic supervision module for feature selection, obtain the image feature reconstruction feature G22, and then add it together with the first-level degraded feature E2 to obtain the first-level comprehensive feature Z2; S05: Perform S03 and S04 respectively on the secondary degraded image I3 and the tertiary degraded image I4 to obtain the corresponding secondary comprehensive features Z3 and tertiary comprehensive features Z4 for feature reconstruction; S06: The third-level degraded image features G41 are enhanced by a dual-branch context attention module and selected by an adaptive selective intrinsic supervision module to obtain the third-level degraded image restoration features R4; S07: Add the three-level comprehensive feature Z4 to the three-level degraded image restoration feature R4 and input it into the dual-branch context attention module to perform feature restoration and obtain the three-level restoration feature D4; S08: The secondary degraded image features G31 are enhanced by a dual-branch context attention module and selected by an adaptive selective intrinsic supervision feature module to obtain secondary degraded image restoration features R3 that are beneficial for feature restoration. S09: Upsample the third-level restoration feature D4, superimpose the second-level comprehensive feature Z3 and the second-level degraded image restoration feature R3, and input them into the dual-branch context attention module for feature restoration to obtain the first-level degraded image restoration feature R2; S10: Repeat S08 and S09 using the first-level degraded image feature G21, the second-level degraded image restoration feature R3 and the first-level comprehensive feature Z2 to obtain the image restoration feature R1; S11: Input the image restoration feature R1 into the dual-branch context attention module and convolutional layer to restore the image and obtain the enhanced image J; S12: Repeat the S11 operation using the first-level degraded image restoration feature R2, the second-level degraded image restoration feature R3, and the third-level degraded image restoration feature R4 to obtain the corresponding enhanced images J2, J3, and J4; S13: Use a multi-stage degradation loss function to constrain the four-stage enhancement process.

2. The collaborative multi-scale underwater image enhancement method based on internal supervision according to claim 1, characterized in that, The degradation operation in S01 includes: a downsampling operation; where I∈R H×W×3 Let R represent the size of the image, H represent the height and width of the image, and W represent the height and width of the image, respectively. I² ∈ R H / 2×W / 2×3 , I3∈R H / 4×W / 4×3 , I4∈R H / 8×W / 8×3 .

3. The collaborative multi-scale underwater image enhancement method based on internal supervision according to claim 1, characterized in that, The feature extraction module in S02 includes a 3*3 convolutional layer with 3 input channels and 32 output channels, and a dual-branch context attention module; the dual-branch context attention module is as follows: S021: Branch 1 of the dual-branch context attention module, which integrates the feature attention module and the resolution-guided intrinsic attention module. The formula for the integrated feature attention module is as follows: f z (x)=Conv(concat(CA(x)×x,PA(x)×x)); (1) Where x represents the input feature, f z The expression represents a comprehensive feature attention operation. Conv indicates a convolutional layer with the same number of channels in the input and output feature maps. CA indicates a channel attention mechanism, PA indicates a spatial attention mechanism, and concat indicates concatenating the feature maps along the channel dimension. S022: Based on the comprehensive feature attention module, the resolution-guided intrinsic attention module is formulated as follows: f u (x)=Sigmoid(Up(Conv(Down(Conv(x)))))×x; (2) Where x represents the input feature, f u This represents the resolution-guided intrinsic attention operation, Conv represents a convolutional layer with the same number of channels in the input and output feature maps, Down represents the downsampling operation, and Up represents the upsampling operation, where Sigmoid(x) = 1 / (1+e^(-1 / 2)). -x ); S023: Based on the comprehensive feature attention module and the resolution-guided intrinsic attention module, the formula for branch one of the dual-branch context attention module is as follows: f1(x)=f u (f z (x)); (3) Where x represents the input feature and f1 represents the branch-one operation; S024: In the second branch of the dual-branch context attention module, the context-aware feature extraction module has the following formula: f2(x)=Conv(Conv(x)+DConv(x)); (4) Where x represents the input feature, f2 represents the context-aware feature extraction operation, and DConv represents the dilated convolution operation; S025: Based on the branch one formula and the branch two formula, the formula for the dual-branch context attention module is as follows: f b (x)=Conv(Gelu(concat(f1(x),f2(x)))); (5) Where x represents the input feature, f b This represents a context attention operation based on a two-branch architecture, where DConv represents a dilated convolution operation, and Gelu represents the Gelu activation function, as shown in the following formula: The feature extraction module in S02 obtains initial feature E1 by performing feature extraction on the input underwater image I, as shown in the following formula: E1=f b (Conv(I)); (7) G21=f b (Conv(I2)); (8) G31=f b (Conv(I3)); (9) G41=f b (Conv(I4)); (10) Where I represents the input underwater image, I2, I3, and I4 represent the first, second, and third level degradation images, respectively, and f b This represents a context attention operation based on two branches, where E1 represents the initial feature, and G21, G31, and G41 represent the first-level, second-level, and third-level degraded image features, respectively.

4. The collaborative multi-scale underwater image enhancement method based on internal supervision according to claim 1, characterized in that, The initial feature E1 in S03 is downsampled and then input into the dual-branch context attention module, as shown in the following formula: E2=f b (E1); (11) Here, E2 represents the first-level degradation characteristic.

5. The collaborative multi-scale underwater image enhancement method based on internal supervision according to claim 1, characterized in that, The formula for the adaptive selective intrinsic supervision module in S04 is as follows: Where x represents the input feature, r represents the reference feature, and f s This represents an adaptive selective intrinsic supervision operation; The first-level degraded image feature G21 is input into the adaptive selective intrinsic supervised feature module for feature selection, as shown in the following formula: G22=f s (G21,E2); (13) Then, it is superimposed with the first-level degenerate feature E2 to obtain the first-level comprehensive feature Z2, as shown in the following formula: Z2 = G22 + E2. (14) 6. The collaborative multi-scale underwater image enhancement method based on internal supervision according to claim 1, characterized in that, In step S07, the third-level comprehensive feature Z4, superimposed with the third-level degraded image restoration feature R4, is input into the dual-branch context attention module for feature restoration to obtain the third-level restoration feature D4, as shown in the following formula: D4=f b (Z4+R4)。 (15) 7. The collaborative multi-scale underwater image enhancement method based on internal supervision according to claim 1, characterized in that, In step S08, the secondary degraded image feature G31 is enhanced through a dual-branch contextual attention module and selected through an adaptive selective intrinsic supervision feature module to obtain the secondary degraded image restoration feature R3, which is beneficial for feature restoration. The formula is as follows: R3=f s (f b (G31))。 (16) 8. The collaborative multi-scale underwater image enhancement method based on internal supervision according to claim 1, characterized in that, In step S09, D4 is upsampled, and the secondary comprehensive feature Z3 and the secondary degraded image restoration feature R3 are superimposed and input into the dual-branch context attention module for feature restoration to obtain the primary degraded image restoration feature R2, as shown in the following formula: R2=f b (Up(D4)+Z3+R3)。 (17) 9. The collaborative multi-scale underwater image enhancement method based on internal supervision according to claim 1, characterized in that, In step S11, the image restoration feature R1 is input into a dual-branch context attention module and a convolutional layer to restore the image and obtain the enhanced image J, as shown in the following formula: J=Conv(f b (R1))。 (18) 10. The method for collaborative multi-scale underwater image enhancement based on internal supervision according to claim 1, characterized in that, In step S12, the corresponding enhanced images J2, J3, and J4 are obtained by repeating the operation of step S11 using the first-level degraded image restoration feature R2, the second-level degraded image restoration feature R3, and the third-level degraded image restoration feature R4, as shown in the following formula: J2=Conv(f b (R2)); (19) J3=Conv(f b (R3)); (20) J4=Conv(f b (R4))。 (21)。

Images