Underwater image enhancement method and device based on color perception fusion attention and background CD-ROM separation contrast learning

Abstract

The invention discloses an underwater image enhancement method and device based on color perception fusion attention and background CD-ROM separation contrast learning. The method comprises the following steps: extracting color information in an underwater original image based on a designed color extraction module; inputting the color information into a local color recovery branch to obtain reasonably distributed color weight information; performing triple attention branching on the underwater original image to recover details and fuzzy information in the image, and obtaining an underwater enhanced image with rich detail texture information; and inputting the reasonably distributed underwater enhanced image with rich color weight information and detail texture information into a global light recovery branch, adjusting the global light characteristics of the underwater image by adopting a contrast learning strategy, and finally obtaining a recovered underwater image enhancement result. According to the method, the enhancement performance in the underwater scene can be effectively improved, the degradation influence of the underwater complex environment on the image can be eliminated, and the color and texture details of the image can be truly restored.

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification, and in particular relates to an underwater image enhancement method and device based on color perception fusion attention and background light drive-off contrast learning. Background Art

[0002] Underwater imaging is essential for applications such as underwater robotics and marine resource exploration. However, the quality of underwater images is significantly degraded due to the absorption and scattering of light, resulting in problems such as color cast, low contrast, haze, and blur. These degradations adversely affect the performance and reliability of underwater vision tasks, highlighting the importance of underwater image enhancement. Underwater image enhancement plays a vital role in improving the visual quality and fidelity of images, which helps the present invention to understand the underwater environment more accurately. The degradation of underwater images can be attributed to two main factors. First, the wavelength of light changes with the propagation distance, which is different from the uniform attenuation of ground light. As the propagation distance increases, longer wavelengths such as red and orange attenuate more violently, resulting in a blue-green cast associated with shorter wavelengths. This effect becomes more pronounced as the distance between the subject and the camera increases. Second, suspended particles in the water, including organic particles and planktonic microorganisms, cause light scattering, deflect the propagation direction of light, and reduce image contrast and clarity. In order to improve the quality of underwater images, many methods have emerged, which can be roughly divided into two categories: traditional methods and deep learning-based methods. Traditional underwater image enhancement methods include prior-based and physical model-based methods. Prior-based methods use rich priors to explore the spatial relationship between pixel values ​​in the original underwater image and enhance the original underwater image by adjusting contrast, brightness and saturation, such as gray world assumption, RGB maximization and white balance. However, these methods often ignore the physical imaging process, which limits their enhancement effect. Physical model-based underwater image enhancement methods focus on accurately estimating the underwater image formation model or medium transmission parameters. These methods aim to obtain a clean image by inverting the physical underwater imaging model. Despite the potential of such methods, the performance of traditional underwater image enhancement methods is often limited by the complexity and diversity of conditions in real underwater environments. Recently, deep learning has driven significant progress in underwater image enhancement technology, with the main methods built on convolutional neural networks (CNNs), generative adversarial networks (GANs) and transformers. In this work, the existing techniques focus on CNN architectures due to their powerful visual feature learning capabilities and flexible training processes. However, CNN-based underwater image enhancement methods encounter challenges in real underwater scenes with complex distortions and diverse image content, which hinders accurate and adaptive restoration. As described by the underwater imaging model, the global color distortion caused by global illumination and the local complex distortion within the scene brightness pose a significant obstacle to underwater image enhancement and significantly reduce image clarity.

[0003] In view of the above problems, it can be seen that how to deal with serious degradation problems such as blurred details and color cast in the process of underwater image enhancement is an urgent problem to be solved. Summary of the invention

[0004] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and to provide an underwater image enhancement method and device based on color perception fusion attention and background light drive-off contrast learning, so as to improve the quality of underwater images, and at the same time be able to reduce the impact of common underwater degradation such as color deviation and detail blur on the image, and truly and delicately restore the color and detail texture of the image, thereby improving the authenticity and visibility of the enhanced image.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides an underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning, comprising the following steps:

[0007] The color extraction module based on color histogram and linear module design extracts color information from the underwater original image;

[0008] Inputting the color information into a local color restoration branch to obtain color weight information with reasonable distribution;

[0009] The underwater original image is passed through a triple attention branch to restore details and blur information in the image, and an underwater enhanced image with rich detail texture information is obtained; the triple attention branch integrates multiple triple attention content perception modules with jump connections in a U-shaped manner to form a hierarchical structure, and enhances key image areas and context dependencies through a series of downsampling and upsampling operations. The triple attention branch uses multi-scale input to capture features of different resolutions, and the output of the triple attention branch is obtained by combining spatial features, channel features and self-attention features; the downsampling is used to obtain high-level semantic features, and the upsampling is used to restore fine-grained details; the triple attention content perception module aims to enhance the features of underwater images by integrating three attention mechanisms: spatial attention, channel attention and self-attention;

[0010] The underwater enhanced image with the reasonably distributed color weight information and rich detail texture information is input into the global light restoration branch, and a contrast learning strategy is used to adjust the global light characteristics of the underwater image, and finally a restored underwater image enhancement result is obtained.

[0011] As a preferred technical solution, the calculation process of the color extraction module extracting the color information in the underwater original image is expressed as:

[0012] hc (i)=∑ x,y δ(I c (x,y)-i),

[0013] where h c (i) is the histogram of color channel c in image i, I c (x, y) is the pixel value of channel c at (x, y), and δ is the indicator function; a histogram is calculated for each color channel (R, G, B) to obtain the color distribution; the histogram is then passed to the linear module, and then through the convolution layer to obtain the color-related feature C; wherein the linear module is used for dimensional alignment.

[0014] As a preferred technical solution, the local color restoration branch is constructed based on the self-attention mechanism of color information, and the self-attention mechanism realizes adaptive adjustment in the following ways:

[0015]

[0016] Among them, C is the color-related feature, K and V are obtained by transforming the input image through the convolution layer, and d k is the dimension of the key vector; a series of deep learning layers are then applied to the output of the attention operation to generate the adaptive color correction map M LCR =f LCR (I); The color correction map is intended to provide accurate, color-guided weights to enhance the model's underwater image enhancement capabilities.

[0017] As a preferred technical solution, the triple attention branch integrates the triple attention content-aware module with jump connections in a U-shaped manner to form a hierarchical structure, which enhances key image areas and contextual dependencies through a series of downsampling and upsampling operations. It ensures global and local feature refinement of the image by gradually applying the triple attention content-aware module and downsampling the input to obtain high-level semantic features, and then upsampling to restore fine-grained details. In addition, multi-scale inputs are used to capture features of different resolutions, which is crucial for processing objects and details of different scales in underwater images. The triple attention content-aware module aims to enhance the features of underwater images by integrating three attention mechanisms: spatial attention, channel attention, and self-attention. The spatial attention in the triple attention content-aware module is used to identify and enhance important spatial regions. By focusing on important spatial regions, the triple attention branch highlights the important features of marine life and underwater structures;

[0018] In the spatial attention, given the input feature map F = R H×W×C , the process is as follows:

[0019]

[0020] Where σ represents the sigmoid function, f 7×7 represents a convolution operation with a filter size of 7×7. AvgPool and MaxPool are two cross-channel pooling operations, and then the spatial mapping M s ∈R 1×H×W The spatial feature map F is obtained by element-wise multiplication with each channel map of F s .

[0021] As a preferred technical solution, the channel attention in the triple attention content perception module focuses on the most informative channel, balances the color channels and enhances key features to correct color distortion caused by water absorption and scattering;

[0022] In the channel attention, given the input feature map F, the channel attention vector M c ∈R C Calculated as:

[0023]

[0024] Where GAP represents global average pooling, is a convolution operation of a 1×1 filter, σ and δ represent sigmoid and ReLU activation functions respectively; GAP is global average pooling, which puts channel-based global spatial information into the channel:

[0025]

[0026] where F c (i,j) represents the cth channel X at position (i,j) c The value of the channel participating in the feature map F c is the element-wise multiplication of F and M c Obtained by replication in the spatial dimension.

[0027] As a preferred technical solution, the self-attention module in the triple attention content perception module is used to capture long-distance dependency features and understand the global context;

[0028] In the self-attention, given F, the self-attention map M self ∈R HW×HW , calculated as:

[0029]

[0030] Among them, Q s =f Q (F), K s =f K (F) are query mapping and key mapping respectively, fQ and f K is a linear transformation, T represents transpose; by applying the self-attention mask M self Mapping value V s =f V (F), get the self-attention feature map F self , which is then further refined by additional deep learning layers.

[0031] As a preferred technical solution, the underwater enhanced image with the reasonably distributed color weight information and rich detail texture information is input into the global light restoration branch, and the global light characteristics of the underwater image are adjusted by using a contrast learning strategy, and finally the restored underwater image enhancement result is obtained, specifically:

[0032] The global light restoration branch simulates image pairs under different lighting conditions, and the network learns robust features to approximate the real solution space of the background light and applies it to global correction; the background light estimation is obtained based on the underwater image formation model, which describes the absorption and scattering process of light propagating in water, thereby affecting the color and clarity of the image. The underwater image formation model is expressed as:

[0033] I(x)=J(x)t(x)+A(1-t(x)),

[0034] Where x is the pixel index, I(x) is the intrinsic brightness of the scene at pixel x, representing the true color and intensity of the scene in clear water or air; A is the global ambient light map with all A, representing the background light in the water; t(x) is the medium transmittance at pixel x, obtained by transformation:

[0035] A=(I(x)-J(x)t(x)) / (1-t(x)),

[0036] For the i-th distorted underwater image τ i and its truth value τ i ′, global background light A′ i Use the above formula to calculate; in order to generate samples that are significantly different from the true value in terms of background light, first calculate the available data τ i The background light is stored in a library In the example, after filtering with the global light filter, the negative sample set is effectively constructed. In the global light filter, each and A′ i The Euclidean distance d between n , then remove l n Less than the threshold d n The calculation process is as follows:

[0037]

[0038] In the formula, r n , g n , b n Indicates having the same b m The three RGB dimensions of the value, in In the equation, r, g, and b represent A′ i The three rgb dimensions of the same a' in;

[0039] The specific synthesis of negative samples is: given the i-th underwater original image τ i , calculate the scene brightness map I according to the above formula for calculating global ambient light i and medium transmission diagram t i , then by τ i The resulting M negative samples are calculated as:

[0040]

[0041] In contrastive learning strategy, using ground-truth and abnormal light images Z are used as positive and negative samples, respectively, to restore the global light of underwater images; the goal of contrastive learning is to learn a representation that clusters the “positive” pairs in the latent feature space together and separates the representations between the “negative” pairs.

[0042] In a second aspect, the present invention provides an underwater image enhancement system based on color perception fusion attention and background light drive-off contrast learning, which is applied to the underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning, including a feature information extraction module, a color restoration module, a detail information restoration module and a global light restoration module;

[0043] The feature information extraction module is used to extract color information from the underwater original image based on the color histogram and the color extraction module designed by the linear module;

[0044] The color restoration module is used to input the color information into the local color restoration branch to obtain color weight information with reasonable distribution;

[0045] The detail information recovery module is used to restore the details and blur information in the image through the triple attention branch of the underwater original image, so as to obtain an underwater enhanced image with rich detail texture information; the triple attention branch integrates multiple triple attention content perception modules with jump connections in a U-shaped manner to form a hierarchical structure, and enhances key image areas and context dependencies through a series of downsampling and upsampling operations. The triple attention branch uses multi-scale input to capture features of different resolutions, and the output of the triple attention branch is obtained by combining spatial features, channel features and self-attention features; the downsampling is used to obtain high-level semantic features, and the upsampling is used to restore fine-grained details; the triple attention content perception module aims to enhance the features of underwater images by integrating three attention mechanisms: spatial attention, channel attention and self-attention;

[0046] The global light restoration module is used to input the underwater enhanced image with reasonably distributed color weight information and rich detail texture information into the global light restoration branch, adopt a contrast learning strategy to adjust the global light characteristics of the underwater image, and finally obtain a restored underwater image enhancement result.

[0047] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0048] at least one processor; and,

[0049] a memory communicatively connected to the at least one processor; wherein,

[0050] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to perform the underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning.

[0051] In a fourth aspect, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the underwater image enhancement method based on color perception fusion attention and background light drive-away contrast learning.

[0052] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0053] (1) The present invention provides a color-based attention branch, which consists of a color extraction module and a local color restoration branch. It uses a color histogram and a linear module to extract color information, and integrates a self-attention mechanism to obtain reasonably distributed color weight information, so that the network can adaptively correct non-uniform color distortion in degraded images and restore enhanced images with reasonable and satisfactory color information distribution.

[0054] (2) The triple attention branch provided by the present invention combines spatial attention, channel attention and self-attention mechanisms to enhance the feature analysis of key channels, key areas and global dependencies. This branch can effectively solve the problems of color deviation, blur and low contrast in underwater images, effectively solve the local and global distortion of image content, and enable the network to restore a clearer and more delicate enhanced image.

[0055] (32) The global light restoration branch provided by the present invention is a new method that combines global uniform background light information with contrast learning technology to reduce the solution space of clean images and promote global color adjustment of underwater images. By generating image pairs simulating different lighting conditions, the model of the present invention learns to identify and correct the deviation between the color of underwater degraded images and natural colors, so that the network can adapt to complex underwater environments and restore enhanced images that are not affected by global light, thus solving the problem of global color deviation caused by global background light. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 is a flow chart of an underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning according to an embodiment of the present invention;

[0058] Figure 2 is a schematic diagram of an underwater image enhancement network based on color perception fusion attention and background light drive-off contrast learning according to an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of a linear module structure in a color extraction module in an embodiment of the present invention;

[0060] Figure 4 is a schematic diagram of a three-attention module in an embodiment of the present invention;

[0061] Figure 5 This is a comparison chart of the enhancement effect of the present invention and several advanced underwater image enhancement algorithms on a reference data set;

[0062] Figure 6 This is a comparison chart of the enhancement effect of the present invention and several advanced underwater image enhancement algorithms on a no-reference dataset;

[0063] Figure 7It is a schematic diagram of the structure of an underwater image enhancement system based on color perception fusion attention and background light drive-off contrast learning according to an embodiment of the present invention;

[0064] Figure 8 4 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0066] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0067] See also Figure 1 , this embodiment proposes a nighttime defogging method based on nighttime physical perception and grayscale world, including the following steps:

[0068] S1. A color extraction module based on color histogram and linear module design extracts color information from the underwater original image.

[0069] The color extraction module is designed based on the color histogram and the linear module, wherein the calculation process of the color histogram extraction can be expressed as:

[0070] h c (i)=∑ x,y δ(I c (x,y)-i),

[0071] where h c (i) is the histogram of color channel c in image i, I c (x, y) is the pixel value of channel c at (x, y), and δ is the indicator function; the histogram of each color channel (R, G, B) is calculated to obtain the color distribution; the histogram is then passed to the linear module, and then through the convolution layer to obtain the color-related feature C; wherein the linear module is used for dimension alignment. The structure of the linear module is as follows Figure 3 shown.

[0072] Furthermore, the color information is input into a local color restoration branch to obtain color weight information with reasonable distribution.

[0073] The local color restoration branch is constructed based on the self-attention mechanism of color information, and the self-attention mechanism achieves adaptive adjustment in the following ways:

[0074]

[0075] Among them, C is the color-related feature, K and V are obtained by transforming the input image through the convolution layer, T represents transposition, and d k is the dimension of the key vector; a series of deep learning layers are then applied to the output of the attention operation to generate the adaptive color correction map M LCR =f LCR (I); The color correction map is intended to provide accurate, color-guided weights to enhance the model's underwater image enhancement capabilities.

[0076] S2. At the same time, the underwater original image is processed through triple attention branches to restore the details and blurred information in the image.

[0077] The proposed triple attention branch integrates the triple attention content-aware module with skip connections in a U-shaped manner to form a hierarchical structure that enhances key image regions and contextual dependencies through a series of downsampling and upsampling operations. It ensures global and local feature refinement of the image by gradually applying the triple attention content-aware module and downsampling the input to obtain high-level semantic features, and then upsampling to recover fine-grained details. In addition, multi-scale inputs are used to capture features of different resolutions, which is crucial for processing objects and details of different scales in underwater images. The triple attention content-aware module aims to enhance the features of underwater images by integrating three attention mechanisms: spatial attention, channel attention, and self-attention.

[0078] The spatial attention module in the triple attention content-aware module is used to identify and enhance important spatial regions, which is crucial for underwater image enhancement tasks due to the uneven lighting and visibility conditions in underwater images. By focusing on important spatial regions, the triple attention branch highlights important features such as marine life and underwater structures. Given an input feature map F = R H×W×C , the process is as follows:

[0079]

[0080] Where σ represents the sigmoid function, f 7×7 represents a convolution operation with a filter size of 7×7. AvgPool and MaxPool are two cross-channel pooling operations, and then the spatial mapping M s∈R 1×H×W The spatial feature map F is obtained by element-wise multiplication with each channel map of F s .

[0081] The channel attention module in the triple attention content-aware module is designed to focus on the most informative channel, balance the color channels and enhance key features, which is crucial for correcting color distortion caused by water absorption and scattering. Given the input feature map F, the channel attention vector M c ∈R C Calculated as:

[0082]

[0083] Where GAP represents global average pooling, is a convolution operation with a 1×1 filter, σ and δ represent sigmoid and ReLU activation functions respectively. GAP is global average pooling, which puts channel-based global spatial information into channels:

[0084]

[0085] where F c (i,j) represents the cth channel X at position (i,j) c The value of the channel participating in the feature map F c is the element-wise multiplication of F and M c Obtained by replication in the spatial dimension.

[0086] The self-attention module in the triple attention content perception module has the function of capturing long-distance dependency features and understanding the global context. Given f, the self-attention map M self ∈R HW×HW , calculated as:

[0087]

[0088] Among them, Q s =f Q (F), K s =f K (F) are query mapping and key mapping respectively, f Q and f K is a linear transformation, T represents transpose; by applying the self-attention mask M self Mapping value V s =f V (F), get the self-attention feature map F self , which is then further refined by additional deep learning layers.

[0089] The final output of the triple attention content-aware module is obtained by effectively combining spatial features, channel features, and self-attention features. By using the connection of two convolutional layers and one ReLU layer, the channel dimension can be reduced and the fused representation can be learned, thereby improving the overall feature quality.

[0090] S3. Then, the underwater enhanced image with reasonable color weight information and rich detail texture information is input into the global light restoration branch, and the global light characteristics of the underwater image are adjusted by using a contrast learning strategy, and finally the restored underwater image enhancement result is obtained, such as Figure 2 shown.

[0091] The global light restoration branch simulates image pairs under different lighting conditions, and the network learns robust features to approximate the real solution space of the background light and applies it to global correction; the background light estimation is obtained based on the underwater image formation model, which describes the absorption and scattering process of light propagating in water, thereby affecting the color and clarity of the image. The underwater image formation model is expressed as:

[0092] I(x)=J(x)t(x)+A(1-t(x)),

[0093] Where x is the pixel index, I(x) is the intrinsic brightness of the scene at pixel x, representing the true color and intensity of the scene in clear water or air; A is the global ambient light map with all A, representing the background light in the water; t(x) is the medium transmittance at pixel x, obtained by transformation:

[0094] A=(I(x)-J(x)t(x)) / (1-t(x)),

[0095] For the i-th distorted underwater image τ i and its truth value τ i ′, global background light A′ i Use the above formula to calculate; in order to generate samples that are significantly different from the true value in terms of background light, first calculate the available data τ i The background light is stored in a library In the example, after filtering with the global light filter, the negative sample set is effectively constructed. In the global light filter, each and A′ i The Euclidean distance d between m , then remove l n Less than the threshold d n The calculation process is as follows:

[0096]

[0097] In the formula, r n , g n , b n Indicates having the same b m The three RGB dimensions of the value, in In the equation, r, g, and b represent i The three rgb dimensions of the same a' in .

[0098] The specific synthesis of negative samples is: given the i-th underwater original image τ i , the scene brightness map I can be calculated based on the above formula for calculating global ambient light i and medium transmission diagram t i . Then τ i The resulting M negative samples can be calculated as:

[0099]

[0100] In contrastive learning strategy, using ground-truth and abnormal light images Z are used as positive and negative samples, respectively, to restore the global light of underwater images; the goal of contrastive learning is to learn a representation that clusters the “positive” pairs in the latent feature space together and separates the representations between the “negative” pairs.

[0101] In one embodiment, training is performed on RTX3090 with batchsize=8, during which the Adam optimizer is used and the cosine annealing strategy is used for training. The present invention crops each image into a size of 256×256 and uses the obtained image as the input network for training.

[0102] This embodiment can better cope with various degradation factors of underwater scenes, such as uneven illumination, color degradation, low brightness, severe detail loss, color cast, etc., and obtain underwater enhanced images with rich detail textures and correct color cast processing. Figure 5 As shown in parts (a) and (b) of the paper, UNTV, USUIR, PUGAN, GDCP, HLRP, UWnet and MLLE do not fully solve the problem. Figure 5 (4th image) results in a residual color cast. In contrast, this embodiment produces more natural, lifelike colors. Figure 5 (Figure 6) once again confirms the advantages of the present invention in similar scenarios. In addition, blue distortion is another common situation, such as Figure 5(Figure 2). Only PUIE-Net, PUGAN and the present embodiment can effectively alleviate the blue color cast, while GDCP, UNTV, UWnet, USUIR, HLRP and MLLE have poor effects. Although PUIE-Net and PUGAN corrected the blue tint, the present embodiment achieved better color restoration in key areas. In summary, these examples verify the advantages of the present embodiment in adaptively correcting various color distortions (such as green and bluish casts). Compared with the current technology, the present embodiment restores more natural and vivid colors. In underwater images, forward scattering usually causes low contrast and insufficient lighting, and enhancement methods are needed to increase color contrast while eliminating blur. As Figure 6 As shown in parts (a), (b), and (c) of Figure 1, conventional techniques have difficulty processing low-contrast images and sometimes introduce distortions such as HLRP. Deep learning methods may not be able to sufficiently improve brightness without introducing color distortions, such as UWnet and USUIR. This embodiment effectively processes low-contrast and low-light images without introducing distortions. Figure 6 As shown in the figure, this embodiment performs well in detail restoration. The results verify that this embodiment has a strong advantage in adaptively increasing brightness and contrast to restore clarity, while avoiding the human factors such as over-enhancement, color deviation and detail loss that exist in current technologies. This embodiment can reliably enhance low-contrast and low-light underwater images. At the same time, Figure 6 (Figure 5) This embodiment shows superior performance in a complex coral-rich scene. Although the presence of many corals of different colors and depths poses a challenging restoration task, this embodiment outperforms the other methods. HLRP, MLLE, UNTV, and PUIE-Net restore coral colors inaccurately. UWnet and USUIR have difficulty restoring red coral tones, while GDCP appears over-enhanced, resulting in obvious color deviation. Although PUGAN performs fairly well, it lacks clarity in details. In contrast, the results obtained by BCTA-Net of this embodiment are closest to the ground truth, accurately adjusting colors and preserving the clarity of corals and background elements.

[0103] Table 1

[0104]

[0105] In order to more specifically reflect the technical effect of this embodiment, as shown in the objective evaluation results of the method in Table 1, the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) are used as evaluation indicators, where red and blue represent the first and second ranked indicators respectively, and the higher the indicator value, the closer the generated image is to the standard clear image. In the table, this embodiment is trained on the UIEB training set and tested on the UIEB test set, and trained on the EUVP training set and tested on the EUVP test set. The values ​​of its peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are significantly higher than those of other image processing algorithms. As shown in the objective evaluation results of the method in Table 2, single image indicators UIQM and ILNIQE are used as evaluation indicators, where red and blue represent the first and second ranked indicators respectively, and the higher the UIQM or the lower the ILNIQE, the better the image quality. In the table, this embodiment is trained on the UIEB training set and tested on the UIEB test set, EUVP test set, MABLS dataset, U45 dataset, and SUIM dataset respectively. The present embodiment has the best UIQM index on EUVP, MABLS and SUIM datasets, and the second best UIQM index on UIEB and U45 datasets, which proves the advantages of the present embodiment in enhancing key attributes such as color, clarity and contrast. In addition, in terms of the ILNIQE index, the present embodiment ranks first among the learning-based methods on UIEB, EUVP, MABLS and U45. The ILNIQE index is used to measure the perceptual quality of natural images and ranks second on SUIM. These comprehensive results verify that the method of the present invention reliably outperforms existing technologies and has consistently excellent performance on different datasets, confirming the effectiveness of the present embodiment in improving the overall quality of underwater images, as well as its outstanding performance and robustness in processing images.

[0106] Table 2

[0107]

[0108] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0109] Based on the same idea as the underwater image enhancement method based on color perception fusion attention and background light drive away contrast learning in the above-mentioned embodiment, the present invention also provides an underwater image enhancement system based on color perception fusion attention and background light drive away contrast learning, which can be used to execute the above-mentioned underwater image enhancement method based on color perception fusion attention and background light drive away contrast learning. For ease of explanation, the structural schematic diagram of the embodiment of the underwater image enhancement system based on color perception fusion attention and background light drive away contrast learning only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0110] See also Figure 7 In another embodiment of the present application, an underwater image enhancement system 100 based on color perception fusion attention and background light drive-off contrast learning is provided, the system comprising a feature information extraction module 101, a color restoration module 102, a detail information restoration module 103 and a global light restoration module 104;

[0111] The feature information extraction module 101 is used to extract color information from the underwater original image based on the color histogram and the color extraction module designed by the linear module;

[0112] The color restoration module 102 is used to input the color information into the local color restoration branch to obtain color weight information with reasonable distribution;

[0113] The detail information recovery module 103 is used to restore the details and blur information in the image through the triple attention branch of the underwater original image, so as to obtain an underwater enhanced image with rich detail texture information; the triple attention branch integrates multiple triple attention content perception modules with jump connections in a U-shaped manner to form a hierarchical structure, and enhances key image areas and context dependencies through a series of downsampling and upsampling operations. The triple attention branch uses multi-scale input to capture features of different resolutions, and the output of the triple attention branch is obtained by combining spatial features, channel features and self-attention features; the downsampling is used to obtain high-level semantic features, and the upsampling is used to restore fine-grained details; the triple attention content perception module aims to enhance the features of underwater images by integrating three attention mechanisms: spatial attention, channel attention and self-attention;

[0114] The global light restoration module 104 is used to input the underwater enhanced image with reasonably distributed color weight information and rich detail texture information into the global light restoration branch, and adopt a contrast learning strategy to adjust the global light characteristics of the underwater image, and finally obtain a restored underwater image enhancement result.

[0115] It should be noted that the underwater image enhancement system based on color perception fusion attention and background light drive away contrast learning of the present invention corresponds one to one with the underwater image enhancement method based on color perception fusion attention and background light drive away contrast learning of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the underwater image enhancement method based on color perception fusion attention and background light drive away contrast learning are applicable to the embodiment of underwater image enhancement based on color perception fusion attention and background light drive away contrast learning. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here.

[0116] In addition, in the implementation of the underwater image enhancement system based on color perception fusion attention and background light drive away contrast learning in the above embodiment, the logical division of each program module is only an example. In actual applications, the above functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the underwater image enhancement system based on color perception fusion attention and background light drive away contrast learning is divided into different program modules to complete all or part of the functions described above.

[0117] See also Figure 8 In one embodiment, an electronic device for implementing an underwater image enhancement method based on color perception fusion attention and background light drive-away contrast learning is provided. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as an underwater image enhancement program 203 based on color perception fusion attention and background light drive-away contrast learning.

[0118] The first memory 202 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory 202 may be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 may also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 200. Further, the first memory 202 may also include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can not only be used to store application software and various types of data installed in the electronic device 200, such as the code of the underwater image enhancement program 203 based on color perception fusion attention and background light drive away contrast learning, but also can be used to temporarily store data that has been output or is to be output.

[0119] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The first processor 201 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions and processes data of the electronic device 200 by running or executing programs or modules stored in the first memory 202, and calling data stored in the first memory 202.

[0120] Figure 8 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on the electronic device 200 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0121] The underwater image enhancement program 203 based on color perception fusion attention and background light drive-off contrast learning stored in the first memory 202 of the electronic device 200 is a combination of multiple instructions. When running in the first processor 201, it can achieve:

[0122] The color extraction module based on color histogram and linear module design extracts color information from the underwater original image;

[0123] Inputting the color information into a local color restoration branch to obtain color weight information with reasonable distribution;

[0124] The underwater original image is passed through a triple attention branch to restore the details and blur information in the image, and an underwater enhanced image with rich detail texture information is obtained; the triple attention branch integrates multiple triple attention content perception modules with jump connections in a U-shaped manner to form a hierarchical structure, and enhances key image areas and context dependencies through a series of downsampling and upsampling operations. The triple attention branch uses multi-scale input to capture features of different resolutions, and the output of the triple attention branch is obtained by combining spatial features, channel features and self-attention features; the downsampling is used to obtain high-level semantic features, and the upsampling is used to restore fine-grained details; the triple attention content perception module aims to enhance the features of underwater images by integrating three attention mechanisms: spatial attention, channel attention and self-attention;

[0125] The underwater enhanced image with the reasonably distributed color weight information and rich detail texture information is input into the global light restoration branch, and a contrast learning strategy is used to adjust the global light characteristics of the underwater image, and finally a restored underwater image enhancement result is obtained.

[0126] Furthermore, if the module / unit integrated in the electronic device 200 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0127] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0128] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning, characterized in that: The steps include: The color extraction module based on color histogram and linear module design extracts color information from the underwater original image; Inputting the color information into a local color restoration branch to obtain color weight information with reasonable distribution; The underwater original image is subjected to triple attention branches to restore details and blur information in the image, thereby obtaining an underwater enhanced image with rich detail texture information; The triple attention branch integrates multiple triple attention content-aware modules in a U-shaped manner with jump connections to form a hierarchical structure, and enhances key image regions and context dependencies through a series of downsampling and upsampling operations. The triple attention branch uses multi-scale input to capture features of different resolutions, and the output of the triple attention branch is obtained by combining spatial features, channel features, and self-attention features; the downsampling is used to obtain high-level semantic features, and the upsampling is used to restore fine-grained details; the triple attention content-aware module aims to enhance the features of underwater images by integrating three attention mechanisms: spatial attention, channel attention, and self-attention; The underwater enhanced image with the reasonably distributed color weight information and rich detail texture information is input into the global light restoration branch, and a contrast learning strategy is used to adjust the global light characteristics of the underwater image, and finally a restored underwater image enhancement result is obtained.

2. The underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning according to claim 1 is characterized in that: The calculation process of the color extraction module to extract the color information from the underwater original image is expressed as: h c (i)=∑ x，y δ(I c (x,y)-i), where h c (i) is the histogram of color channel c in image i, I c (x, y) is the pixel value of channel c at (x, y), and δ is the indicator function; a histogram is calculated for each color channel (R, G, B) to obtain the color distribution; the histogram is then passed to the linear module, and then through the convolution layer to obtain the color-related feature C; wherein the linear module is used for dimensional alignment.

3. The underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning according to claim 1 is characterized in that: The local color restoration branch is constructed based on the self-attention mechanism of color information, and the self-attention mechanism achieves adaptive adjustment in the following ways: Among them, C is the color-related feature, K and V are obtained by transforming the input image through the convolution layer, and d k is the dimension of the key vector; a series of deep learning layers are then applied to the output of the attention operation to generate the adaptive color correction map M LCR =f LCR (I); The color correction map is intended to provide accurate, color-guided weights to enhance the model's underwater image enhancement capabilities.

4. The underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning according to claim 1 is characterized in that: The triple attention branch integrates the triple attention content-aware module with skip connections in a U-shaped manner to form a hierarchical structure that enhances key image regions and contextual dependencies through a series of downsampling and upsampling operations. It ensures global and local feature refinement of the image by gradually applying the triple attention content-aware module and downsampling the input to obtain high-level semantic features, and then upsampling to recover fine-grained details. In addition, multi-scale inputs are used to capture features of different resolutions, which is crucial for processing objects and details of different scales in underwater images. The triple attention content-aware module aims to enhance the features of underwater images by integrating three attention mechanisms: spatial attention, channel attention, and self-attention. The spatial attention in the triple attention content-aware module is used to identify and enhance important spatial regions. By focusing on important spatial regions, the triple attention branch highlights the important features of marine life and underwater structures; In the spatial attention, given the input feature map F = R H×W×C , the process is as follows: Where σ represents the sigmoid function, f 7×7 represents a convolution operation with a filter size of 7×7. AvgPool and MaxPool are two cross-channel pooling operations, and then the spatial mapping M s ∈R 1×H×W The spatial feature map F is obtained by element-wise multiplication with each channel map of F s .

5. The underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning according to claim 1 is characterized in that: The channel attention in the triple attention content-aware module focuses on the most informative channels, balances color channels and enhances key features to correct color distortion caused by water absorption and scattering; In the channel attention, given the input feature map F, the channel attention vector M c ∈R C Calculated as: Where GAP represents global average pooling, is a convolution operation of a 1×1 filter, σ and δ represent sigmoid and ReLU activation functions respectively; GAP is global average pooling, which puts channel-based global spatial information into the channel: where F c (i, j) represents the cth channel X at position (i, j) c The value of the channel participating in the feature map F c is the element-wise multiplication of F and M c Obtained by replication in the spatial dimension.

6. The underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning according to claim 1 is characterized in that: The self-attention module in the triple attention content perception module is used to capture long-distance dependency features and understand the global context; In the self-attention, given F, the self-attention map M self ∈R HW×HW , calculated as: Among them, Q s =f Q (F), K s =f K (F) are query mapping and key mapping respectively, f Q and f K is a linear transformation, T represents transpose; by applying the self-attention mask M self Mapping V to values s =f V (F), get the self-attention feature map F self , which is then further refined by additional deep learning layers.

7. The underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning according to claim 1 is characterized in that: The underwater enhanced image with the reasonably distributed color weight information and rich detail texture information is input into the global light restoration branch, and the global light characteristics of the underwater image are adjusted by using a contrast learning strategy, and finally the restored underwater image enhancement result is obtained, specifically: The global light restoration branch simulates image pairs under different lighting conditions, and the network learns robust features to approximate the real solution space of the background light and applies it to global correction; the background light estimation is obtained based on the underwater image formation model, which describes the absorption and scattering process of light propagating in water, thereby affecting the color and clarity of the image. The underwater image formation model is expressed as: I(x)=J(x)t(x)+A(1-t(x)), Where x is the pixel index, , (x) is the intrinsic brightness of the scene at pixel x, representing the true color and intensity of the scene in clear water or air; A is the global ambient light map with all A, representing the background light in the water; t(x) is the medium transmittance at pixel x, obtained by transformation: A=(I(x)-J(x)t(x)) / (1-t(x)), For the i-th distorted underwater image τ i and its truth value τ′ i , global background light A′ i Use the above formula to calculate; in order to generate samples that are significantly different from the true value in terms of background light, first calculate the available data τ i The background light is stored in a library In the example, after filtering with the global light filter, the negative sample set is effectively constructed. In the global light filter, each and A′ i The Euclidean distance d between n , then remove l n Less than the threshold d n The calculation process is as follows: In the formula, r n , g n , b n Indicates having the same b m The three RGB dimensions of the value, in In the equation, r, g, and b represent A′ i The three rgb dimensions of the same a' in; The specific synthesis of negative samples is: given the i-th underwater original image τ i , calculate the scene brightness map I according to the above formula for calculating global ambient light i and medium transmission diagram t i , then by τ i The resulting M negative samples are calculated as: In contrastive learning strategy, using ground-truth and abnormal light images Z are used as positive and negative samples, respectively, to restore the global light of underwater images; the goal of contrastive learning is to learn a representation that clusters "positive" pairs in the latent feature space together and separates the representations between "negative" pairs.

8. An underwater image enhancement system based on color perception fusion attention and background light drive-off contrast learning, characterized in that: An underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning applied to any one of claims 1-7, comprising a feature information extraction module, a color restoration module, a detail information restoration module and a global light restoration module; The feature information extraction module is used to extract color information from the underwater original image based on the color histogram and the color extraction module designed by the linear module; The color restoration module is used to input the color information into the local color restoration branch to obtain color weight information with reasonable distribution; The detail information recovery module is used to restore the details and blur information in the image through the triple attention branch of the underwater original image, so as to obtain an underwater enhanced image with rich detail texture information; the triple attention branch integrates multiple triple attention content perception modules with jump connections in a U-shaped manner to form a hierarchical structure, and enhances key image areas and context dependencies through a series of downsampling and upsampling operations. The triple attention branch uses multi-scale input to capture features of different resolutions, and the output of the triple attention branch is obtained by combining spatial features, channel features and self-attention features; the downsampling is used to obtain high-level semantic features, and the upsampling is used to restore fine-grained details; the triple attention content perception module aims to enhance the features of underwater images by integrating three attention mechanisms: spatial attention, channel attention and self-attention; The global light restoration module is used to input the underwater enhanced image with reasonably distributed color weight information and rich detail texture information into the global light restoration branch, adopt a contrast learning strategy to adjust the global light characteristics of the underwater image, and finally obtain a restored underwater image enhancement result.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the underwater image enhancement method based on color perception fusion attention and background light drive-away contrast learning as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the underwater image enhancement method based on color perception fusion attention and background light drive-off contrast learning described in any one of claims 1 to 7 is implemented.

Images