Underwater image enhancement method driven by multi-prior aggregation learning

Abstract

The invention relates to an underwater image enhancement method driven by multi-prior aggregation learning, and the method comprises the steps: obtaining an underwater image, inputting the underwater image into a pre-trained multi-prior aggregation learning image enhancement model, and obtaining an enhanced underwater image; the multi-priori aggregation learning image enhancement model comprises an underwater multi-priori extraction module, a multi-priori perception enhancement module and a multi-priori guide modulation module; the process of enhancing the underwater image by the multi-priori aggregation learning image enhancement model comprises the following steps of: extracting underwater priori information from the underwater image through the underwater multi-priori extraction module; the underwater degradation in the underwater image is removed from the feature level through a multi-prior perception enhancement module based on the underwater prior information; through a multi-priori guidance modulation module, based on underwater priori information, dynamic modulation is carried out to generate reconstruction features; and generating and outputting an enhanced underwater image based on the reconstructed features. Compared with the prior art, the method can adapt to and enhance underwater images of various different water qualities.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of ocean engineering and computer vision, and particularly to an underwater image enhancement method driven by multi-prior aggregation learning. BACKGROUND

[0002] As an important medium for obtaining marine information, underwater images play a crucial role in marine research. However, due to the limitations of the complex imaging environment, lighting conditions, and underwater imaging equipment, the clarity of underwater images is generally low, and there are quality degradation problems such as color distortion, low contrast, low illumination, and blurred details. These problems seriously affect the display quality and measurement and recognition accuracy of underwater images, bringing difficulties to marine practical applications and scientific research. Therefore, it is of great theoretical significance and practical value to preprocess the underwater images for clarity to improve the image quality and make it meet the needs of human visual perception or machine recognition in specific tasks.

[0003] In recent years, the rapid development of deep learning technology has also provided new solutions for the field of underwater image processing. More and more network structures have been designed to enhance underwater images, which have improved the visual quality of underwater images to varying degrees. However, research has found that most existing deep underwater image enhancement models mainly rely on engineering experience (such as multi-scale, degradation decoupling, cross-iteration, Retinex decomposition, etc.) when designing, resulting in that the performance of these models is highly dependent on the scale and quality of the training data, and the robustness is poor in real underwater scenes. In fact, many domain-specific knowledge of underwater images implicitly reflects the degradation characteristics. Using these prior knowledge can guide the network to learn more discriminative features, thereby improving the image restoration quality and enhancing the generalization ability of the model.

[0004] Underwater images contain original underwater images and their corresponding three kinds of prior information: background light, water type, and scene depth. Among them, the background light information reflects the distortion related to the color of the image, which is helpful for color correction of underwater images. Chinese patent CN116309232B discloses an underwater image enhancement method combining physical prior and deep learning, which considers the background light of underwater images; the water type prior information can help the enhancement model adapt to different water type images; the scene depth implicitly reflects the degree of quality degradation of underwater images, which can further improve the response ability of the network to different degradation degree regions. However, existing deep models rarely consider the specific multi-domain knowledge of underwater images in underwater image processing, and rely on limited training data, which seriously limits the generalization performance of the model. SUMMARY

[0005] The underwater image enhancement method driven by multi-prior aggregation learning aims to overcome the defects of the prior art.

[0006] The object of the present application can be achieved by the following technical solutions:

[0007] The underwater image enhancement method driven by multi-prior aggregation learning comprises:

[0008] The underwater image is obtained and input into a pre-trained multi-prior aggregation learning image enhancement model to obtain an enhanced underwater image.

[0009] The process of enhancing the underwater image by the multi-prior aggregation learning image enhancement model comprises: performing multi-prior extraction on the underwater image by a multi-prior extraction module to extract underwater prior information, including background light, scene depth and water type information; removing underwater degradation in the underwater image from a feature level based on the underwater prior information by a multi-prior perception enhancement module to obtain a picture feature after degradation reduction; dynamically modulating to generate a reconstructed feature based on the underwater prior information and the picture feature after degradation reduction by a multi-prior guide modulation module; and generating and outputting an enhanced underwater image based on the reconstructed feature.

[0010] Further, the architecture of the multi-prior aggregation learning image enhancement model is based on a U-Net network.

[0011] Further, the reconstructed feature is processed by a final convolutional layer and an activation layer of the U-Net network to generate and output the enhanced underwater image.

[0012] Further, the process of performing multi-prior extraction on the underwater image to extract underwater prior information comprises:

[0013] The underwater image is processed by a physical model-based underwater image enhancement method to obtain background light.

[0014] The underwater image is processed by a DPT method to generate a scene depth map.

[0015] The underwater image is classified by a preset water classification network to obtain a corresponding water type.

[0016] Further, the multi-prior perception enhancement module comprises a stacked convolution-activation layer and a stacked convolution-Sigmoid layer; the multi-prior guidance modulation module comprises a stacked convolution-instance normalization-activation layer, a double compression unit and a double convolution layer, the double compression unit comprising a stacked global average pooling-full connection layer and a stacked global maximum pooling-full connection layer.

[0017] Further, the process of obtaining the picture feature after degradation mitigation by the multi-prior perception enhancement module comprises:

[0018] The underwater prior information is mapped to a feature-level prior parameter space by the stacked convolution-activation layer to obtain prior features matching the feature dimension;

[0019] Based on the prior features, adjustment parameters related to the attenuation coefficient, depth and background light are calculated by the stacked convolution-Sigmoid layer;

[0020] The initial features are extracted from the original underwater image, and the initial features are adjusted based on the adjustment parameters to obtain the picture feature after degradation mitigation.

[0021] Further, the calculation expression of the adjustment parameters is:

[0022] α prior =C CS (F D ·C CS (F β ))

[0023] γ prior =C CR (C CR (F B )+C CR (F B )·F D ·C CS (F β ))

[0024] Wherein, α prior is the adjustment parameter related to the attenuation coefficient and the depth, γ prior is the adjustment parameter related to the attenuation coefficient, the depth and the background light, C CS represents the stacked convolution-Sigmoid activation function operation, F D , F β and F B are prior features, and C CR represents the stacked convolution-activation layer operation.

[0025] The calculation expression for obtaining the picture feature after degradation mitigation is:

[0026] X out = α prior · X + γ prior

[0027] wherein, X represents initial features extracted from the original underwater image.

[0028] Further, the process of generating the reconstructed features by the multi-prior guided modulation module comprises:

[0029] obtaining the picture features after the degradation mitigation from the multi-prior perception enhancement module as initial reconstructed features;

[0030] mapping the underwater prior information to a feature space by a stacked convolution-instance normalization-activation layer to obtain reconstructed prior features matching the feature dimension;

[0031] inputting the reconstructed prior features and the initial reconstructed features into a double compression unit to extract global statistical information and salient region information of the underwater prior features;

[0032] generating attention weights of the global statistical information and the salient region information by a Sigmoid activation, and fusing the global statistical information of each underwater prior feature with its corresponding salient region information based on the attention weights to obtain prior information feature descriptors;

[0033] concatenating the initial reconstructed features and the prior information feature descriptors into hybrid features, and capturing mutual dependency of underwater prior information in a channel domain and underwater images by a double convolution layer, and generating importance weights of each underwater prior information by a Softmax operation;

[0034] multiplying the reconstructed prior features and the importance weights of each underwater prior information channel by channel, adaptively selecting and combining preset underwater domain knowledge from the reconstructed prior features to obtain fused multi-prior features;

[0035] extracting prior guidance features from the fused multi-prior features, and converting them into a pixel-level scale transformation matrix and a translation transformation matrix, adjusting the normalized initial reconstructed features based on the scale transformation matrix and the translation transformation matrix to obtain optimized reconstructed features.

[0036] Further, the calculation expression of the optimized reconstructed features is:

[0037]

[0038] wherein, Y out represents the optimized reconstructed features, represents the normalized initial reconstructed features, γ is the scale transformation matrix, and β is the translation transformation matrix.

[0039] Further, the loss function for training the multi-prior aggregation learning image enhancement model is:

[0040] L total = λ1 * L com + λ2 * L per

[0041] Wherein, L com is the content loss, L per is the perceptual loss, λ1 and λ2 represent weight parameters;

[0042] L con = |UMPAN(I n )-J n |1

[0043]

[0044] Wherein, Indicates the feature map of the pool-3 layer of the VGG-19 network pre-trained on the ImageNet dataset, UMPAN(·) indicates the underwater multi-prior aggregation model proposed in the method, I n Indicates the input original underwater image, J n Indicates the corresponding underwater label image.

[0045] Compared with the prior art, the beneficial effects of the present application include:

[0046] 1. The present application proposes a new underwater image enhancement algorithm based on multi-prior aggregation, which first integrates multiple underwater field-specific knowledge as adaptive auxiliary information into the deep model to improve the performance and robustness of the underwater image enhancement task, and can stably process underwater images of different water qualities and different degradation types; The multi-prior perceptual enhancement module removes underwater degradation from the feature level, the multi-prior guide modulation module can dynamically analyze the importance of different underwater prior information through dynamic modulation, generate more rich and accurate reconstruction features, avoid over-enhancement and artifacts, and finally generate and output enhanced underwater images through the reconstruction features, In real underwater scenes, the present application can cause significant enhancement effect on images, and the present application provides a new idea for the future development direction of underwater image enhancement technology.

[0047] 2. The multi-prior perception enhancement module in the application modifies the original degraded features by mapping the multi-prior to the feature space and calculating the adjustment parameters related to degradation, so that the output picture features after reducing degradation can retain more enhancement-friendly details, such as underwater biological contours and rock textures, effectively reducing the impact of underwater degradation at the feature level, thereby learning more content features conducive to enhancement and avoiding the problem of low feature recognition in traditional algorithms that only correct at the pixel level, providing high-quality content features for subsequent reconstruction.

[0048] 3. The multi-prior guided modulation module in the application is used to improve the image reconstruction quality, which dynamically analyzes the importance of different underwater prior information and adaptively selects and combines valuable domain knowledge to guide the meaningful interaction between domain knowledge and data features, thereby obtaining richer reconstruction feature representation, and also avoids gradient disappearance and over-enhancement through data regularization.

[0049] 4. The application systematically integrates three kinds of underwater prior information, and integrates domain knowledge as adaptive auxiliary information into the model, so that the model no longer relies only on data statistical rules, but can understand the essence of degradation based on physical principles such as water attenuation and background light reflection, and upgrade the generalization ability from specific data set adaptation to multi-scene generalization, providing a new research direction of data plus prior for underwater image enhancement technology. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 The structure block diagram of the multi-prior aggregation learning image enhancement model of the application;

[0051] Figure 2 The process block diagram of extracting underwater prior information of the application;

[0052] Figure 3 The structure block diagram of the multi-prior aggregation enhancement module of the application;

[0053] Figure 4 The structure block diagram of the multi-prior guided modulation module of the application;

[0054] Figure 5 The result graph comparison of different methods in the embodiment of the application on the EUVP data set;

[0055] Figure 6 The result graph comparison of different methods in the embodiment of the application on the SQUID data set;

[0056] Figure 7 The result graph comparison of different methods in the embodiment of the application on the UIEB data set;

[0057] Figure 8 The result graph comparison of different methods in the embodiment of the application on the UIEB-Challenging data set. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the protection scope of the present application.

[0059] Embodiment 1

[0060] The embodiment discloses an underwater image enhancement method driven by multi-prior aggregation learning. The underwater image enhancement is based on an underwater image enhancement algorithm driven by multi-prior aggregation learning (UMPAN algorithm). Specifically, it embodies a pre-trained multi-prior aggregation learning image enhancement model. The specific enhancement process is as follows:

[0061] An underwater image is obtained and input into the pre-trained multi-prior aggregation learning image enhancement model to obtain an enhanced underwater image.

[0062] In the embodiment, the structure block diagram of the multi-prior aggregation learning image enhancement model is as shown in Figure 1 which includes a multi-prior extraction module, a multi-prior perception enhancement module, and a multi-prior guided modulation module.

[0063] The overall architecture of the multi-prior aggregation learning image enhancement model is based on the U-Net network and includes two stages of feature extraction and image reconstruction. In the feature extraction stage, a multi-prior perception enhancement module (MPAE) is designed for the degradation process of the underwater image. The module reduces the influence of underwater degradation at the feature level through physical inversion constraints, aiming to learn more content features conducive to enhancement. In the image reconstruction stage, a multi-prior guided modulation module (MPGM) is proposed, which still makes full use of multiple underwater prior knowledge.

[0064] It is found through observation that the importance of each domain knowledge is not fixed for different degrees of underwater image degradation, but varies with the change of underwater scene. Some underwater images are mainly affected by color distortion, while some images are mainly affected by blur distortion. For the former, color prior is more important, while for the latter, scene depth prior is more critical. Therefore, the multi-prior guided modulation module MPGM deeply explores the importance of each prior, adaptively selects and aggregates existing and preset valuable domain knowledge, and generates two dynamic modulation maps to guide the generation of rich reconstruction features. In addition, the training process of the multi-prior aggregation learning image enhancement model is completely data-driven, which enables it to effectively tolerate errors caused by inaccurate prior information estimation.

[0065] The underwater image enhancement algorithm driven by multi-prior aggregation learning proposed by the present application is not intended to estimate more accurate priors, but focuses on dynamically analyzing multiple priors and adaptively combining them according to their physical principles and practicality, thereby significantly improving the robustness of the enhancement model in different underwater environments.

[0066] In this embodiment, the specific process of the multi-prior aggregation learning image enhancement model to enhance the underwater image includes: performing multi-prior extraction on the underwater image through the underwater multi-prior extraction module to extract underwater prior information, including background light I B , scene depth I D and water type information I W ; through the multi-prior perception enhancement module, based on the underwater prior information, remove the underwater degradation in the underwater image to obtain the picture features after reducing the degradation; through the multi-prior guided modulation module, based on the underwater prior information and the picture features after reducing the degradation, dynamically modulate to generate reconstruction features; based on the reconstruction features, process the reconstruction features through the final convolution layer and the activation layer of the U-Net network to generate and output the enhanced underwater image.

[0067] The process of multi-prior extraction on the underwater image to extract underwater prior information is as shown in Figure 2 , including:

[0068] The underwater image is processed by the underwater image enhancement (UIBLA) method based on the physical model to obtain the background light B;

[0069] In the current underwater image scene depth estimation network, the depth map generated by the DPT (Depth Prediction Transformer) method has the highest quality, so the DPT method is used to process the underwater image to generate the scene depth map D;

[0070] The underwater image is classified by a preset water classification network to obtain the corresponding water type W.

[0071] The preset water classification network comes from existing water classification work experience and the like.

[0072] The multi-prior perception enhancement module comprises a stacked convolution-activation layer and a stacked convolution-Sigmoid layer; the multi-prior guidance modulation module comprises a stacked convolution-instance normalization-activation layer, a dual-compactor block (DCB) and a double convolution layer, the dual-compactor block comprising a stacked global average pooling-full connection layer and a stacked global maximum pooling-full connection layer.

[0073] Compared with land images, there are significant differences in the imaging and degradation processes of underwater images. The underwater imaging process reduces the clarity of underwater images, resulting in a decrease in feature recognition, thereby increasing the difficulty of recognition. In view of this feature, the present application uses underwater prior knowledge to alleviate the degradation at the feature level.

[0074] Based on the existing simplified underwater imaging model, the expression of the enhanced image J λ (x) is as follows:

[0075] J λ (x)=e β(λ)·d(x) ·I λ (x)+B(λ)-B(λ)·e β(λ)·d(x)

[0076] =α prior ·I λ (x)+γ proor

[0077] It can be seen that the adjustment parameter α prior is associated with the attenuation coefficient β(λ) and the depth map d(x), and the adjustment parameter γ prior is related to β(λ), d(x) and the background light B(λ). Since β(λ) is closely related to the water type, the present application selects the water type prior information to replace the attenuation coefficient.

[0078] For the above formula, a multi-prior aggregation enhancement module MPAE is designed, and the detailed structure is as shown in Figure 3 , which aims to perform feature enhancement at the feature level according to the above formula to alleviate the influence of underwater degradation.

[0079] The process of obtaining the picture features after alleviating degradation through the multi-prior perception enhancement module comprises:

[0080] Through the stacked convolution-activation layer, the underwater prior information is mapped to the feature-level prior parameter space to obtain prior features matching the feature dimension, and the expression is as follows:

[0081] F B ,F D ,F β =C CR(I B ),C CR (I D ),C CR (I W )

[0082] wherein C CR represents a stacked convolution-activation layer (ReLU) operation;

[0083] Based on the prior features, the adjustment parameters a prior and g prior related to the attenuation coefficient, the depth and the background light are calculated through the stacked convolution-Sigmoid layer.

[0084] The initial features are extracted from the original underwater image, and the initial features are adjusted based on the adjustment parameters to obtain the picture features after degradation mitigation.

[0085] The calculation expression of the adjustment parameters is:

[0086] a prior =C CS (F D ·C CS (F β ))

[0087] g prior =C CR (C CR (F B )+C CR (F B )·F D ·C CS (F β ))

[0088] wherein a prior is the adjustment parameter related to the attenuation coefficient and the depth, g prior is the adjustment parameter related to the attenuation coefficient, the depth and the background light, C CS represents a stacked convolution-Sigmoid activation function operation, F D , F β and F B are prior features, and C CR represents a stacked convolution-activation layer operation.

[0089] The calculation expression of the picture features after degradation mitigation is:

[0090] X out =a prior ·X+g prior

[0091] wherein X represents the initial features extracted from the original underwater image.

[0092] The multi-prior aggregation enhancement module MPAE can effectively remove underwater degradation and learn more content features conducive to enhancement by introducing multi-prior knowledge and applying underwater physical model inversion constraints at the feature level.

[0093] In the image reconstruction process, the multi-prior guided modulation module still introduces multiple underwater prior knowledge to enhance the features of the reconstructed image. However, different underwater prior knowledge exhibits significant differences and advantages for image reconstruction of different degradation levels. Therefore, the present application designs a multi-prior guided modulation module MPGM, which aims to adaptively integrate and utilize different underwater prior information to improve the reconstruction effect. The specific structure of the multi-prior guided modulation module is as shown in Figure 4

[0094] The process of generating reconstructed features through the multi-prior guided modulation module includes:

[0095] Obtain the picture features after alleviating degradation from the multi-prior perception enhancement module as initial reconstructed features Y;

[0096] Map the underwater prior information background light I B , scene depth I D , and water type information I W to the feature space through the stacked convolution-instance normalization-activation layer to obtain reconstructed prior features R B , R D , and R β matching the feature dimension, and the process can be represented as:

[0097] R B , R D , R β = C CIR (I B ), C CIR (I D ), C CIR (I W )

[0098] Wherein, C CIR represents the stacked convolution-instance normalization (IN)-activation layer (ReLU) operation;

[0099] ​The reconstructed prior feature and the initial reconstructed feature are input into a dual-compactor block (DCB) to extract global statistical information and significant region information of the underwater prior feature. Specifically, the dual-compactor block aims to learn a global distribution descriptor related to a channel domain to achieve effective coding of information. The dual-compactor block includes two sub-compactors: a stacked global average pooling-full connection layer unit and a stacked global maximum pooling-full connection layer unit, which extract and retain key information of the spatial dimension in different ways, i.e., global statistical information and significant region information of the underwater prior feature, denoted as F k Mean ∈R c×1×1 and F k Max ∈R c×1×1 ;

[0100] Studies have shown that the maximum pooling feature encoding the most significant part can compensate for the deficiency of the average pooling feature encoding the global statistical information. Therefore, two Sigmoid activation functions are further used to generate attention weights of the two sub-compactors, denoted as W k Mean and W k Max , i.e., the attention weights of the global statistical information and the significant region information are generated by Sigmoid activation. Based on the attention weights, the global statistical information of each underwater prior feature is fused with its corresponding significant region information by element-wise addition to obtain a prior information feature descriptor Z k . The fusion expression is:

[0101]

[0102] where k = {B, D, β, Y}, C DCB represents the processing procedure of the DCB module.

[0103] After obtaining the initial reconstructed feature and the plurality of prior information feature descriptors, the features are further connected to splice the initial reconstructed feature and the prior information feature descriptors into a hybrid feature, i.e., Z = [Z B ; Z D ; Z β ; Z Y ], where Z represents a hybrid feature descriptor. In order to make the image reconstruction network more sensitive to information features and to adaptively recalibrate and combine multiple priors according to the importance of the input image, the mutual dependence between the underwater prior information and the underwater image in the channel domain is captured by a double convolution layer, and the importance weights of the underwater prior information are generated by a Softmax operation. The expression is:

[0104] t = σ(W2*(W1*Z))

[0105] where W1 and W2 represent two learnable convolutional weights, * represents a convolution operation, and σ represents a Softmax operation for calculating the importance of each prior information;

[0106] After multiple calculations, the importance weight vector set t of the underwater prior information is obtained n where n = {B, D, β};

[0107] The reconstructed prior feature is multiplied by the importance weight of each underwater prior information channel by channel, and the reconstructed prior feature is adaptively selected and combined with the preset underwater domain knowledge to obtain the fused multi-prior feature F s = [t1F1; t2F2; t3F3];

[0108] It can be observed that the expression of the enhanced image J λ (x) is composed of two independent terms: the basic content of the degraded underwater image and the multiple prior knowledge of the underwater image. Based on this observation, two learnable and principle-inspired modulation maps are introduced to extract the prior guidance features from the fused multi-prior feature representation and convert them into a pixel-level scale transformation matrix γ and a translation transformation matrix β. These two modulation maps are generated by convolution layers with different convolution kernels and obtain information from multiple domain knowledge through learning. The learned dynamic modulation map can capture the prior features most relevant to the degradation of the input image, thereby more flexibly adjusting the image content features to improve the image quality. In addition, in order to avoid the problem of gradient disappearance, the MPGM module introduces a data regularization term to constrain the optimization process of the content features, making them train towards the expected stable state, i.e. where μ c and σ c represent the mean and standard deviation of Y in channel c, respectively. Finally, the normalized initial reconstructed feature is adjusted by the modulation map to obtain the optimized reconstructed feature, thereby improving the image enhancement effect. The calculation process expression is:

[0109]

[0110] where Y out represents the adjusted content feature map, and Y represents the normalized original content feature.

[0111] The structure and working process of the multi-prior aggregation learning image enhancement model in the above enhancement method are described in detail. This model is trained based on pre-established or existing data sets in an end-to-end manner. The loss function used for training includes a content loss L con and a perception loss L per, the content loss adopts L1 norm, aiming to minimize the pixel difference between the enhanced image and the label image, and the perception loss L per aims to ensure the consistency of the output image and the label image in the high-dimensional space, and improve the subjective perception quality of the output image. The expression of the complete loss function is:

[0112] L total = λ1*L com + λ2*L per

[0113] Wherein, L com is the content loss, L per is the perception loss, λ1 and λ2 represent the weight parameters, which are respectively taken as 0.8 and 0.2 in this embodiment.

[0114] L con = |UMPAN(I n )-J n |1

[0115]

[0116] Wherein, represents the feature map of the pool-3 layer of the VGG-19 network pre-trained on the ImageNet dataset, UMPAN(·) represents the underwater multi-prior aggregation model proposed in this method, I n represents the input original underwater image, and J n represents the corresponding underwater label image.

[0117] Embodiment 2

[0118] Based on the above embodiment 1, this embodiment discloses an actual application and effect comparison example of the underwater image enhancement method driven by multi-prior aggregation learning.

[0119] Model training:

[0120] To train the enhancement network, this embodiment divides the disclosed HUIS synthetic dataset into two parts: 3547 pairs as training images, denoted as Train-S3547, and the remaining 887 pairs as test images, denoted as Test-S887. The underwater image enhancement algorithm driven by multi-prior aggregation learning (UMPAN) is realized based on the PyTorch platform, and all experiments are carried out on a Linux workstation equipped with two NVIDIA Titan V GPUs.

[0121] The model is trained for 100 cycles, and the learning rate is fixed at 1×10 -4The network is trained using the Adam optimizer with a batch size of 6, default values of 0.5 and 0.999 for β1 and β2, and a weight decay of 0.00005. During the training and testing phases, the input images are resized to 256x256 and the pixel values are normalized to the range [-1, 1]. To prevent the model from overfitting, data augmentation techniques are used during the training phase, including random rotations of 90°, 180°, and 270°, and horizontal flipping to further expand the training samples.

[0122] Comparison algorithm, data set, evaluation index:

[0123] This embodiment compares eight advanced underwater image enhancement methods: UWCNN, FUIE-GAN, UWGAN, LCNet, Ucolor, TOPAL, SGUIE, and UTransformer (abbreviated as UTrans).

[0124] The performance of all algorithms is verified on four real underwater image datasets: the EUVP dataset, the SQUID dataset, the UIEB dataset, and the more challenging UIEB-C dataset. To ensure the fairness of the experiment, all comparative methods use the models and parameters published by the original authors for testing, and three no-reference indicators, UIQM, CCF, and Edge intensity, are used for performance evaluation. To more objectively evaluate the performance of each algorithm, this embodiment also conducts a subjective evaluation comparison experiment, inviting 15 users with image processing knowledge to subjectively score 100 enhancement results randomly selected from each dataset. To eliminate subjective bias, participants are not aware of the specific algorithm corresponding to the results before scoring. The scoring uses a 5-point scale, with 5 points representing very good image subjective quality, 4 points representing good image subjective quality, 3 points representing general image subjective quality, 2 points representing poor image subjective quality, and 1 point representing very poor image subjective quality. The scoring results are shown in Table 1.

[0125] Table 1 Comparison of objective indicators of different methods on four datasets

[0126]

[0127] Table 1 shows the quantitative comparison results of different algorithms on four real underwater image datasets. From the table, it can be seen that the UMPAN algorithm disclosed in Embodiment 1 achieves the best performance in the indicators, and significantly outperforms other advanced methods (such as SGUIE and UTrans), which indicates that the algorithm can achieve enhancement results with better perceptual quality and has stable performance, and can adapt to underwater images of different water qualities.

[0128] These results show that introducing multiple underwater priors as auxiliary information can more effectively enhance underwater images. The UMPAN results also demonstrate that the generated images have pleasing visual quality. In contrast, FUIE-GAN and LCNet aim to build fast and lightweight models with fewer parameters, which are prone to performance bottlenecks on complex distorted training samples. UWCNN and UWGAN employ end-to-end training without introducing underwater priors, which perform poorly on these metrics. Ucolor and SGUIE only introduce a single prior and ignore other image domain knowledge, resulting in limited performance. These results show that the invented method can be well generalized to different real underwater scenes and further verify the advantages of the network structure designed specifically for underwater image enhancement.

[0129] Figure 5 and Figure 6 Some examples of EUVP and SQUID datasets are shown. These images present obvious blue-green or blue color tones. For such images, the UMPAN algorithm proposed in Embodiment 1 can effectively remove color bias, improve contrast, and further enhance structural details. This effect benefits from the adaptive combination and application of multiple domain knowledge by the algorithm. In contrast, the enhancement effect of the comparative methods is limited. Ucolor, TOPAL, and UTrans still retain a certain degree of color distortion. FUIE-GAN and UWGAN introduce additional color bias, reducing the result quality. LCNet and UWCNN even introduce obvious over-enhancement phenomena in the results. Figure 7 and Figure 8 Some examples of UIEB and UIEB-C datasets are shown. It is observed that Figure 7 As can be seen, for images with light green and dark green tones in different water areas, the proposed UMPAN algorithm can effectively enhance the foreground structure (such as plants and turtles), reduce color bias, and remove haze on the image, ultimately presenting more natural and more visually perceptual results. In contrast, the comparative algorithms often introduce unpleasant artifacts in the results due to the neglect of multiple beneficial underwater domain knowledge, and even destroy image structures and introduce additional noise. As shown in Figure 8 For these severely distorted underwater images, the UMPAN algorithm can still restore natural appearance results and avoid noise interference. In contrast, the comparative methods cannot restore the details of the nearby coral reef and even introduce more serious color artifacts, further reducing the image quality.

[0130] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements shall be encompassed within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An underwater image enhancement method driven by multi-prior aggregation learning, characterized in that, The method comprises: obtaining an underwater image and inputting a pre-trained multi-prior aggregated learning image enhancement model to obtain an enhanced underwater image; the multi-prior aggregated learning image enhancement model comprises an underwater multi-prior extraction module, a multi-prior perception enhancement module and a multi-prior guided modulation module; the process of enhancing the underwater image by the multi-prior aggregated learning image enhancement model comprises: performing multi-prior extraction on the underwater image by the underwater multi-prior extraction module to extract underwater prior information, including background light, scene depth and water type information; removing underwater degradation in the underwater image from a feature level based on the underwater prior information by the multi-prior perception enhancement module to obtain picture features after degradation mitigation; dynamically modulating to generate reconstruction features based on the underwater prior information and the picture features after degradation mitigation by the multi-prior guided modulation module; and generating and outputting the enhanced underwater image based on the reconstruction features.

2. The method of claim 1, wherein, The architecture of the multi-prior aggregated learning image enhancement model is based on a U-Net network.

3. The method of claim 2, wherein, The reconstruction features are processed by a final convolutional layer and an activation layer of the U-Net network to generate and output the enhanced underwater image.

4. The method of claim 1, wherein, The process of performing multi-prior extraction on the underwater image to extract underwater prior information comprises: processing the underwater image by using a physical model-based underwater image enhancement method to obtain background light; processing the underwater image by using a DPT method to generate a scene depth map; classifying the underwater image by using a preset water classification network to obtain a corresponding water type.

5. The method of claim 1, wherein, The multi-prior perception enhancement module comprises stacked convolution-activation layers and stacked convolution-Sigmoid layers; the multi-prior guided modulation module comprises stacked convolution-instance normalization-activation layers, a double compression unit and double convolutional layers, the double compression unit comprises stacked global average pooling-full connection layers and stacked global maximum pooling-full connection layers.

6. The method of claim 5, wherein, The process of obtaining the picture features after degradation mitigation by the multi-prior perception enhancement module comprises: mapping the underwater prior information to a feature-level prior parameter space by the stacked convolution-activation layers to obtain prior features matching the feature dimension; calculating adjustment parameters related to the attenuation coefficient, depth and background light based on the prior features by the stacked convolution-Sigmoid layers; extracting initial features from the original underwater image, adjusting the initial features based on the adjustment parameters to obtain the picture features after degradation mitigation.

7. The method of claim 6, wherein, The calculation expression of the adjustment parameters is: a prior = C CS (F D · C CS (F β )) gamma prior = C CR (C CR (F B )+ C CR (F B ) · F D · C CS (F β )) where α prior is an adjustment parameter related to the attenuation coefficient and depth, γ prior is an adjustment parameter related to the attenuation coefficient, depth, and background light, C CS represents a stacked convolution-Sigmoid activation function operation, F D , F β , and F B are prior features, C CR represents a stacked convolution-activation layer operation; The calculation expression of the picture features after degradation mitigation is: X out = a prior • X + γ prior wherein X represents the initial features extracted from the original underwater image.

8. The method of claim 5, wherein, The process of generating the reconstruction features by the multi-prior guided modulation module comprises: obtaining the picture features after degradation mitigation from the multi-prior perception enhancement module as initial reconstruction features; mapping the underwater prior information to a feature space by the stacked convolution-instance normalization-activation layers to obtain reconstruction prior features matching the feature dimension; inputting the reconstruction prior features and the initial reconstruction features into the double compression unit to extract global statistical information and significant area information of the underwater prior features; The attention weight of the global statistical information and the salient region information is generated by a Sigmoid activation, and the global statistical information of each underwater prior feature is fused with the corresponding salient region information based on the attention weight, to obtain a prior information feature descriptor; The initial reconstruction feature and the prior information feature descriptor are spliced into a hybrid feature, and a mutual dependency between the underwater prior information and the underwater image in a channel domain is captured by a double convolution layer, and an importance weight of each underwater prior information is generated by a Softmax operation; The reconstruction prior feature is multiplied with the importance weight of each underwater prior information channel by channel, and a preset underwater domain knowledge is adaptively selected and combined from the reconstruction prior feature, to obtain a fused multi-prior feature; A prior guidance feature is extracted from the fused multi-prior feature, and is converted into a pixel-level scale transformation matrix and a translation transformation matrix, and the normalized initial reconstruction feature is adjusted based on the scale transformation matrix and the translation transformation matrix, to obtain an optimized reconstruction feature.

9. The method of claim 8, wherein, A calculation expression of the optimized reconstruction feature is as follows: where Y out denotes the optimized reconstructed feature, denotes the normalized initial reconstructed feature, γ is a scale transformation matrix, and β is a translation transformation matrix.

10. The method of claim 1, wherein, A loss function for training the multi-prior aggregation learning image enhancement model is as follows: L total = γ1*L com + λ2*L per wherein L com is the content loss, L per is the perceptual loss, and λ1and λ2represent weight parameters; L con = | UMPAN(I n )-J n |1 wherein, denotes the feature map of the pool-3 layer of the VGG-19 network pre-trained on the ImageNet dataset, UMPAN(·) denotes the underwater multi-prior aggregation model proposed in this method, I n denotes the input original underwater image, J n denotes the corresponding underwater label image.

Images