Vision language model guided heterogeneous degradation divide-and-conquer underwater image enhancement method

Abstract

The application provides a visual language model guided heterogeneous degradation divide-and-conquer underwater image enhancement method, including the following steps: S1, obtaining an original underwater image to be enhanced; S2, generating a degradation composition weight from a macroscopic perspective analysis of the overall degradation composition of the original underwater image; S3, generating a specific degradation degree map corresponding to each degradation type from a microscopic perspective perception of the spatial severity distribution of each degradation type in the original underwater image; S4, obtaining a mixed expert structure including multiple parallel specific degradation task experts, performing image enhancement processing on the original underwater image by each specific degradation task expert under the spatial guidance of the specific degradation degree map to obtain corresponding output features, and then dynamically fusing each output feature by using the degradation composition weight to obtain an enhanced underwater image. The beneficial effect is that the application can realize effective enhancement of complex underwater images under the premise of fully considering the diversity and spatial distribution heterogeneity of the degradation types.

Description

Technical Field

[0001] This invention relates to the technical field of underwater image enhancement, and more specifically, to a visual language model-guided divide-and-conquer method for underwater image enhancement based on heterogeneous degradation. Background Technology

[0002] High-quality underwater imagery is crucial for applications such as marine archaeology, autonomous underwater robotic operations, marine ecological monitoring, and underwater security. However, due to the selective absorption of light waves by the water medium and the scattering effect of suspended particles in the water, raw underwater images generally suffer from multiple visual quality degradation problems, including color cast, haze effects, and insufficient brightness. Therefore, underwater image enhancement technology has emerged, aiming to eliminate or suppress these degradation effects and restore low-quality raw images to high-quality images with higher color fidelity, clarity, and visual naturalness, thereby providing reliable image data support for various underwater visual tasks.

[0003] Currently, various underwater image enhancement methods have been proposed, which can be broadly categorized into physical model-based methods, traditional image processing-based methods, and deep learning-based methods. Among these, deep learning-based methods have become the mainstream research direction in this field in recent years due to their powerful data-driven fitting capabilities and end-to-end mapping learning capabilities.

[0004] However, underwater image degradation presents two core challenges that existing methods have failed to address effectively.

[0005] First, multiple degradation types often coexist and are coupled with each other. In real underwater scenes, degradation types such as green color cast, blue color cast, haze effect, and low illumination usually occur together, affecting the same image, and different degradations influence and are interconnected. Under such multi-coupled degradation conditions, the restoration process faces inherent conflicts between different optimization objectives—for example, eliminating haze may exacerbate the loss of details in dark areas, while increasing brightness may amplify color cast. Most existing methods use a single network to directly learn the end-to-end mapping from degraded images to clear images, or blindly decompose coupled degradations implicitly, lacking explicit analysis and clear judgment of which degradation types are specifically contained in the image and the proportion of each type. This approach makes it difficult for the network to have a clear processing basis when facing complex mixed degradations, making it difficult to effectively coordinate the priority and resource allocation between different degradation restoration tasks, resulting in the final enhancement effect performing well on some degradation types but poorly on others.

[0006] Second, the spatial distribution of different degradation types is severely uneven. In a single underwater image, the severity of degradation often varies significantly across different regions: the same image may simultaneously contain areas with severe haze and relatively clear areas, or areas with significant color casts and areas with normal colors. However, existing methods typically employ a uniform spatially invariant enhancement strategy to treat all regions of the entire image the same way, with very few methods characterizing the independent spatial severity distribution for each specific degradation type. Even those few methods that introduce general attention mechanisms or a single universal degradation map to guide enhancement cannot distinguish the differences in spatial distribution among different degradation types—for example, haze may be mainly concentrated in the distant areas of the image, while low illumination may be evenly distributed across the entire image, and color casts may only be significant in local areas. A single universal degradation map cannot simultaneously and accurately represent the heterogeneous spatial distributions of these different types, resulting in a lack of precise spatial prior information for specific degradation types during the local region restoration process, making it difficult to achieve true spatial differential enhancement.

[0007] In conclusion, how to effectively enhance complex underwater images while fully considering the diversity of degradation types and the heterogeneity of spatial distribution remains a pressing technical challenge in the current field. Summary of the Invention

[0008] The technical problem to be solved by this invention is how to effectively enhance complex underwater images while fully considering the diversity of degradation types and the heterogeneity of spatial distribution. In order to overcome the defects of the above-mentioned prior art (or related art), this invention provides a divide-and-conquer underwater image enhancement method for heterogeneous degradation guided by a visual language model.

[0009] This invention provides a visual language model-guided divide-and-conquer method for underwater image enhancement based on heterogeneous degradation, comprising the following steps: Step S1: Obtain the original underwater image to be enhanced; Step S2: Analyze the overall degradation composition of the original underwater image from a macroscopic perspective to generate degradation composition weights, and quantify the composition ratio of various degradation types in the original underwater image through the degradation composition weights. Step S3: Perceive the spatial severity distribution of each degradation type in the original underwater image from a microscopic perspective, and generate a specific degradation degree map corresponding to each degradation type; Step S4: Obtain a hybrid expert structure including multiple parallel experts for specific degradation tasks. Under the spatial guidance of the specific degradation degree map, perform image enhancement processing on the original underwater image through each of the specific degradation task experts to obtain corresponding output features. Then, use the degradation composition weights to dynamically fuse each of the output features to obtain the enhanced underwater image.

[0010] The visual language model-guided divide-and-conquer underwater image enhancement method of this invention has the following advantages compared with the prior art: This invention first quantifies the proportion of various degradation types in the entire original underwater image at a macroscopic level, and then perceives the spatial severity distribution of each degradation type at a microscopic level. This explicitly decomposes the originally coupled complex degradation problem into multiple relatively independent sub-tasks, thereby reducing the processing difficulty of simultaneous restoration of multiple degradations. Furthermore, by generating a specific degradation degree map independently for each degradation type, subsequent enhancement operations can perform differentiated processing based on the actual severity of different locations and degradation types, avoiding the drawbacks of a one-size-fits-all approach. At the same time, the degradation composition weights obtained from macroscopic analysis are used to dynamically fuse the output features of experts for each specific degradation task. This allows the final enhanced underwater image to adaptively adjust the contribution ratio of each degradation type restoration processing according to the actual degradation composition of each input original underwater image for personalized enhancement. Under the premise of fully considering the diversity of degradation types and spatial distribution heterogeneity, this invention achieves effective enhancement of complex underwater images.

[0011] In one possible implementation, step S2 includes: Step S21: Perform adaptive average pooling and multilayer perceptron processing on the original underwater image to obtain the corresponding image feature path score vector; Step S22: Input the original underwater image and the fine-tuned and optimized degradation type prompt word embedding into the image editor and text editor of the visual language model, respectively, and calculate the cosine similarity between the image features of the original underwater image and the text prompt features of the degradation type prompt word embedding as the degradation composition score vector; Step S23: The image feature path score vector and the degradation composition score vector are weighted and fused, and then normalized by the Softmax function to generate the degradation composition weight.

[0012] Compared with existing technologies, the above technical solution can make full use of the dual-path collaborative approach of image feature path and visual language model guided path. The image feature path directly extracts discrimination information from the original underwater image and has strong task adaptability. The visual language model guided path utilizes the rich semantic knowledge contained in the large-scale pre-trained visual language model, which can more accurately understand the abstract degradation concept of degradation type. The two paths are weighted and fused to complement each other's strengths and weaknesses, which can utilize prior knowledge and adapt to specific data distribution.

[0013] In one possible implementation, the method further includes the following steps before performing step S21: Construct a prompt word embedding that includes each of the aforementioned degradation types, and fine-tune and optimize the prompt word embedding using relation ranking loss to obtain the degraded type prompt word embedding. The relation ranking loss includes primary and secondary relation loss and saliency relation loss. The primary and secondary relation loss is used to constrain the matching score corresponding to the primary degradation type to be higher than the matching score corresponding to the secondary optimization type, and the saliency relation loss is used to constrain the matching score corresponding to the saliency degradation type to be higher than the matching score corresponding to the non-saliency optimization type.

[0014] Compared with existing technologies, the above-mentioned technical solution can fine-tune the cue word embedding by using a relation ranking loss that includes primary and secondary relation loss and saliency relation loss, enabling the visual language model to more accurately identify the primary and secondary relations, saliency relations, and non-saliency relations between different degradation types in the same original underwater image.

[0015] In one possible implementation, step S3 includes: Step S31: Divide the original underwater image into multiple grid image blocks, and construct corresponding severity and mildness cue words for each type of degradation. Step S32: Perform cosine similarity calculation on each grid image block with each severity cue word and each minor cue word to obtain the corresponding severity matching score and minor matching score; Step S33: Subtract the severity matching score and the slightness matching score corresponding to each grid image block and transform them using the Sigmoid function to obtain the degradation score of each grid image block under the corresponding degradation type; Step S34: Assemble the degradation degree scores of all the grid image blocks under the same degradation type into a map, and generate the corresponding specific degradation degree map after Gaussian smoothing.

[0016] Compared with existing technologies, the above-mentioned technical solution can divide the entire original underwater image into grid image blocks and make independent judgments at the block level, capturing the differences in degradation degree between different spatial locations and generating a specific degradation degree map with fine spatial resolution, providing accurate location priors for subsequent spatial differentiation enhancement. Furthermore, for each degradation type, two extreme prompt words, severe and mild, are set simultaneously, and the actual degradation degree is measured by the difference between their matching scores. This is equivalent to establishing a relative measurement scale, which is more accurate than a single absolute score and can effectively eliminate the bias caused by the inherent similarity differences between different prompt words.

[0017] In one possible implementation, in step S4, the specific degradation level map is convolved and split to obtain multiplicative space modulation parameters and additive space modulation parameters. The original underwater image is then modulated using the multiplicative space modulation parameters and the additive space modulation parameters to obtain modulation features with degradation space prior. Subsequently, the modulation features are processed sequentially through Mamba blocks and gated depth convolutional feedforward networks to obtain the output features of the specific degradation task expert.

[0018] Compared with existing technologies, the above-mentioned technical solution can convert a specific degradation level map into multiplicative and additive spatial modulation parameters and then perform pixel-by-pixel modulation on the image features. This enables the feature response to be adaptively adjusted according to the actual degradation severity at different spatial locations, achieving true spatially perceptive enhancement. The Mamba block has the ability to model long program sequences with linear complexity, which can capture global contextual dependencies in the original underwater image. The gated deep convolutional feedforward network preserves local spatial details through deep convolution, while the gating mechanism enhances nonlinear expression capabilities. The combination of the two achieves both global structure restoration and local detail preservation.

[0019] In one possible implementation, in step S4, for each specific degradation task expert, the output feature of the specific degradation task expert is multiplied by the weight element corresponding to the specific degradation task expert in the degradation composition weight to obtain a weighted output feature. Then, all the weighted output features are added element by element to obtain the fused output feature as the enhanced underwater image.

[0020] Compared with existing technologies, the above-mentioned technical solution can assign fusion weights that match the degradation ratio of each specific degradation task expert, making the fused output features more consistent with the actual situation of each input original underwater image. The training of each specific degradation task expert and the generation of fusion weights are completed in an integrated manner in the network. The weight allocation not only reflects the initial degradation composition judgment, but is also supervised by the final enhancement quality, forming a complete closed loop from degradation judgment to expert processing to on-demand fusion to quality feedback, ensuring that experts of each specific degradation task cooperate and complement each other rather than interfere with each other.

[0021] In one possible implementation, each degradation type in step S2 includes green color bias, blue color bias, haze, and low illumination. The degradation composition weight includes four weight elements, which correspond to green color bias, blue color bias, haze, and low illumination, respectively. Each specific degradation degree map includes four specific degradation degree maps, which correspond to green color bias, blue color bias, haze, and low illumination, respectively. Multiple parallel specific degradation task experts include green bias expert, blue bias expert, haze expert, and low illumination expert, which correspond to green color bias, blue color bias, haze, and low illumination, respectively.

[0022] Compared with existing technologies, the above technical solution enables each degradation type to have independent weight elements, independent specific degradation degree maps, and independent specific degradation task experts, forming a complete independent channel from perception to modeling to processing to fusion, which is conducive to the full development of image enhancement in four parallel dimensions.

[0023] In one possible implementation, the visual language model-guided heterogeneous degradation divide-and-conquer underwater image enhancement method is implemented on a U-shaped network, which includes multiple identical processing blocks, each of which is used to perform steps S2, S3, and S4.

[0024] Compared with existing technologies, the above technical solution enables the degradation composition weights and specific degradation degree maps to be continuously updated at each level of the U-shaped structure network. Low-level features provide fine spatial location information, while high-level features provide semantic discrimination criteria. The system is refined step by step from coarse to fine, and the enhancement quality is continuously improved.

[0025] In one possible implementation, step S2 is implemented through a degradation-based perceptual routing module, step S3 is implemented through a degradation-level perceptual mapping module, and step S4 is implemented through a degradation processing and fusion module. After executing step S4, the method further includes: Using an underwater image dataset containing paired enhanced and reference images, a Smooth L1 loss is calculated based on the pixel values ​​of each pixel in the enhanced and reference images. An SSIM loss is calculated based on the mean, standard deviation, and covariance of the grid image patches formed with each pixel as the center in the enhanced and reference images. The weighted sum of the Smooth L1 loss and the SSIM loss is used as the total loss. End-to-end training is then performed on the overall network consisting of the degradation perception routing module, the degradation degree perception mapping module, and the degradation processing and fusion module, which are sequentially connected.

[0026] Compared with existing technologies, the above-mentioned technical solution can compare the enhanced image and the reference image point by point at the pixel level based on Smooth L1 loss, ensuring basic color and brightness fidelity; while SSIM loss constrains brightness, contrast and structural consistency at the grid image block level, which is closer to the quality perceived by the human eye. The combination of the two can simultaneously optimize both pixel accuracy and visual perception quality.

[0027] In one possible implementation, the reference images in the underwater image dataset are uniformly scaled to a fixed pixel size before being input to the degradation-based perceptual routing module, and undergo random horizontal and vertical flipping for data augmentation. The weight coefficients of the Smooth L1 loss and the SSIM loss in the total loss are set to 1 and 0.3, respectively.

[0028] Compared with existing technologies, the above-mentioned technical solution can effectively expand the diversity of training samples without changing the semantic content by randomly flipping horizontally and vertically, reducing the overall network's over-reliance on image orientation and spatial location, and improving the generalization performance of visual language models on unseen data. Attached Figure Description

[0029] Figure 1 This is a flowchart of the steps of the present invention; Figure 2 This is a schematic diagram of the overall HDDC network structure of the present invention; Figure 3 This is a flowchart illustrating step S2 of the present invention. Figure 4 This is a schematic diagram illustrating the fine-tuning and optimization principle of the embedding of degradation type prompt words in this invention. Figure 5 This is a flowchart illustrating step S5 of the present invention. Figure 6 This is a schematic diagram of the feature modulation principle of the specific degradation task expert of the present invention; Figure 7 This is a schematic diagram of the quantitative measurement results of the nine methods of the present invention on the UIEB-T89 test dataset; Figure 8 This is a schematic diagram of the quantitative measurement results of the nine methods of the present invention on the UIEB-C60, SQUID-T16, U45, and EUVP-T515 test datasets; Figure 9 This is a schematic diagram illustrating the visual enhancement results of the RAW, WaterNet, UWCNN, Ucolor, and PUIE-Net methods of the present invention; Figure 10 This is a schematic diagram illustrating the visual enhancement results of Semi-UIR, NU2Net, HCLR-Net, SS-UIE, and the method of this invention. Detailed Implementation

[0030] First, those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0031] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0032] See Figure 1 This invention discloses a visual language model-guided divide-and-conquer method for underwater image enhancement based on heterogeneous degradation, comprising the following steps: Step S1: Obtain the original underwater image to be enhanced; Step S2: Analyze the overall degradation composition of the original underwater image from a macroscopic perspective to generate degradation composition weights, and quantify the composition ratio of various degradation types in the original underwater image through degradation composition weights. Step S3: Perceive the spatial severity distribution of each degradation type in the original underwater image from a microscopic perspective, and generate a specific degradation degree map corresponding to each degradation type; Step S4: Obtain a hybrid expert structure including multiple parallel experts for specific degradation tasks. Under the spatial guidance of a specific degradation degree map, perform image enhancement processing on the original underwater image through each specific degradation task expert to obtain the corresponding output features. Then, use degradation composition weights to dynamically fuse each output feature to obtain the enhanced underwater image.

[0033] In this embodiment of the invention, the "divide and conquer" approach to heterogeneous degradation includes a "divide" stage and a "conquer" stage. In the "divide" stage, a visual language model is introduced to analyze the overall degradation composition from a macroscopic perspective. The Degradation Composition Aware Routing (DCAR) module analyzes the overall degradation composition of the input raw underwater image to obtain degradation composition weights. These weights quantify the proportion of each degradation type and achieve explicit decoupling of the restoration task. Furthermore, the spatial severity distribution of each degradation type is perceived from a microscopic perspective. A specific degradation degree map is generated through the Degradation Degree Aware Mapping (DDAM) module. This specific degradation degree map reflects the fine-grained spatial severity of each degradation type, explicitly decomposing the spatial distribution of general degradation into the spatial distribution of specific degradation. In the "conquer" stage, the Degradation Composition Fusion module... The Module (DCFM) constructs an enhancement mechanism based on hybrid experts. Degradation constitutes weights as dynamic weights to allocate the contributions of each expert for a specific degradation task, while a specific degradation degree map provides spatial degradation priors to guide the adaptive and spatially aware enhancement within each expert for a specific degradation task.

[0034] See Figure 2In this embodiment of the invention, step S2 is implemented through a degradation composition-aware routing module, step S3 through a degradation degree-aware mapping module, and step S4 through a degradation processing and fusion module. The visual language model-guided heterogeneous degradation divide-and-conquer underwater image enhancement method is implemented on a U-shaped network. The U-shaped network includes five structurally identical processing blocks, two downsampling operations, and two upsampling operations. Each processing block embeds a degradation composition-aware routing module, a degradation degree-aware mapping module, and a degradation processing and fusion module, which are repeatedly embedded at different levels. The aforementioned modules can continuously perform degradation composition decomposition, degradation spatial distribution modeling, and multi-expert collaborative restoration in a multi-scale feature space. In each processing block, the input feature map of the current layer, i.e., the original underwater image, is processed by the degradation composition perception routing module and the degradation degree perception mapping module to generate degradation composition weights and specific degradation degree maps, respectively. Then, the degradation processing and fusion module performs degradation processing and fusion to output the enhanced feature map of the current layer. This enhanced feature map is downsampled and input into the next processing block, or upsampled and concatenated with the corresponding layer features and input into the previous level processing block, thereby refining and enhancing the effect step by step.

[0035] In this embodiment of the invention, in step S2, the multiple degradation types include four types: green color bias, blue color bias, haze, and low illumination. The degradation composition weights include four weight elements corresponding to green color bias, blue color bias, haze, and low illumination, respectively. Each specific degradation degree map includes four specific degradation degree maps corresponding to green color bias, blue color bias, haze, and low illumination, respectively. Multiple parallel specific degradation task experts include green bias expert, blue bias expert, haze expert, and low illumination expert, respectively, corresponding to green color bias, blue color bias, haze, and low illumination.

[0036] See Figure 3 In this embodiment of the invention, step S2 includes: Step S21: Perform adaptive average pooling and multilayer perceptron processing on the original underwater image to obtain the corresponding image feature path score vector; Step S22: Input the original underwater image and the fine-tuned and optimized degradation type prompt word embedding into the image editor and text editor of the visual language model, respectively, and calculate the cosine similarity between the image features of the original underwater image and the text prompt features of the degradation type prompt word embedding as the degradation composition score vector. Step S23: The image feature path score vector and the degradation composition score vector are weighted and fused, and then normalized by the Softmax function to generate the degradation composition weight.

[0037] See Figure 2 and Figure 4 In this embodiment of the invention, before performing step S2, a fine-tuning and optimization process for the embedding of degenerate type prompt words is included, comprising: Step A1, obtaining training samples, specifically: 801 pairs of paired images were selected from the publicly available underwater image augmentation dataset UIEB as training samples, denoted as UIEB-T801; Step A2 involves constructing the embedding of the degradation type cue words and obtaining the degradation composition score vector, specifically: For four typical underwater degradation types, cue words Text 1 is constructed, namely: A photo of a green-tinted underwater scene; A photo of a blue-tinted underwater scene; A photo of a hazy underwater scene; A photo of a low-light underwater scene. Correspondingly, the degradation type cue words constructed in Text 1 are embedded... Then it is defined as: ; in, Indicates the first Embedding of prompt words corresponding to class degradation types, The degradation type indices are 1, 2, 3, and 4, which correspond to green color shift, blue color shift, haze, and low light, respectively. Original underwater images The input is a frozen CLIP image encoder. The aforementioned prompts are embedded into the input frozen CLIP text encoder. The cosine similarity between image features and various text prompt features is calculated to obtain the degenerate composition score vector. : ; in, This represents a degeneracy composition score vector consisting of matching scores corresponding to four types of degeneracy. This indicates that the input raw underwater image is compared with the first... Matching score between the embeddings of prompt words corresponding to class degradation types; This represents the cosine similarity operation; This indicates CLIP image encoder processing; This indicates CLIP text encoder processing; This represents the original underwater image input. Indicates the first Embedding of prompt words corresponding to class degradation types; Step A3 involves constructing the relation ranking loss and optimizing the cue word embedding, specifically as follows: For each training sample, define a binary label vector: ; Among them, the element Indicates the first The saliency label of the class degradation type in the current training samples, when When, it indicates the first The class degradation type is significant in the current training samples; when When, it indicates the first The class of degradation types is not significantly present in the current training samples; meanwhile, in all significantly present degradation types (i.e. Define a dominant degradation index in ) This is used to represent the primary degradation type in the original underwater image. Further, all degradation types can be divided into three sets: the primary degradation set... Secondary degenerate set Non-significantly degenerate set ; Define the loss of primary and secondary relationships : ; in, Indicates the interval parameter representing the primary and secondary relationship; This represents the degeneracy composition score corresponding to the main degeneracy set; This represents the degeneracy score corresponding to the sub-degeneracy set; This indicates a maximum value operation. The primary and secondary relation loss is used to constrain the matching score corresponding to the primary degenerate set to be higher than the matching score corresponding to the secondary degenerate set. Take 0.1; Define the significance relationship loss : ; in, This represents the set of significant degradations, consisting of all degradation types that significantly exist. Indicates an index in a significantly degenerate set; Indicates an index in a non-significantly degenerate set; The interval parameter represents the significance of the relationship; This represents the degradation component score corresponding to significant degradation. This represents the degradation score corresponding to the non-significant degradation set; this significant relation loss is used to constrain the matching score corresponding to the significant degradation set to be higher than the matching score corresponding to the non-significant degradation set. Take 0.1; Define the final relation ranking loss : ; Minimize relation ranking loss This allows the optimized degeneration type cue word embeddings to better learn the primary and secondary relationships between complex degenerations.

[0038] In this embodiment of the invention, the fine-tuning and optimization process of the above-mentioned degradation type prompt word embedding is performed using the PyTorch framework, and training is completed on a single NVIDIA RTX 4090 GPU; the AdamW optimizer is used, with a learning rate set to 5e-5, a batch size set to 16, a total number of training epochs set to 100, and the training loss is the aforementioned relational ranking loss. .

[0039] See Figure 5 In this embodiment of the invention, step S3 includes: Step S31: Divide the original underwater image into multiple grid image blocks, and construct corresponding severity and mildness cue words for each degradation type; Step S32: Calculate the cosine similarity between each grid image block and each severity cue word and each minor cue word to obtain the corresponding severity matching score and minor matching score. Step S33: Subtract the severity matching score and the slightness matching score corresponding to each grid image block and transform them using the Sigmoid function to obtain the degradation score of each grid image block under the corresponding degradation type; Step S34: Assemble the degradation degree scores of all grid image blocks under the same degradation type into a map, and generate the corresponding specific degradation degree map after Gaussian smoothing.

[0040] In this embodiment of the invention, end-to-end training is also required on the overall network consisting of the degradation-aware routing module, the degradation-level-aware mapping module, and the degradation processing and fusion module, which are sequentially connected. This includes: Step B1, construct the overall HDDC network, specifically as follows: The HDDC network adopts a U-shaped structure, taking a single original underwater image, a finely tuned and optimized degradation type cue word embedding, and Text2 as input, and the output is the enhanced underwater image. Step B2 involves generating degradation composition weights using the degradation composition-aware routing module, specifically as follows: The degradation composition-aware routing module's role is to determine the overall degradation composition of the input raw underwater image from a macroscopic perspective, generating degradation composition weights denoted as... This degradation constitutes a dual-path fusion mechanism in the perceptual routing module, including image feature paths. and visual language model guided path ; For image feature paths, the current layer is input to the feature map. The data is sequentially fed into an adaptive average pooling layer and a multilayer perceptron to obtain the feature path output: ; in, This represents the image feature path score vector generated from the image feature path. This represents a multilayer perceptron; This indicates an adaptive average pooling operation; the MLP consists of two 1×1 convolutional layers and an intermediate ReLU activation function; For the visual language model-guided path, it takes the original underwater image and the finely tuned and optimized degradation type cue word embeddings as input, and calculates the cosine similarity using the frozen CLIP image encoder and CLIP text encoder to obtain the degradation composition score vector. ; After obtaining the image feature path score vector and the degradation composition score vector, the two path outputs are weighted and fused, and the final degradation composition weight is obtained by Softmax normalization. The calculation formula is as follows: ; in, This represents the Softmax normalization function; The weighting coefficients represent the feature paths in the image. The weighting coefficients represent the guiding paths of the visual language model; Take 0.8, If we set it to 1, the resulting degradation composition weights include four weight elements, which correspond to the degradation composition weights of four categories: green bias, blue bias, haze, and low illumination. These weights are used to guide the dynamic weighted fusion of experts for different specific degradation tasks. Step B3 involves generating a specific degradation level map using the degradation level-aware mapping module, specifically as follows: The degradation level perception mapping module's function is to perceive the spatial distribution and severity of different degradation types at a microscopic level, generating a specific degradation level map. First, the input raw underwater image is divided into... 1 grid, resulting in a total of 1 grid. Each grid image block is then input into a frozen CLIP image encoder. Simultaneously, for each degradation type, a pair of severity and severity cue words are constructed, denoted as Text 2, as follows: "green_cast": { "bad": "A photo of an underwater scene with a heavy green colorcast.", "good": "A photo of an underwater scene with a slight greencolor cast." }, "blue_cast": { "bad": "A photo of an underwater scene with a heavy blue colorcast.", "good": "A photo of an underwater scene with a slight bluecolor cast." }, "haziness": { "bad": "A photo of an underwater scene with heavy haze.", "good": "A photo of an underwater scene with slight haze." }, "illumination": { "bad": "A photo of a dark underwater scene.", "good": "A photo of a bright underwater scene." } For the Class Degeneration Type and the First For each grid image patch, calculate its cosine similarity to the severity cue word to obtain: ; in, Indicates the first The grid image patch and the first Matching scores between severity indicators of the degradation type; This represents the first segment of the original underwater image after it has been divided. A grid image block; Indicates the first Severity indicators for class degradation types; Indicates a degenerate type index; Indicates the grid image patch index; This represents the cosine similarity operation; For the Class Degeneration Type and the First For each grid image patch, calculate its cosine similarity to a mild cue word to obtain: ; in, Indicates the first The grid image patch and the first Match scores between mild degenerate type cue words; This represents the first segment of the original underwater image after it has been divided. A grid image block; Indicates the first Embedding of mild hints corresponding to class degenerate types; Subtract the two matching scores and apply a Sigmoid transform to obtain the degradation score of the corresponding grid image patch under this degradation type: ; in, Indicates the first The grid image block in the first The score representing the degree of degradation under class degradation; Represents the Sigmoid function; Indicates the first The grid image patch and the first Matching scores between severity indicators of the degradation type; This indicates an element-by-element subtraction operation. The value ranges from 0 to 1. The larger the value, the more severe the degradation of the grid image patch under the corresponding degradation type. Subsequently, the degradation scores corresponding to all grid image patches under the same degradation type are assembled into a fractional map, then mapped to obtain a coarse specific degradation level. Finally, after Gaussian smoothing, the final specific degradation level map is generated, denoted as . : ; in, Indicates the first A specific degradation degree map corresponding to a degradation type. The whiter the area in this specific degradation degree map, the higher the severity of the corresponding degradation type in that area. Step B4 involves performing degradation processing and fusion using the degradation processing and fusion module, specifically as follows: The degradation processing and fusion module adopts a hybrid expert MoE structure, which comprehensively utilizes the degradation composition weights generated by the degradation composition perception routing module and the specific degradation degree map generated by the degradation degree perception mapping module to perform collaborative enhancement processing on different degradation types. The degradation processing and fusion module includes four parallel specific degradation task experts, corresponding to green bias expert, blue bias expert, haze expert and low light expert, respectively. Each specific degradation task expert performs targeted processing on the input features under the spatial guidance of the corresponding specific degradation degree map. Subsequently, the outputs of the four experts are dynamically fused through the specific degradation degree map to obtain the final output features. The overall calculation process of the degradation processing and fusion module is represented as follows: ; in, This indicates the output characteristics of the degradation processing and fusion module; This represents an expert index for a specific degradation task. These correspond to four degradation types: green tint, blue tint, haze, and low illumination, respectively. Indicates the weight of degradation The corresponding number in the middle Weighting elements for a specific degradation task expert; This represents element-wise multiplication; Indicates the first Output characteristics of an expert for a specific degradation task; Indicates the first A specialist in a specific degradation task; like Figure 6 As shown, within each specific degradation task expert, spatial feature transformation is first used to modulate the input features. Specifically, the spatial transformation parameters are obtained by convolution and splitting operations on the corresponding specific degradation level map. and : ; in, Indicates the first Multiplicative space modulation parameters corresponding to a specific degenerate task expert; Indicates the first Additive spatial modulation parameters corresponding to a specific degenerate task expert; Indicates a splitting operation; Indicates the convolution operation; Subsequently, the input features are modulated using spatial feature transformation methods to obtain modulated features with degenerate spatial priors: ; in, Indicates the first Modulated features obtained from spatial feature transformation in a specific degradation task expert; through this operation, spatial degradation information contained in a specific degradation level map can be injected into the current feature stream, enabling the overall network to perform differentiated recovery for different degrees of severity at different locations; Subsequently, the modulation features with degradation space priors are sequentially fed into the Mamba block (MB) and the gated depthwise convolutional feedforward network (GDFN) to obtain the first... Output characteristics of an expert for a specific degradation task: ; in, Indicates the first Output characteristics of an expert for a specific degradation task; This represents a gated depthwise convolutional feedforward network; Indicates a Mamba block; Step B5, construct the training loss and optimize the overall HDDC network, specifically as follows: After completing the overall HDDC network construction, the network was trained using the UIEB-T801. During the training phase, Smooth L1 loss and SSIM loss were jointly used to supervise the network. The training loss is as follows: ; in, This represents the total loss during the overall network training phase of HDDC; Indicates enhanced image Compared with reference image Smooth L1 loss between; Indicates enhanced image Compared with reference image SSIM loss between; and These are the weighting coefficients corresponding to the loss. Take 1, Take 0.3; Specifically, the Smooth L1 loss is defined as: ; in, This represents the total number of pixels in the image; Indicates the pixel index; Indicates the first Smooth L1 loss corresponding to each pixel; Indicates enhanced image In the The pixel value at each pixel point; Represents reference image In the The pixel value at each pixel point; This represents the absolute value operation; SSIM loss is defined as: ; in, Indicates the first A grid of image blocks centered on a single pixel in image enhancement The mean of the values; Indicates the first A grid image patch centered on a pixel in the reference image The mean of the values; Indicates the first A grid of image blocks centered on a single pixel in image enhancement The standard deviation in; Indicates the first A grid image patch centered on a pixel in the reference image The standard deviation in; Indicates the first A grid of image blocks centered on a single pixel in image enhancement and reference image Covariance between them; and This represents a constant used to avoid a denominator of zero. Take 0.02, Take 0.03; The HDDC underwater image enhancement network was trained using the PyTorch framework on a single NVIDIA RTX 4090 GPU. The total number of training epochs was 200. The optimizer was AdamW. The initial learning rate was set to 1e-4. The learning rate scheduling strategy used cosine annealing, which was activated after the 40th epoch and eventually decayed to 1e-6. The batch size was set to 4. For data preprocessing, all input images were uniformly scaled to 256×256 pixels and enhanced with random horizontal and vertical flips. The grid scale G in the degradation-aware mapping module was set to 32. After training, an HDDC underwater image enhancement model suitable for testing applications was obtained.

[0041] In this embodiment of the invention, in order to further verify the feasibility and effectiveness of the method of the present invention, the method of the present invention is experimentally verified. Four widely recognized and publicly available underwater image enhancement benchmark datasets are selected for the experiment, namely UIEB, EUVP, SQUID and U45. Specifically, the test dataset includes five sub-datasets: UIEB-T89, UIEB-C60, SQUID-T16, U45 and EUVP-T515. In terms of evaluation metrics, two reference image quality evaluation metrics, PSNR and SSIM, and two referenceless underwater image quality evaluation metrics, UIQM and UCIQE, are adopted. In terms of comparison methods, the method of the present invention is compared with eight representative advanced underwater image enhancement methods, namely WaterNet, UWCNN, Ucolor, PUIE-Net, Semi-UIR, NU2Net, HCLR-Net and SS-UIE. The experiment consists of two parts: a quantitative indicator comparison experiment and a qualitative visual comparison experiment. For the quantitative index comparison experiment, based on the above evaluation indicators, the quantitative measurement results of nine methods, including the method of this invention, were compared on various test datasets. Specific results are as follows: Figure 7 and Figure 8 As shown, the top three results of each evaluation index are marked in red, green and blue respectively. The experimental results show that the method of the present invention has achieved better or more competitive results on multiple test datasets and evaluation indexes, indicating that the method of the present invention can effectively improve the quality of underwater image enhancement. For the qualitative visual contrast experiment, four typical underwater degradation types—green color cast, blue color cast, haze, and low illumination—were selected. The visual enhancement results of nine methods, including the method of this invention, were compared. Specific results are as follows: Figure 9 and Figure 10 As shown, the experimental results indicate that, compared with other comparison methods, the method of the present invention can more effectively correct color deviation, reduce the impact of haze, improve the brightness of low-light areas, and maintain a more natural visual effect.

[0042] In the description of this invention, the references to "one embodiment," "some embodiments," "in this embodiment," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0043] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A visual language model-guided divide-and-conquer method for heterogeneous degradation underwater image enhancement, characterized in that, Includes the following steps: Step S1: Obtain the original underwater image to be enhanced; Step S2: Analyze the overall degradation composition of the original underwater image from a macroscopic perspective to generate degradation composition weights, and quantify the composition ratio of various degradation types in the original underwater image through the degradation composition weights. Step S3: Perceive the spatial severity distribution of each degradation type in the original underwater image from a microscopic perspective, and generate a specific degradation degree map corresponding to each degradation type; Step S4: Obtain a hybrid expert structure including multiple parallel experts for specific degradation tasks. Under the spatial guidance of the specific degradation degree map, perform image enhancement processing on the original underwater image through each of the specific degradation task experts to obtain corresponding output features. Then, use the degradation composition weights to dynamically fuse each of the output features to obtain the enhanced underwater image. Step S2 includes: Step S21: Perform adaptive average pooling and multilayer perceptron processing on the original underwater image to obtain the corresponding image feature path score vector; Step S22: Input the original underwater image and the fine-tuned and optimized degradation type prompt word embedding into the image editor and text editor of the visual language model, respectively, and calculate the cosine similarity between the image features of the original underwater image and the text prompt features of the degradation type prompt word embedding as the degradation composition score vector; Step S23: The image feature path score vector and the degradation composition score vector are weighted and fused, and then normalized by the Softmax function to generate the degradation composition weights; Step S3 includes: Step S31: Divide the original underwater image into multiple grid image blocks, and construct corresponding severity and mildness cue words for each type of degradation. Step S32: Perform cosine similarity calculation on each grid image block with each severity cue word and each minor cue word to obtain the corresponding severity matching score and minor matching score; Step S33: Subtract the severity matching score and the slightness matching score corresponding to each grid image block and transform them using the Sigmoid function to obtain the degradation score of each grid image block under the corresponding degradation type; Step S34: Assemble the degradation degree scores of all the grid image blocks under the same degradation type into a map, and generate the corresponding specific degradation degree map after Gaussian smoothing.

2. The underwater image enhancement method for heterogeneous degradation guided by a visual language model according to claim 1, characterized in that, Before performing step S21, the method further includes: Construct a prompt word embedding that includes each of the aforementioned degradation types, and fine-tune and optimize the prompt word embedding using relation ranking loss to obtain the degraded type prompt word embedding. The relation ranking loss includes primary and secondary relation loss and saliency relation loss. The primary and secondary relation loss is used to constrain the matching score corresponding to the primary degradation type to be higher than the matching score corresponding to the secondary optimization type, and the saliency relation loss is used to constrain the matching score corresponding to the saliency degradation type to be higher than the matching score corresponding to the non-saliency optimization type.

3. The underwater image enhancement method for heterogeneous degradation guided by a visual language model according to claim 1, characterized in that, In step S4, the specific degradation level map is convolved and split to obtain multiplicative space modulation parameters and additive space modulation parameters. The original underwater image is then modulated using the multiplicative space modulation parameters and the additive space modulation parameters to obtain modulation features with degradation space prior. Subsequently, the modulation features are processed sequentially through Mamba blocks and gated depth convolutional feedforward networks to obtain the output features of the specific degradation task expert.

4. The underwater image enhancement method for heterogeneous degradation guided by a visual language model according to claim 1, characterized in that, In step S4, for each specific degradation task expert, the output feature of the specific degradation task expert is multiplied by the weight element corresponding to the specific degradation task expert in the degradation composition weight to obtain a weighted output feature. Then, all the weighted output features are added element by element to obtain the fused output feature as the enhanced underwater image.

5. The underwater image enhancement method for heterogeneous degradation guided by a visual language model according to claim 1, characterized in that, The degradation types in step S2 include green color bias, blue color bias, haze, and low illumination. The degradation weights include four weight elements, which correspond to green color bias, blue color bias, haze, and low illumination, respectively. The specific degradation degree maps include four specific degradation degree maps, which correspond to green color bias, blue color bias, haze, and low illumination, respectively. The multiple parallel specific degradation task experts include green bias expert, blue bias expert, haze expert, and low illumination expert, which correspond to green color bias, blue color bias, haze, and low illumination, respectively.

6. The underwater image enhancement method for heterogeneous degradation guided by a visual language model according to claim 1, characterized in that, The visual language model-guided heterogeneous degradation divide-and-conquer underwater image enhancement method is implemented on a U-shaped structure network, which includes multiple processing blocks with identical structures. Each processing block is used to execute steps S2, S3, and S4.

7. The underwater image enhancement method for heterogeneous degradation guided by a visual language model according to claim 1, characterized in that, Step S2 is implemented through the degradation-based perceptual routing module, step S3 is implemented through the degradation degree-perceptual mapping module, and step S4 is implemented through the degradation processing and fusion module. After executing step S4, the following steps are also included: Using an underwater image dataset containing paired enhanced and reference images, a Smooth L1 loss is calculated based on the pixel values ​​of each pixel in the enhanced and reference images. An SSIM loss is calculated based on the mean, standard deviation, and covariance of the grid image patches formed with each pixel as the center in the enhanced and reference images. The weighted sum of the Smooth L1 loss and the SSIM loss is used as the total loss. End-to-end training is then performed on the overall network consisting of the degradation perception routing module, the degradation degree perception mapping module, and the degradation processing and fusion module, which are sequentially connected.

8. The underwater image enhancement method for heterogeneous degradation guided by a visual language model according to claim 7, characterized in that, The reference images in the underwater image dataset are uniformly scaled to a fixed pixel size before being input into the degradation-based perceptual routing module, and undergo random horizontal and vertical flipping for data augmentation. The weight coefficients of the Smooth L1 loss and the SSIM loss in the total loss are set to 1 and 0.3, respectively.

Images