Underwater Image Enhancement Method and System Based on Physical Priors and Target Features

Abstract

This invention discloses an underwater image enhancement method and system based on physical priors and target features, belonging to the field of image processing technology. To address the technical problem that existing underwater image enhancement methods struggle to simultaneously achieve imaging physical consistency, visual quality, and usability for downstream target recognition tasks, this invention extracts physical prior information from underwater images, including transmittance maps and background illumination maps; acquires target features from underwater images; extracts image features from underwater images through an enhancement network; and fuses and reconstructs image features, physical prior information, and target features to obtain an enhanced image; trains the enhancement network; and uses the trained enhancement network to process underwater images to obtain the enhanced image. This invention achieves high-quality visual restoration while satisfying the physical constraints of underwater imaging, and significantly improves the accuracy of underwater target recognition and the performance of related downstream tasks.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to an underwater image enhancement method and system based on physical priors and target features. Background Technology

[0002] Water covers more than 70% of the Earth's surface, serving as a vital natural environment for global life and containing abundant marine resources and ecological information. With the development of marine science research and marine engineering, automated underwater vehicles (AUVs / ROVs) have gradually become important tools for exploring the ocean world, widely used in tasks such as marine environmental monitoring, marine resource surveys, seabed equipment inspection, underwater security, and ecological protection. In these tasks, underwater optical imaging is a crucial medium for underwater robots to acquire environmental information and make autonomous perception and decision-making. Robots need to use optical images to complete key steps such as target detection, target recognition, scene analysis, and navigation and positioning; therefore, image quality directly affects their overall operational performance. However, compared to terrestrial imaging, the underwater environment has significant optical complexity. Due to the absorption and scattering of light by water at different wavelengths, optical images experience various degradation phenomena during propagation, such as color shift, reduced contrast, blurred details, and increased noise. Especially under conditions such as turbid seawater, low light in the deep sea, and the presence of suspended particles, the visual quality of images deteriorates significantly, severely interfering with subsequent underwater target identification tasks. This makes it difficult to meet practical requirements in scenarios such as underwater archaeology, marine ecological monitoring, naval military applications, and telecommunications cable inspection. Therefore, effectively enhancing degraded underwater images has become an important prerequisite and necessary means to improve the performance of underwater target identification.

[0003] Current underwater image enhancement techniques can be broadly categorized into two main types: traditional methods and deep learning-based methods. Traditional methods primarily rely on the pixel distribution of the image itself or on physical priors based on underwater imaging models to improve visual quality, while deep learning methods utilize data-driven end-to-end learning capabilities to automatically restore images.

[0004] Traditional underwater image enhancement techniques have been developed for a long time and mainly fall into two categories: traditional digital image enhancement techniques and enhancement methods based on physical priors. Traditional digital image enhancement techniques primarily improve the visual effect of images by directly transforming their brightness, contrast, color components, or frequency domain features. Typical methods include histogram equalization, adaptive contrast enhancement (CLAHE), gamma correction, sharpening filtering, white balance correction, and Retinex-like algorithms. These methods are computationally simple and easy to implement, and can improve image clarity and contrast to a certain extent in some mildly degraded underwater scenes. Enhancement methods based on physical priors further combine underwater imaging models and image degradation mechanisms for restoration. Typical underwater imaging models usually attribute image degradation to the absorption and scattering of light as it propagates in water. Representative works include dark channel priors, saliency priors, red channel priors, and underwater dark channel priors. These methods utilize physical model constraints to infer a clear image by recovering the attenuation process of light by water. For example, they correct for illumination attenuation by estimating transmittance maps and correct color shifts by inferring background light.

[0005] In recent years, with the development of deep learning, underwater image enhancement methods based on convolutional neural networks, generative adversarial networks, and Transformer architectures have rapidly emerged. These methods utilize massive amounts of data for training, automatically establishing a mapping from degraded underwater images to clear underwater images through end-to-end learning. The advantages of these methods are mainly reflected in three aspects: First, deep networks possess powerful feature representation capabilities, automatically learning functions such as color restoration, detail enhancement, and structural completion, without the need for manually designing complex priors; second, deep models can be trained on large-scale data to learn and handle different types of underwater optical degradation, improving adaptability in various scenarios; and third, some methods, employing attention mechanisms, residual structures, and generative adversarial training strategies, demonstrate outstanding performance in maintaining naturalness and clarity.

[0006] A representative method with a similar technical approach to this solution is HUPE: Heuristic Underwater Perceptual Enhancement with Semantic Collaborative Learning. This method constructs a bidirectional mapping between underwater and clear images based on a reversible network. It improves visual quality through a forward enhancement process and uses inverse reconstruction constraints to suppress artifacts and information loss. Simultaneously, HUPE introduces heuristic underwater priors into the reversible framework to enhance the model's adaptability to diverse underwater environments; and designs a semantic collaborative learning module to narrow the gap between the visual appearance of the image and high-level semantic features, improving the semantic consistency of the enhancement results. Although HUPE balances visual quality and semantic preservation in its structural design, its core objective remains primarily focused on general underwater visual enhancement, with a relatively weak dependence on performance improvements for downstream specific tasks (such as underwater target recognition). Furthermore, reversible networks struggle to explicitly model physical degradation factors in the real underwater imaging process, resulting in limitations in the physical consistency and task adaptability of the enhancement results.

[0007] Traditional digital image enhancement techniques fail to consider the complex physical mechanisms of underwater optical degradation, relying solely on pixel distribution or simple statistics. This often results in a failure to recover structural information damaged by scattering and absorption, leading to color distortion, over-enhancement, or insufficient detail. While physics-based enhancement methods offer advantages in improving color reproduction and restoring illumination consistency, their prior assumptions often fail to hold true in complex scenarios (such as strong planktonic objects, localized illumination variations, or non-uniform scattering), leading to inaccurate estimations and impacting restoration results. Furthermore, these methods typically require manual setting of multiple parameters, making them highly dependent on environmental conditions and lacking generalization ability. Deep learning-based enhancement methods possess strong modeling capabilities and have achieved significant improvements in image visual quality. However, they are highly dependent on data distribution, lack physical interpretability, and thus have limited enhancement stability and generalization ability under extreme lighting conditions, complex suspended particles, or different imaging devices.

[0008] In addition, most studies still focus on improving subjective visual effects, with optimization metrics mainly concentrated on color naturalness, contrast, and sharpness, while paying insufficient attention to the actual needs of downstream tasks such as underwater target recognition. In some enhancement results, although the visual effect is improved, it may destroy the structural features or local details of the target, thereby affecting the target recognition performance.

[0009] Therefore, how to balance physical consistency, visual quality, and usability for downstream recognition tasks in underwater image enhancement methods remains a technical problem that needs to be solved. Summary of the Invention

[0010] The purpose of this invention is to propose an underwater image enhancement method and system based on physical priors and target features, in order to solve the technical problem that existing underwater image enhancement methods cannot simultaneously take into account imaging physical consistency, visual quality and the usability of downstream target recognition tasks. The goal is to achieve high-quality visual restoration while meeting the physical constraints of underwater imaging, and to significantly improve the accuracy of underwater target recognition and the performance of related downstream tasks.

[0011] To achieve the above objectives, the present invention adopts the following technical solution.

[0012] An underwater image enhancement method based on physical priors and target features includes the following steps:

[0013] Physical prior information, including transmittance maps and background light maps, is extracted from underwater images.

[0014] Target features are obtained from underwater images;

[0015] Image features of underwater images are extracted by an augmentation network. The image features, physical prior information and target features are jointly encoded and downsampled to obtain encoded features. The encoded features are then decoded and upsampled, and the image is reconstructed to obtain the augmented image.

[0016] The enhancement network is trained, and the trained enhancement network is used to process underwater images to obtain enhanced images.

[0017] Furthermore, physical prior information, including transmittance maps and background light maps, is extracted based on underwater images, including:

[0018] Transmittance maps of underwater images are obtained based on the underwater dark channel prior method;

[0019] Background light map of underwater images is obtained based on statistical characteristics of underwater images.

[0020] Furthermore, target features are obtained based on underwater images, including:

[0021] Based on a vision-language multimodal model, target category recognition is performed on underwater images to generate text descriptions containing target categories;

[0022] The text description is encoded using a text encoder to obtain the target features.

[0023] Furthermore, the image features, physical prior information, and target features are jointly encoded and downsampled, including:

[0024] Image features are input into a multi-layer cascaded encoder for encoding, serving as the input features for the first encoder layer. In each encoder layer, the input features, physical prior information, and target features are fused to obtain the encoded output features.

[0025] Spatially downsample the encoded features to generate the input features for the next layer encoder;

[0026] The features after multi-layer encoding and downsampling are input into the bottleneck module and fused with physical prior information and target features to obtain bottleneck encoded features.

[0027] Furthermore, the encoded features are decoded and upsampled, including:

[0028] The bottleneck coding features are spatially upsampled, and the upsampled features are input into the third-layer decoder for decoding.

[0029] Before subsequent decoders decode, the output features of the previous layer are spatially upsampled, and the upsampled features are concatenated and fused with the encoded output features of the corresponding encoder layer to generate fused features.

[0030] The fused features are input into the decoder of the corresponding layer to generate the decoded output features of that layer.

[0031] Furthermore, the encoder performs encoding, and the decoder performs decoding, including:

[0032] Normalize the input features;

[0033] The normalized input features are input into the physics guidance module and combined with the transmittance map and background light map to generate physics prior enhancement features;

[0034] The physical prior enhancement features are input into the feature enhancement module based on the state space model to generate state space enhancement features.

[0035] The state space enhancement features are input into the target feature injection module to inject the target features and output semantic enhancement features.

[0036] The output enhanced features are normalized again and added to the input features through a residual connection to obtain the final enhanced features.

[0037] Furthermore, the normalized input features are input into the physics guidance module, and combined with the transmittance map and background light map to generate physical prior enhancement features, including:

[0038] Map the transmittance map and background light map to the same spatial dimension as the input features;

[0039] Using the input features as queries, the features corresponding to the transmittance are used as keys and values ​​to perform the first cross-attention process;

[0040] Using the input features as queries, and the background light corresponding features as keys and values, a second cross-attention process is performed.

[0041] Channel-level fusion is performed on the outputs of the two cross-attention operations to obtain physical prior enhancement features.

[0042] Furthermore, the physical prior enhancement features are input into the feature enhancement module based on the state-space model to generate state-space enhanced features, including:

[0043] Normalize the input features;

[0044] In the main branch, after performing linear transformation, convolution and nonlinear activation on the normalized features, global context modeling is achieved through two-dimensional selective scanning, and then normalization is performed.

[0045] In the gated branch, the normalized features are subjected to linear transformation and nonlinear activation to generate a gated signal;

[0046] The main branch output and the gated signal are fused element by element, and the features are enhanced by linear transformation and normalization of the output state space.

[0047] Furthermore, the state space enhancement features are input into the target feature injection module to inject target features and output semantic enhancement features, including:

[0048] Use target features as keys and values, and state space augmentation features as queries;

[0049] The attention mapping weights are obtained by cross-calculating the query and the key.

[0050] The attention map weights and values ​​are weighted and combined to generate semantically constrained features;

[0051] The semantic constraint features are concatenated with the original state space enhancement features to obtain the final semantic enhancement features.

[0052] Further, the augmentation network is trained, including:

[0053] The underwater image, the corresponding clear reference image, and the target category information of the image are used as training data to iteratively train the augmentation network.

[0054] Calculate pixel reconstruction loss based on the generated enhanced image and the reference sharp image;

[0055] The physical consistency loss is calculated based on the error between the reconstructed degraded image and the input underwater image.

[0056] The recognition result is obtained based on the generated enhanced image, and the target recognition loss is calculated;

[0057] The total loss is obtained by combining the three types of losses according to their weights.

[0058] Backpropagation is performed based on the total loss to update and enhance the network weights, thus completing the training.

[0059] An underwater image enhancement system based on physical priors and target features includes:

[0060] The physical prior extraction module is used to extract physical prior information based on underwater images, including transmittance maps and background light maps;

[0061] The target feature extraction module is used to obtain target features based on underwater images;

[0062] The enhancement network module is used to extract image features from underwater images through the enhancement network, and jointly encode and downsample the image features, physical prior information and target features to obtain encoded features. Then, the encoded features are decoded and upsampled, and the image is reconstructed to obtain the enhanced image.

[0063] The network training module is used to train the augmentation network and then use the trained augmentation network to process underwater images to obtain enhanced images.

[0064] The present invention has achieved the following technical effects.

[0065] 1. This invention constructs a physically consistent underwater degradation prior modeling mechanism based on an underwater optical imaging model. It employs UDCP transmittance estimation and a background light modeling method based on statistical experience to recover pixel-level light attenuation patterns and background scattering components from a single underwater image. The degradation prior is then embedded into the enhancement network in a learnable manner, imposing light transmission theory constraints on the image enhancement process. This achieves an explicit physical description of the underwater imaging degradation process. While avoiding color shift correction errors, spurious enhancements, and structural damage caused by relying entirely on data-driven models, this invention achieves a balance between physical consistency and network interpretability, and improves the stability and generalization ability of the enhancement model under different water environments and imaging conditions.

[0066] 2. This invention constructs a semantic-driven feature extraction mechanism for target recognition tasks, introduces a large visual language model, and generates semantic descriptions of target categories in images based on prompts. These descriptions are then mapped to a unified semantic feature space via a text encoder, allowing the semantic features to participate as explicit priors in the image feature enhancement process. This endows the enhancement network with target perception capabilities, improving the overall visual quality of underwater images while strengthening the structural expression of key target regions. It also reduces the destruction of recognition-related features during the enhancement process and significantly improves the effectiveness, usability, and recognition performance of the enhancement results in downstream target recognition tasks.

[0067] 3. This invention constructs a three-stage feature enhancement framework that combines physical-guided global spatial modeling with semantic injection. It sequentially sets up a physical-guided feature enhancement module, a global feature enhancement module based on a state-space model, and a target feature injection module. This allows the network to first perform degradation compensation based on underwater imaging physical priors during feature processing, then use SS2D to model long-range spatial dependencies, and finally fuse high-level target features. This results in a feature representation that combines physical consistency, global structural expressiveness, and target semantic sensitivity. The enhancement process achieves a progressive modeling from physical compensation to structural enhancement and then to semantic reinforcement. This improves the stability, global consistency, and task adaptability of the enhancement results when dealing with complex degradation factors such as color cast, water mist, and blur, and avoids the local optima problem caused by a single enhancement strategy.

[0068] 4. This invention constructs a cross-modal interactive attention fusion mechanism based on multi-source priors. In the physical guidance stage, transmittance maps and background light maps are used as key-value pairs to introduce cross-attention, achieving adaptive compensation for pixel-level light attenuation and effective correction of background scattering components. In the semantic injection stage, textual semantic features are further used as key-value pairs, and the target semantic information is explicitly fused into the image features through cross-modal cross-attention. This enables the enhancement model to simultaneously possess the dual perception capabilities of physical degradation phenomena and key task targets, improves the fine control level of color restoration, contrast enhancement, and target morphology enhancement, and significantly improves the overall visual quality of the enhancement results and the feature separability in downstream target recognition tasks.

[0069] 5. This invention introduces a task-friendly loss function optimized for target recognition accuracy. The enhanced underwater image is input into a pre-trained target recognition network, and a task-driven loss term complementary to the visually reconstructed target is constructed based on the recognition accuracy. This allows the enhancement model to jointly optimize visual quality and target recognition performance during training, thereby forming a recognition-driven image enhancement mechanism. While generating clear and natural images, it retains and enhances the texture, edge, and structural features related to recognition, avoiding enhancement results that only improve visual appearance but are detrimental to recognition. This significantly improves the practical value of enhanced images in real-world downstream target recognition scenarios.

[0070] 6. This invention constructs a unified optimization framework for a hybrid loss function composed of pixel-level reconstruction loss, physical consistency degradation reconstruction loss, and target recognition task-driven loss. During the same training process, it collaboratively constrains and optimizes the low-level pixel consistency of the image, the consistency of the underwater imaging physical model, and the high-level semantic and task consistency. This achieves an optimal balance between perceptual visual quality, physical interpretability, and task recognizableness, enabling the augmentation network to obtain underwater image enhancement results that are more natural, reliable, and suitable for downstream tasks. Attached Figure Description

[0071] Figure 1 This is a flowchart of an underwater image enhancement method based on physical priors and target features.

[0072] Figure 2 This is a structural diagram of the physical prior extraction module.

[0073] Figure 3 This is a structural diagram of the target feature extraction module.

[0074] Figure 4 To enhance the network structure diagram.

[0075] Figure 5 This is a structural diagram of the physical-state feature enhancement module.

[0076] Figure 6 This is a structural diagram of the physical boot module.

[0077] Figure 7 This is a structural diagram of the feature enhancement module based on the state-space model.

[0078] Figure 8 This is a structural diagram of the target feature injection module.

[0079] Figure 9 This is a framework diagram of an underwater image enhancement system based on physical priors and target features. Detailed Implementation

[0080] To make the various technical features, advantages, or effects of the present invention more apparent and understandable, detailed descriptions are provided below through embodiments.

[0081] One embodiment of the present invention provides an underwater image enhancement method based on physical priors and target features, such as... Figure 1 As shown,

[0082] Step S1: Extract physical prior information based on underwater images, including transmittance maps and background light maps;

[0083] Step S2: Obtain target features based on underwater images;

[0084] Step S3: Extract image features from underwater images through an augmentation network, and jointly encode and downsample the image features, physical prior information, and target features to obtain encoded features. Then, decode and upsample the encoded features and reconstruct the image to obtain the augmented image.

[0085] Step S4: Train the augmentation network and use the trained augmentation network to process the underwater image to obtain an enhanced image.

[0086] In an optional embodiment of the present invention, step S1 may include:

[0087] Step S11: Obtain the transmittance map of the underwater image based on the underwater dark channel prior method;

[0088] Step S12: Obtain the background light map of the underwater image based on the statistical characteristics of the underwater image.

[0089] In this embodiment, the underwater optical imaging process is affected by both absorption and scattering. Its degradation mechanism can usually be described by an underwater imaging model. This model can effectively characterize typical degradation phenomena in underwater images, such as color cast, low contrast, blurring, and uneven brightness. It is the theoretical basis for underwater image enhancement techniques with physical consistency. The model is expressed as follows:

[0090]

[0091] in, Images of underwater degradation observed; The true radiance of the target scene under non-degradation conditions, i.e., a clear image; pixel position Transmittance reflects the degree of energy attenuation caused by absorption and scattering when light propagates in water. The background light is represented by the uniform scattering component contributed by the water medium.

[0092] Based on the theory of the aforementioned underwater imaging model, this embodiment estimates the transmittance map and background light map (e.g., ...) from a single degraded underwater image using a physical prior extraction module consisting of a transmittance map estimator and a background light extractor. Figure 2 As shown, this serves as interpretable physical prior information, providing a structured description of degradation for enhancing the network.

[0093] Step S11 involves obtaining the transmittance map of the underwater image based on the underwater dark channel prior method, including:

[0094] In this embodiment, the transmittance map estimator adopts the Underwater Dark Channel Prior (UDCP) method. This method is designed for the spectral characteristics of underwater environments, where red light is severely attenuated and blue and green bands dominate. It constructs dark channel features using only the green and blue channels, and then estimates pixel transmittance through minimum filtering and normalization operations. This method can effectively avoid the estimation bias caused by the lack of red channel signals in traditional DCP in underwater scenes.

[0095] Let the input underwater image be The transmittance map estimator based on UDCP processes the following steps:

[0096] Step S111, Dark Channel Calculation: Minimize the G and B channels within a local window to obtain the underwater dark channel map:

[0097]

[0098] in, For a local window, the possible values ​​are... ;

[0099] Step S112, Dark Channel Background Light Estimation: Select a small subset of pixels with the highest brightness from the dark channel and estimate the background light at the corresponding positions in the original image. ;

[0100] Step S113, Estimate Transmittance Map: Estimate underwater transmittance based on dark channel priors:

[0101]

[0102] in, This is a diagram of an underwater tunnel.

[0103] Step S12 involves obtaining the background light map of the underwater image based on its statistical characteristics, including:

[0104] In this embodiment, the background light extractor employs an empirical formula method based on the statistical characteristics of underwater images. This method establishes an empirical mapping relationship between the mean, standard deviation, and median of pixel intensity and background light intensity by performing brightness statistical analysis on a large number of underwater image samples. Estimation models are constructed for the red, green, and blue channels respectively to accommodate the different attenuation levels of light propagation in water at different wavelengths. The background light calculated through the empirical model can more accurately reflect the scattering contribution of the water medium, providing a reliable physical reference for degradation reconstruction and image enhancement.

[0105] The processing steps for the background light extractor are as follows:

[0106] Step S121, Blue and Green Channel Background Light Estimation: For the blue and green channels, considering their long underwater propagation distance and significant scattering effects, the ambient light is modeled as the mean of the pixel distribution of that channel. with standard deviation A linear combination of is expressed as:

[0107]

[0108]

[0109] in, This represents the blue channel background light. Indicates the background light of the green channel;

[0110] Step S122, Red channel background light estimation: Since red light attenuates most severely in water, its ambient light component is usually weak and greatly affected by outliers. Therefore, a median-based method is used. The nonlinear mapping function estimates the ambient light in the red channel:

[0111]

[0112] in, Indicates the background light of the red channel;

[0113] Step S123, according to , and Obtain the background light map of the underwater image.

[0114] In an optional embodiment of the present invention, step S2 may include:

[0115] Step S21: Based on the visual-language multimodal model, target category recognition is performed on the underwater image to generate a text description containing the target category;

[0116] Step S22: Encode the text description using a text encoder to obtain the target features.

[0117] In this embodiment, to improve the adaptability of the enhanced results to downstream underwater target recognition tasks, a target feature extraction module consisting of a vision-language multimodal large model and a text encoder is used to automatically identify potential target categories from the input image and generate corresponding semantic features (such as...). Figure 3 As shown, this method can overcome the shortcomings of traditional underwater image enhancement methods that only focus on visual visibility and lack perception of the semantic structure of the target, thereby achieving a synergistic improvement in visual quality and target recognition performance.

[0118] Step S21: Based on the vision-language multimodal model, target category recognition is performed on the underwater image, and a text description containing the target category is generated, including:

[0119] In this embodiment, the visual-language multimodal large model adopts a multimodal large model with image understanding capabilities, such as Qwen-VL, BLIP2, Gemini, etc., which has powerful cross-modal feature alignment and open set recognition capabilities. It can drive the model to output the target category appearing in the image through natural language prompts.

[0120] To improve the applicability and flexibility of target extraction, this invention sets up two types of prompting strategies:

[0121] General target description prompts. Examples include: "What targets are in the image?", "List the object categories in the image?", and "What identifiable objects are in the image?". These prompts are used to identify all potential targets in a scene and are suitable for complex environments or recognition scenarios where categories are not limited.

[0122] Specific category cues. Based on key targets in the underwater scene, more targeted cues can be designed, such as: "List the types of fish in the image," "What corals are in the image," "Are there starfish, shellfish, or artificial structures?" This strategy can enhance the salience of small targets or specific categories in the semantic space, increasing the attention of subsequent enhancement modules to key targets.

[0123] Based on the above prompts, the visual-language model generates a natural language text description that includes the target category.

[0124] Step S22, using a text encoder to encode the text description to obtain target features, including:

[0125] In this embodiment, the text description is encoded using a CLIP model text encoder, mapping the target information to a semantic feature vector space to obtain structured target features. These target features can be cross-modal aligned with image features to guide the enhancement network in preserving and strengthening structural information relevant to the target recognition task.

[0126] In an optional embodiment of the present invention, in step S3:

[0127] Build as Figure 4 The illustrated enhancement network employs an encoder-decoder architecture to process input underwater degraded images. This is the input used to extract enhanced features.

[0128] In an optional embodiment of the present invention, step S3 extracts image features from the underwater image using an augmentation network, including:

[0129] Step S31: Extract image features using the shallow feature extraction module. This module consists of several residual blocks, which can effectively capture underlying texture and color information.

[0130] In an optional embodiment of the present invention, step S3 involves jointly encoding and downsampling image features, physical prior information, and target features, including:

[0131] Step S32: The image features are input into a multi-layer cascaded encoder for encoding, serving as the input features for the first encoder layer; in each encoder layer, the input features, physical prior information, and target features are fused to obtain the encoded output features;

[0132] Step S33: Spatial downsampling is performed on the encoded features to generate the input features of the next layer encoder;

[0133] Step S34: Input the features after multi-layer encoding and downsampling into the bottleneck module, and fuse and enhance them with physical prior information and target features to obtain bottleneck encoded features.

[0134] In this embodiment, image features are sequentially fed into a three-layer cascaded encoder. Each encoder layer contains N physical-state feature enhancement modules used to fuse physical prior information, target features, and image features. After feature enhancement is completed at each encoder layer, the output features are spatially downsampled, reduced to half their original size, enabling the next encoder layer to extract high-dimensional semantic information within a larger receptive field. Specifically, image features... The output features are obtained after inputting into the first layer encoder. The input features of the second-layer encoder are obtained after downsampling. The output features are obtained after passing through three encoder layers. After downsampling, the result is obtained The input bottleneck module, similar in structure to the single encoder described above, also consists of N physical-state feature enhancement modules. It is used to perform physical consistency enhancement, target-guided feature fusion, and global space modeling in the deepest semantic space, generating bottleneck encoded features. .

[0135] In an optional embodiment of the present invention, step S3 involves decoding and upsampling the encoded features, including:

[0136] Step S35: Spatial upsampling of the bottleneck coding features, and inputting the upsampled features into the third-layer decoder for decoding;

[0137] Step S36: Before decoding each subsequent layer of decoder, the output features of the previous layer are spatially upsampled, and the upsampled features are concatenated and fused with the encoded output features of the corresponding encoder layer to generate fused features.

[0138] Step S37: Input the fused features into the decoder of the corresponding layer to generate the decoded output features of that layer.

[0139] In this embodiment, bottleneck coding features are used. The input is a three-layer cascaded decoder, symmetrical to the encoder. Each decoder layer contains N physical-state feature enhancement modules to compensate for color shift, contrast loss, and detail loss caused by underwater degradation. Before being input into the decoder, the feature is first spatially upsampled. Then, the upsampled feature is concatenated with the output of the encoder in the same layer, and fused through convolution. The fused feature is then used as the input to the decoder. Specifically, the feature is first... Upsampling features are obtained after upsampling. Then, the output features of the third layer encoder are compared. By performing convolution and splicing, fused features are obtained. The fused features are then input into the third-layer decoder for processing. The output features of the third-layer decoder are... Upsampling features are obtained after upsampling. In relation to the output features of the second layer encoder By performing convolution and splicing, fused features are obtained. The data is then fed into the second-layer decoder for processing. This process continues, with each of the three layers of decoding reconstructing the spatial structure of the features, fully reconstructing the spatial structure and integrating the target features, physical prior information, and other relevant data. Finally, the first-layer decoder outputs the features. .

[0140] In an optional embodiment of the present invention, image reconstruction includes:

[0141] feature The input feature reconstruction module, consisting of several residual blocks, is used to recover image details and texture information, generating an enhanced image. This enables the restoration of physical consistency and task-friendly enhancement of underwater images.

[0142] In an optional embodiment of the present invention, step S3, in which the encoder performs encoding and the decoder performs decoding, includes:

[0143] Normalize the input features;

[0144] The normalized input features are input into the physics guidance module and combined with the transmittance map and background light map to generate physics prior enhancement features;

[0145] The physical prior enhancement features are input into the feature enhancement module based on the state space model to generate state space enhancement features.

[0146] The state space enhancement features are input into the target feature injection module to inject the target features and output semantic enhancement features.

[0147] The output enhanced features are normalized again and added to the input features through a residual connection to obtain the final enhanced features.

[0148] In this embodiment, the physical-state feature enhancement module included in the decoder and encoder is used to sequentially introduce underwater physical prior information, image features, and target features during feature extraction and reconstruction to establish a physically consistent, semantically friendly image enhancement feature representation with long-range dependency characterization capabilities. Its structure is as follows: Figure 5 As shown, the input features are first normalized to eliminate differences in feature distribution across different network depths, improving the stability of subsequent information fusion. The normalized features first enter the physical guidance module. This module takes both the transmittance map and the background light map as input to form physical prior enhanced features. Next, the physically enhanced features are fed into a feature enhancement module based on a state-space model to generate state-space enhanced features. Subsequently, the state-space enhanced features enter the target feature injection module, outputting semantically enhanced features to further improve the adaptability of the enhancement results to the recognition task. Finally, after the three-stage serial processing of physical guidance, state-space enhancement, and target feature injection, the module normalizes the output features again and adds them to the module's input features through residual connections to obtain the final enhanced features. This maintains feature consistency, improves training stability, and promotes the effective flow of deep features. The output features not only contain an explicit description of the physical degradation mechanism but also possess global structure modeling capabilities and target semantic sensitivity, providing high-quality enhanced feature representation for subsequent encoder or decoder processing.

[0149] In an optional embodiment of the present invention, the normalized input features are input into the physical guidance module, and physical prior enhancement features are generated by combining the transmittance map and the background light map, including:

[0150] Map the transmittance map and background light map to the same spatial dimension as the input features;

[0151] Using the input features as queries, the features corresponding to the transmittance are used as keys and values ​​to perform the first cross-attention process;

[0152] Using the input features as queries, and the background light corresponding features as keys and values, a second cross-attention process is performed.

[0153] Channel-level fusion is performed on the outputs of the two cross-attention operations to obtain physical prior enhancement features.

[0154] In this embodiment, the structure of the physical guidance module is as follows: Figure 6 As shown, the physical guidance module in the first-layer encoder is used as an example for illustration. In this module, the transmittance map... With background light map The data are fed into an adaptive module, where they are mapped to the input features through lightweight convolutions and resize operations. Same spatial dimension. Then, the input features are used as the query. Transmittance characteristics as a bond Sum The first cross-attention step is performed, enabling the network to selectively correct color and brightness according to the attenuation level of different pixels; then, background light features are used as keys. Sum A second cross-attention step is performed to suppress the bright haze effect caused by scattering from suspended particles and to correct global color cast. The results of the two cross-attention steps are concatenated by channel and then fused by convolution to form the physical prior enhancement features. .

[0155] In an optional embodiment of the present invention, physical prior enhancement features are input into a feature enhancement module based on a state-space model to generate state-space enhancement features, including:

[0156] Normalize the input features;

[0157] In the main branch, after performing linear transformation, convolution and nonlinear activation on the normalized features, global context modeling is achieved through two-dimensional selective scanning, and then normalization is performed.

[0158] In the gated branch, the normalized features are subjected to linear transformation and nonlinear activation to generate a gated signal;

[0159] The main branch output and the gated signal are fused element by element, and the features are enhanced by linear transformation and normalization of the output state space.

[0160] In this embodiment, the structure of the feature enhancement module based on the state-space model is as follows: Figure 7 As shown, a two-dimensional selective-scan (SS2D) approach is used to perform structured recursive inference on features in the spatial domain, achieving efficient modeling of long-range dependencies and global structures. Compared to attention mechanisms, SS2D can capture the correlation of large-scale continuous regions with linear computational complexity, making it particularly suitable for compensating for structural ambiguity caused by out-of-domain scattering and the loss of high-frequency information in deep-water regions. The features output by SS2D are more balanced in both semantic and structural aspects than the original physically enhanced features. The feature enhancement module based on the state-space model adopts a gating structure. Given input features, normalization (Norm) is first performed. Subsequently, the features are divided into two parallel branches. In the main scanning branch, the features are sequentially processed through a linear layer, a convolutional layer, and an activation function (SiLU) to extract deep features, and global context modeling is performed using SS2D. Then, the scanned features are further refined using Norm. Simultaneously, the gating branch generates a gating signal through a linear layer and an activation function. Finally, the two branches are fused through element-wise multiplication and processed by a final linear layer and normalization.

[0161] In an optional embodiment of the present invention, state space enhancement features are input into a target feature injection module to inject target features and output semantic enhancement features, including:

[0162] Use target features as keys and values, and state space augmentation features as queries;

[0163] The attention mapping weights are obtained by cross-calculating the query and the key.

[0164] The attention map weights and values ​​are weighted and combined to generate semantically constrained features;

[0165] The semantic constraint features are concatenated with the original state space enhancement features to obtain the final semantic enhancement features.

[0166] In this embodiment, the structure of the target feature injection module is as follows: Figure 8 As shown, the target feature injection module employs a cross-modal cross-attention mechanism, using the target features generated by the text encoder as keys K and values ​​V, and the state space augmentation features as queries Q. Semantic constraints are injected into the image feature space through attention mapping. The specific processing flow includes: first, multiplying the state space augmentation features (as queries) with the target features (as keys) to calculate the correlation between the query and the key; then, performing softmax normalization on the correlation results to obtain attention weights; next, weighted summing of the attention weights and target feature values ​​to generate semantic constraint features; finally, concatenating and fusing the generated semantic constraint features with the original state space augmentation features, obtaining the final semantic augmentation features through convolution or linear mapping. This process enables the network to automatically enhance the texture, contour, and structural information related to the target object, while avoiding the destruction of key target information by general augmentation operations, thereby improving the contribution of the augmented image to downstream intelligent tasks.

[0167] In an optional embodiment of the present invention, training the augmentation network in step S4 includes:

[0168] Step S41: Use the underwater image, the corresponding reference clear image, and the target category information of the image as training data to iteratively train the augmentation network;

[0169] Step S42: Calculate pixel reconstruction loss based on the generated enhanced image and the reference sharp image;

[0170] Step S43: Calculate the physical consistency loss based on the error between the reconstructed degraded image and the input underwater image;

[0171] Step S44: Obtain the recognition result based on the generated enhanced image and calculate the target recognition loss;

[0172] Step S45: Combine the above three types of losses according to their weights to obtain the total loss;

[0173] Step S46: Perform backpropagation based on the total loss to update and enhance the network weights, thus completing the training.

[0174] In this embodiment, to achieve a balance between enhanced quality, physical consistency, and target identifiability, the present invention designs a hybrid loss function consisting of three parts to optimize the enhancement network. The overall loss can be expressed as a weighted combination of the three types of losses:

[0175]

[0176] in, , and The weights for each loss term are used to adjust the relative importance of the three.

[0177] Introducing reconstruction loss at the pixel level enhances the image output by the network. To approximate the sharp reference image as closely as possible in color, brightness, and detail. The reconstruction loss is more sensitive to edges and textures and has stable gradients, which can effectively help the network recover the details lost in underwater degradation and is a core component to ensure basic visual quality.

[0178]

[0179] in, This indicates the losses incurred during reconstruction.

[0180] To enhance the physical consistency of the network, a physically guided degradation image reconstruction loss, termed physical consistency loss, is incorporated into the underwater imaging model. Specifically, in the enhanced image output by the network, the transmittance map obtained through the aforementioned transmittance map estimator and background light estimator of this invention... With background light map Substituting this into an underwater imaging model, degraded images can be reconstructed. By minimizing the reconstructed degraded image... With input underwater images To mitigate the errors, the network is explicitly constrained to operate within a physically reasonable solution space, thereby avoiding over-enhancement, color shifts, or artifacts that do not conform to the laws of light transmission, and improving the interpretability and robustness of the enhancement results.

[0181]

[0182]

[0183] in, This represents the loss of physical consistency.

[0184] To ensure higher recognizability of the enhanced image in downstream target recognition tasks, this invention introduces a target recognition loss term. Specifically, the enhanced image... Input the data into a pre-trained target recognition network to obtain recognition results. Based on the relationship between the predicted category and the true target category in the image, calculate the true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN), where:

[0185] TP: The number of samples that were predicted as positive and were actually positive.

[0186] TN: The number of samples that were predicted as negative and were actually negative;

[0187] FP: The number of samples predicted as positive but actually negative;

[0188] FN: The number of samples predicted as negative but actually positive.

[0189] Based on this, object recognition networks enhance images. The recognition accuracy is defined as:

[0190]

[0191] Furthermore, the target recognition loss is defined as a complementary form of the recognition accuracy:

[0192]

[0193] in, This represents the target recognition loss.

[0194] In this embodiment, the training process for the augmented network is as follows:

[0195] First, the degraded underwater image I, the corresponding sharp reference image J, and the target category information in the images are input. The weights θ of the augmentation network F are initialized, as are the transmittance map estimator T, the atmospheric light extractor A, the natural language cue P, and the fixed-weight visual-language multimodal large model VLM and CLIP text encoder. Set the number of training epochs;

[0196] Then, the following iterative process is performed sequentially according to the training rounds:

[0197] (1) For the input underwater image I, the transmittance map t(x) is calculated using the transmittance map estimator T, denoted as And using atmospheric light extractor A, the background light map A is calculated, denoted as... ;

[0198] (2) Based on the Visual-Language Multimodal Model (VLM) and Text Encoder The underwater image I and the natural language prompt P are processed to generate target features. , can be represented as ;

[0199] (3) Combine underwater image I, transmittance map t(x), background light map A, and target features Input the augmentation network F to obtain the augmented image. , represented as ;

[0200] (4) Based on the enhanced image Reference clear image Input underwater image I and target category information, and calculate pixel reconstruction loss respectively. Physical consistency loss and target recognition loss ;

[0201] (5) Weight the three parts of the loss. , and The weighted combination yields the total loss. ;

[0202] (6) Use the total loss L for backpropagation to update the weights of the augmentation network F. ;

[0203] After the loop completes, return the weights of the trained augmented network F. .

[0204] An embodiment of the present invention also provides an underwater image enhancement system based on physical priors and target features, such as... Figure 9 As shown, it includes:

[0205] The physical prior extraction module is used to extract physical prior information based on underwater images, including transmittance maps and background light maps;

[0206] The target feature extraction module is used to obtain target features based on underwater images;

[0207] The enhancement network module is used to extract image features from underwater images through the enhancement network, and jointly encode and downsample the image features, physical prior information and target features to obtain encoded features. Then, the encoded features are decoded and upsampled, and the image is reconstructed to obtain the enhanced image.

[0208] The network training module is used to train the augmentation network and then use the trained augmentation network to process underwater images to obtain enhanced images.

[0209] In an optional embodiment of the present invention, the physical prior extraction module is further configured to:

[0210] Transmittance maps of underwater images are obtained based on the underwater dark channel prior method;

[0211] Background light map of underwater images is obtained based on statistical characteristics of underwater images.

[0212] In an optional embodiment of the present invention, the target feature extraction module is further configured to:

[0213] Based on a vision-language multimodal model, target category recognition is performed on underwater images to generate text descriptions containing target categories;

[0214] The text description is encoded using a text encoder to obtain the target features.

[0215] In an optional embodiment of the present invention, the enhanced network module is further configured to:

[0216] Image features are input into a multi-layer cascaded encoder for encoding, serving as the input features for the first encoder layer. In each encoder layer, the input features, physical prior information, and target features are fused to obtain the encoded output features.

[0217] Spatially downsample the encoded features to generate the input features for the next layer encoder;

[0218] The features after multi-layer encoding and downsampling are input into the bottleneck module and fused with physical prior information and target features to obtain bottleneck encoded features.

[0219] In an optional embodiment of the present invention, the enhanced network module is further configured to:

[0220] The bottleneck coding features are spatially upsampled, and the upsampled features are input into the third-layer decoder for decoding.

[0221] Before subsequent decoders decode, the output features of the previous layer are spatially upsampled, and the upsampled features are concatenated and fused with the encoded output features of the corresponding encoder layer to generate fused features.

[0222] The fused features are input into the decoder of the corresponding layer to generate the decoded output features of that layer.

[0223] In an optional embodiment of the present invention, the enhanced network module is further configured to:

[0224] Normalize the input features;

[0225] The normalized input features are input into the physics guidance module and combined with the transmittance map and background light map to generate physics prior enhancement features;

[0226] The physical prior enhancement features are input into the feature enhancement module based on the state space model to generate state space enhancement features.

[0227] The state space enhancement features are input into the target feature injection module to inject the target features and output semantic enhancement features.

[0228] The output enhanced features are normalized again and added to the input features through a residual connection to obtain the final enhanced features.

[0229] In an optional embodiment of the present invention, the normalized input features are input into the physical guidance module, and physical prior enhancement features are generated by combining the transmittance map and the background light map, including:

[0230] Map the transmittance map and background light map to the same spatial dimension as the input features;

[0231] Using the input features as queries, the features corresponding to the transmittance are used as keys and values ​​to perform the first cross-attention process;

[0232] Using the input features as queries, and the background light corresponding features as keys and values, a second cross-attention process is performed.

[0233] Channel-level fusion is performed on the outputs of the two cross-attention operations to obtain physical prior enhancement features.

[0234] In an optional embodiment of the present invention, physical prior enhancement features are input into a feature enhancement module based on a state-space model to generate state-space enhancement features, including:

[0235] Normalize the input features;

[0236] In the main branch, after performing linear transformation, convolution and nonlinear activation on the normalized features, global context modeling is achieved through two-dimensional selective scanning, and then normalization is performed.

[0237] In the gated branch, the normalized features are subjected to linear transformation and nonlinear activation to generate a gated signal;

[0238] The main branch output and the gated signal are fused element by element, and the features are enhanced by linear transformation and normalization of the output state space.

[0239] In an optional embodiment of the present invention, state space enhancement features are input into a target feature injection module to inject target features and output semantic enhancement features, including:

[0240] Use target features as keys and values, and state space augmentation features as queries;

[0241] The attention mapping weights are obtained by cross-calculating the query and the key.

[0242] The attention map weights and values ​​are weighted and combined to generate semantically constrained features;

[0243] The semantic constraint features are concatenated with the original state space enhancement features to obtain the final semantic enhancement features.

[0244] In an optional embodiment of the present invention, the network training module is further configured to:

[0245] The underwater image, the corresponding clear reference image, and the target category information of the image are used as training data to iteratively train the augmentation network.

[0246] Calculate pixel reconstruction loss based on the generated enhanced image and the reference sharp image;

[0247] The physical consistency loss is calculated based on the error between the reconstructed degraded image and the input underwater image.

[0248] The recognition result is obtained based on the generated enhanced image, and the target recognition loss is calculated;

[0249] The total loss is obtained by combining the three types of losses according to their weights.

[0250] Backpropagation is performed based on the total loss to update and enhance the network weights, thus completing the training.

[0251] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions made by those skilled in the art to the technical solutions of the present invention should be covered within the protection scope of the present invention, which is defined by the claims.

Claims

1. An underwater image enhancement method based on physical priors and target features, characterized in that, Includes the following steps: Physical prior information, including transmittance maps and background light maps, is extracted from underwater images. Target features are obtained from underwater images; Image features of underwater images are extracted by an augmentation network. The image features, physical prior information and target features are jointly encoded and downsampled by an encoder to obtain encoded features. The encoded features are then decoded and upsampled by a decoder, and the image is reconstructed to obtain an augmented image. The encoder performs encoding, and the decoder performs decoding, including: normalizing the input features; inputting the normalized input features into the physical guidance module and combining them with the transmittance map and background light map to generate physical prior enhancement features; inputting the physical prior enhancement features into the feature enhancement module based on the state space model to generate state space enhancement features; inputting the state space enhancement features into the target feature injection module to inject target features and output semantic enhancement features; normalizing the output enhancement features again and adding them to the input features through a residual connection to obtain the final enhancement features; The enhancement network is trained, and the trained enhancement network is used to process underwater images to obtain enhanced images.

2. The method of claim 1, wherein, Physical prior information is extracted from underwater images, including transmittance maps and background light maps, including: Transmittance maps of underwater images are obtained based on the underwater dark channel prior method; Obtain the background light map of underwater images based on statistical characteristics; Target features are obtained from underwater images, including: Based on a vision-language multimodal model, target category recognition is performed on underwater images to generate text descriptions containing target categories; The text description is encoded using a text encoder to obtain the target features.

3. The method of claim 1, wherein, Joint encoding and downsampling of image features, prior physical information, and target features include: Image features are input into a multi-layer cascaded encoder for encoding, serving as the input features for the first encoder layer. In each encoder layer, the input features, physical prior information, and target features are fused to obtain the encoded output features. Spatially downsample the encoded features to generate the input features for the next layer encoder; The features after multi-layer encoding and downsampling are input into the bottleneck module and fused with physical prior information and target features to obtain bottleneck encoded features.

4. The method as described in claim 3, characterized in that, Decoding and upsampling the encoded features includes: The bottleneck coding features are spatially upsampled, and the upsampled features are input into the third-layer decoder for decoding. Before subsequent decoders decode, the output features of the previous layer are spatially upsampled, and the upsampled features are concatenated and fused with the encoded output features of the corresponding encoder layer to generate fused features. The fused features are input into the decoder of the corresponding layer to generate the decoded output features of that layer.

5. The method as described in claim 1, characterized in that, The normalized input features are fed into the physics guidance module and combined with the transmittance map and background light map to generate physics prior enhancement features, including: Map the transmittance map and background light map to the same spatial dimension as the input features; Using the input features as queries, the features corresponding to the transmittance are used as keys and values ​​to perform the first cross-attention process; Using the input features as queries, and the background light corresponding features as keys and values, a second cross-attention process is performed. Channel-level fusion is performed on the outputs of the two cross-attention operations to obtain physical prior enhancement features.

6. The method as described in claim 1, characterized in that, The physical prior augmentation features are input into the feature augmentation module based on the state-space model to generate state-space augmentation features, including: Normalize the input features; In the main branch, after performing linear transformation, convolution and nonlinear activation on the normalized features, global context modeling is achieved through two-dimensional selective scanning, and then normalization is performed. In the gated branch, the normalized features are subjected to linear transformation and nonlinear activation to generate a gated signal; The main branch output and the gated signal are fused element by element, and the features are enhanced by linear transformation and normalization of the output state space.

7. The method as described in claim 1, characterized in that, The state-space augmentation features are input into the target feature injection module to inject target features, and the semantic augmentation features are output, including: Use target features as keys and values, and state space augmentation features as queries; The attention mapping weights are obtained by cross-calculating the query and the key. The attention map weights and values ​​are weighted and combined to generate semantically constrained features; The semantic constraint features are concatenated with the original state space enhancement features to obtain the final semantic enhancement features.

8. The method as described in claim 1, characterized in that, Training the augmented network includes: The underwater image, the corresponding clear reference image, and the target category information of the image are used as training data to iteratively train the augmentation network. Calculate pixel reconstruction loss based on the generated enhanced image and the reference sharp image; The physical consistency loss is calculated based on the error between the reconstructed degraded image and the input underwater image. The recognition result is obtained based on the generated enhanced image, and the target recognition loss is calculated; The total loss is obtained by combining the three types of losses according to their weights. Backpropagation is performed based on the total loss to update and enhance the network weights, thus completing the training.

9. An underwater image enhancement system based on physical priors and target features, characterized in that, include: The physical prior extraction module is used to extract physical prior information based on underwater images, including transmittance maps and background light maps; The target feature extraction module is used to obtain target features based on underwater images; The enhancement network module is used to extract image features from underwater images through the enhancement network. The encoder jointly encodes and downsamples the image features, physical prior information and target features to obtain coded features. The decoder then decodes and upsamples the coded features and performs image reconstruction to obtain the enhanced image. The encoder performs encoding, and the decoder performs decoding, including: normalizing the input features; inputting the normalized input features into the physical guidance module and combining them with the transmittance map and background light map to generate physical prior enhancement features; inputting the physical prior enhancement features into the feature enhancement module based on the state space model to generate state space enhancement features; inputting the state space enhancement features into the target feature injection module to inject target features and output semantic enhancement features; normalizing the output enhancement features again and adding them to the input features through a residual connection to obtain the final enhancement features; The network training module is used to train the augmentation network and then use the trained augmentation network to process underwater images to obtain enhanced images.

Images