A physically constrained underwater image enhancement method
By introducing physically constrained encoder-decoder deep learning networks and multi-scale fusion architectures into underwater image enhancement, the problems of color shift and noise in complex optical environments are solved, thereby improving image quality and algorithm stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HOHAI UNIV
- Filing Date
- 2026-03-06
- Publication Date
- 2026-06-09
AI Technical Summary
Existing underwater image enhancement techniques suffer from color shift, artifacts, and noise in complex underwater optical environments, and deep learning-based methods lack physical interpretability and are difficult to adapt to complex scene changes.
A physical constraint-based encoder-decoder deep learning network was designed. Optical physical information was incorporated into the neural network training through a multi-scale fusion architecture. Underwater physical information was used to constrain the encoding and decoding process. A physical constraint loss function and a multi-scale fusion architecture were constructed to improve image enhancement effect and algorithm stability.
It improves the robustness and model learning ability of underwater image enhancement, obtains high-quality underwater images, and overcomes the challenge of image enhancement under small sample conditions.
Smart Images
Figure CN122175809A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a physically constrained underwater image enhancement method, particularly applicable to underwater image sharpening under harsh conditions in turbid waters. It aims to eliminate physical attenuation during underwater imaging propagation, improve underwater image quality, and can be generalized for engineering applications such as flaw detection in water conservancy and hydropower projects, and underwater exploration in inland and marine environments. Background Technology
[0002] Underwater imagery serves as a data source and scientific basis for numerous engineering scenarios. For example, in the operation and maintenance of large-scale water conservancy and hydropower projects, underwater imaging acquires crucial data about underwater structures. However, due to the complex and harsh optical and physical characteristics of underwater environments (selective absorption by water and scattering by suspended particles), underwater images commonly suffer from degradation problems such as color distortion, low contrast, and blurred details. The different attenuation rates of light at different wavelengths also lead to color distortion and loss of target information, making it difficult to provide stable and reliable data for underwater exploration and other tasks.
[0003] Currently, underwater image enhancement techniques mainly rely on two categories of methods: physical model-based and learning model-based image enhancement methods. Traditional physical model methods improve underwater image quality by estimating underwater background light and transmittance. Due to their physical basis, these methods have good stability and interpretability. However, they are prone to introducing color shift, artifacts, and noise when dealing with complex underwater optical environments, and they are difficult to adapt to complex scene changes. Deep learning-based image enhancement methods improve underwater image quality through data-driven model training. However, because they do not consider the physical processes of underwater optical imaging, their interpretability is insufficient, and their effectiveness and algorithm stability need further improvement. Summary of the Invention
[0004] Purpose of the Invention: The purpose of this invention is to provide a physically constrained underwater image enhancement method. A physically constrained encoder-decoder deep learning network is designed, and optical physical information is integrated into the neural network training through a multi-scale fusion architecture. The underwater physical information is used to constrain the encoding and decoding process to solve the problem of insufficient physical interpretability of traditional deep learning algorithms, thereby improving the effect and stability of underwater image enhancement.
[0005] First, based on the physical principles of underwater optical imaging, an underwater physical information extraction module is proposed to calculate the reflected light and transmittance of the target scene. A physical information loss function with physical constraints is constructed based on the mean square error loss function to quantify the deviation between the network prediction results and the physical model. Secondly, to prevent the underwater physical information extraction module from getting trapped in local optima in its predicted reflected light and transmittance, thus failing to capture global water body physical characteristics, a multi-scale fusion architecture is proposed to fuse underwater physical information from multiple scales. Furthermore, a physically constrained encoder-decoder deep learning network is proposed, in which the underwater physical information extraction module is embedded in the encoder, and the multi-scale underwater physical information fusion result is input into the decoder to enhance the underwater image. Finally, the design incorporates physical information loss. , The total loss function includes both loss and SSIM loss. During the training phase, the network is trained using data-driven methods until convergence. During the application phase, the current original underwater image is input, and the underwater image enhancement result is output.
[0006] Technical solution: A physically constrained underwater image enhancement method, comprising the following steps: S1. Based on the physical process of underwater optical imaging, construct an underwater physical information extraction module to predict the direct reflected light intensity and transmittance of the target scene. S2. Based on the underwater physical information extraction module proposed in S1, construct the loss function for physical constraints. ; S3. Construct a multi-scale fusion architecture to integrate multi-scale underwater physical information; S4. A physical constraint encoder-decoder deep learning network is proposed, in which the underwater physical information extraction module is embedded in the encoder, and the multi-scale underwater physical information fusion result is input into the decoder to enhance the underwater image. S5, Design includes physical information loss , The total loss function, including the SSIM loss, is used to train the network using data-driven methods until convergence. S6. During the training phase, the network is trained using data-driven methods until convergence. During the application phase, the current original underwater image is input, and the underwater image enhancement result is output.
[0007] The specific contents of the underwater physical information extraction module are as follows: The physical processes underlying underwater optical imaging, i.e., the original degraded underwater image. I Clear underwater images with reflected light J transmittance t The physical relationship between the background light A and the background light A is used to construct a loss function with physical constraints. Optimize the encoder-decoder deep learning network for physical constraints to estimate scene transmittance. t and clear images of reflected light J , The input raw degraded underwater image IImage features are extracted after processing by the CBR module. The CBR module consists of three layers, the first of which is a convolutional layer, and the output is: in, For the original degraded image I exist Image information at location, For convolution kernel, i, j This refers to the vertical and horizontal migration of the convolution kernel; The second layer is the batch normalization layer, and the output is: in, and They are respectively The mean and standard deviation, It is a tiny constant.
[0008] The third layer is the ReLU activation function layer, and its output is: The feature map output by the ReLU activation function layer is reconstructed through the Resize operation to ensure resolution adaptability of the feature space; This completes the three-level calculation of the CBR module, expressed as follows: ; Each physical information extraction module contains two parallel CBR modules, which estimate the scene transmittance respectively. t and clear images of reflected light J : Loss function for constructing physical constraints : in, It represents the number of physical information extraction modules at different scales. n Number of pixels I ( x,y () represents the original input image. J ( x,y For a clear image of reflected light Location information, t ( x,y ) represents the transmittance at Location information, A As background light, J i ( x,y ) is the first i A clear image of reflected light at each scale. Location information, ti ( x,y ) is the first i Transmittance at each scale Location information.
[0009] Preferably, step S3 proposes a multi-scale fusion architecture. This architecture designs a structure with N levels of encoding modules at different scales connected in series. Each level of encoding module is connected to a physical information extraction module. The multi-scale underwater physical information is spliced together in the channel dimension to form a composite feature map, which is then processed by convolution and CBR modules to achieve N levels of different scale conditions. J ( x,y The physical information fusion result is shown in the following formula: in, For the first i The results of physical information extraction at each scale are represented by Concat (for concatenation) and Conv (for convolution).
[0010] During the decoding stage, the multi-scale physical information fusion results are input into the cascaded decoder to form a physically constrained decoder, and the underwater image enhancement results are output.
[0011] Preferably, the physical constraint encoding-decoding deep learning network proposed in step S4 specifically includes the following steps: Step S401 Encoding and Physical Constraint Application: The encoding module achieves feature extraction and spatial dimension reduction through N-level convolution operations. At the same time, an underwater physical information extraction module is used to apply multiple physical constraints based on the underwater optical imaging principle to the features in the encoding stage. Step S402 Multi-scale physical information fusion: To further capture global underwater physical information, a multi-scale fusion architecture is introduced, namely, a structure of N levels of encoding modules at different scales connected in series. Each level of encoding module is connected to the physical information extraction module, thereby forming N levels of physical information fusion under different scale conditions. J ( x,y Physical information fusion results; Step S403 Decoding stage: The multi-scale physical information fusion result is input into the cascaded decoder to form a physically constrained decoder, and the underwater image enhancement result is output.
[0012] Preferably, the loss function of the physical constraints designed in step S2 in step S5 In the training of physically constrained encoder-decoder deep learning networks, it is incorporated as part of the model training loss function, and The loss function and the SSIM function are linearly combined to obtain the total loss function. The specific formula is as follows: in and These are weight parameters, representing the proportion of each function in the overall objective function, and the loss due to physical constraints. Used to quantify the discrepancy between the enhanced image and the underwater physical model.
[0013] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the underwater image enhancement method with physical constraints as described above.
[0014] A computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the underwater image enhancement method with physical constraints as described above.
[0015] Beneficial effects: The physical constraint encoding-decoding deep learning network proposed in this invention realizes underwater image enhancement by combining physical information and data-driven deep learning. It can improve the robustness of underwater image enhancement and overcome the model learning ability of underwater image enhancement under small sample conditions, thus obtaining high-quality underwater images. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating the physical constraints of the underwater image enhancement method according to an embodiment of the present invention. Figure 2 This is a structural diagram of the underwater physical information extraction module according to an embodiment of the present invention; Figure 3 This is a structural diagram of the multi-scale fusion architecture according to an embodiment of the present invention; Figure 4 This is a diagram of the encoding-decoding deep learning network structure for physical constraints in an embodiment of the present invention. Detailed Implementation
[0017] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.
[0018] A physically constrained underwater image enhancement method, the specific process of which is as follows: Figure 1 As shown, it includes: S1. Based on the physical processes of underwater optical imaging, an underwater physical information extraction module is constructed to calculate the reflected light and transmittance of the target scene. The specific content is as follows: First, the underwater imaging model characterizes the original degraded underwater image. I Clear underwater images with reflected light J transmittancet The physical relationship between the background light A and the background light A: in The light that reaches the imaging plane is also a degraded underwater image. The direct component that reaches the imaging plane refers to a sharp image.
[0019] Based on the underwater optical model, an underwater physical information extraction module was designed, which... Figure 2 The module shown calculates the target image. and transmittance The underwater physical information extraction module consists of three CBR modules: each CBR module comprises a 3×3 convolutional kernel, batch normalization, and a ReLU activation function. Input features are first processed by the CBR module for local feature extraction and nonlinear transformation, then adaptively adjusted for spatial resolution via a resize operation, and finally fused element-wise. Each physical information extraction module contains two parallel CBR modules, which estimate the scene transmittance. t and clear images of reflected light J The specific formula is as follows: in, For the original degraded image I exist Image information at location, For convolution kernel; and Images I The mean and standard deviation after convolution. It is a tiny constant.
[0020] A loss function based on physical constraints is constructed using the underwater physical information extraction module. The mean square error is used to measure the deviation between the model's calculation results and the results obtained from optical physical principles. The specific formula is as follows: in, Let n be the number of underwater physical information neural modules, and n be the number of pixels. I(x,y) The original image is the input. J(x,y) and t(x,y) These represent the direct reflection portion and transmittance predicted by the underwater physical information extraction module, respectively, with A representing the background light.
[0021] S2. Construct a multi-scale fusion architecture, the specific content of which is as follows: Multi-scale fusion architecture, such as Figure 3 As shown, its core function is to facilitate computation at different scales and stages. Information exchange and fusion between modules are implemented to better utilize global physical information to optimize underwater image enhancement results. A three-level, cascaded coding module structure with different scales is designed, with each level of coding module connected to a physical information extraction module. The output of each physical information extraction module... First, features are extracted separately using the CBR module, preserving local details. Then, features from different levels are integrated through channel-level Concat operations. Finally, after convolution and the CBR module, the fused multi-scale underwater physical information is obtained, which can be expressed by the formula: in, For the first i The results of physical information extraction at each scale are represented by Concat (for concatenation) and Conv (for convolution).
[0022] During the decoding stage, the multi-scale physical information fusion results are input into the cascaded decoder to form a physically constrained decoder, and the underwater image enhancement results are output.
[0023] S3. Propose a physical constraint encoding-decoding deep learning network, including the following sub-steps: S301. This network uses an encoder-decoder framework and consists of three encoder-decoder layers and one convolutional layer; for example... Figure 4 The encoding module shown contains a three-level cascaded CBR sub-module. Its output is fused in the Add module through a skip connection, and then downsampled by a max pooling layer before being fed into the final CBR module. The decoder module also adopts a three-level cascaded CBR structure. It inputs the features into the Add module through a skip connection to achieve residual superposition. Then, it uses an upsampling layer to restore the spatial dimension, and finally the final CBR module completes the feature transformation.
[0024] S302. Add the underwater physical information extraction module proposed in S1 to the network, such as... Figure 4 As shown, each level of the encoding module is connected to the physical information extraction module, receiving the deep features extracted in the encoding stage as input, and outputting the predicted target image for each stage. and transmittance .
[0025] S303 incorporates a multi-scale fusion architecture, such as Figure 4 As shown, the physical information extraction results at different scales of the encoder are fused at multiple scales. The output multi-scale physical information fusion result is input into the cascaded decoder to form a physical constraint decoding and output the underwater image enhancement result.
[0026] S4. The design includes physical information loss. , The total loss function for SSIM loss is as follows: Physical information loss function With mean absolute error The loss function, a linear combination of the structural similarity loss function (SSIM loss function), serves as the loss function for optimizing model training, achieving a balance between visual effects and quantitative metrics; the mean absolute error loss function... Error is measured by averaging the absolute differences between the network-predicted image and the target image; structural similarity loss function. The structural similarity between the predicted images and the target images is measured by calculating the predicted images from the network. This measurement serves as the optimization objective to guide the network in generating higher-quality images that more closely resemble the target images. The specific formula is as follows: Where n is the number of samples. It is the true value of the i-th sample. It is the predicted value of the i-th sample; , , The table measures the differences in brightness, contrast, and structural similarity between the network-computed image and the target image. , and These are weight parameters used to adjust the influence of the three dimensions on the final similarity score; they are typically set to a value of [value missing]. ; and The values are set to 1 and 1.5 respectively, representing the proportion of each function in the overall objective function.
[0027] S5. During the training phase, the network is trained using data-driven methods until convergence. In the application phase, the current original underwater image is input, and the underwater image enhancement result is output.
[0028] It is obvious to those skilled in the art that the steps of the physically constrained underwater image enhancement method of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by the computing device. Furthermore, in some cases, the steps shown or described can be performed in a different order than presented herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.
Claims
1. A physically constrained underwater image enhancement method, characterized in that, Includes the following steps: S1. Based on the physical process of underwater optical imaging, an underwater physical information extraction module is constructed to predict the clear image and transmittance of the direct reflected light of the target scene. S2. Based on the underwater physical information extraction module proposed in S1, construct the loss function for physical constraints. ; S3. Construct a multi-scale fusion architecture to integrate multi-scale underwater physical information; S4. A physical constraint encoder-decoder deep learning network is proposed, in which the underwater physical information extraction module is embedded in the encoder, and the multi-scale underwater physical information fusion result is input into the decoder to enhance the underwater image. S5, Design includes physical information loss , Loss, SSIM loss, and total loss function; S6. During the training phase, the network is trained using data-driven methods until convergence. During the application phase, the current original underwater image is input, and the underwater image enhancement result is output.
2. The underwater image enhancement method with physical constraints according to claim 1, characterized in that, Step S1 proposes an underwater physical information extraction module. This module, based on the physical process of underwater optical imaging, achieves physical parameter prediction and feature constraint through a multi-stage collaborative architecture: the CBR module completes the local extraction and nonlinear transformation of input features, and the Resize operation maintains the adaptability of spatial resolution; the features are added element by element to form a basic feature processing link; each underwater physical information extraction module contains two parallel CBR modules, which estimate the scene transmittance respectively. t and clear images of reflected light J The specific formula is as follows: For the original degraded image I exist Image information at location, For convolution kernel, i, j This refers to the vertical and horizontal movement of the convolution kernel; and Images I The mean and standard deviation after convolution. It is a constant. Preferably, the loss function of the physical constraints proposed in step S2 The loss function is constructed based on the mean squared error (MSE) loss function, and the specific formula is as follows: in, It represents the number of physical information extraction modules at different scales. n Number of pixels I ( x,y () represents the original input image. J i ( x,y ) is the first i A clear image of reflected light at each scale. Location information, t i ( x,y ) is the first i Transmittance at each scale Location information, A Used as background light.
3. The underwater image enhancement method with physical constraints according to claim 1, characterized in that, The CBR module consists of three layers. The first layer is a convolutional layer, and the output is: in, For the original degraded image I exist Image information at location, For convolution kernel; The second layer is the batch normalization layer, and the output is: in, and They are respectively The mean and standard deviation, It is a constant; The third layer is the ReLU activation function layer, and its output is: The feature map output by the ReLU activation function layer is reconstructed through the Resize operation to ensure resolution adaptability of the feature space.
4. The underwater image enhancement method with physical constraints according to claim 1, characterized in that, Step S3 proposes a multi-scale fusion architecture. This architecture designs a structure with N levels of encoding modules at different scales connected in series. Each level of encoding module is connected to a physical information extraction module. The multi-scale underwater physical information is concatenated along the channel dimension to form a composite feature map, which is then processed by convolution and CBR modules to achieve N levels of different scale conditions. J ( x,y The physical information fusion result is shown in the following formula: in, For the first i The physical information extraction results at each scale, Concat is the concatenation calculation, and Conv is the convolution calculation; During the decoding stage, the multi-scale physical information fusion results are input into the cascaded decoder to form a physically constrained decoder, and the underwater image enhancement results are output.
5. The underwater image enhancement method with physical constraints according to claim 1, characterized in that, The physical constraint encoding-decoding deep learning network proposed in step S4 specifically includes the following steps: Step S401 Encoding and Physical Constraint Application: The encoding module achieves feature extraction and spatial dimension reduction through N-level convolution operations. At the same time, an underwater physical information extraction module is used to apply multiple physical constraints based on the underwater optical imaging principle to the features in the encoding stage. Step S402 Multi-scale physical information fusion: To further capture global underwater physical information, a multi-scale fusion architecture is introduced, namely, a structure of N levels of encoding modules at different scales connected in series. Each level of encoding module is connected to the physical information extraction module, thereby forming N levels of physical information fusion under different scale conditions. J ( x,y Physical information fusion results; Step S403 Decoding stage: The multi-scale physical information fusion result is input into the cascaded decoder to form a physically constrained decoder, and the underwater image enhancement result is output.
6. The underwater image enhancement method with physical constraints according to claim 1, characterized in that, The loss function of the physical constraints designed in step S2 in step S5 In the training of physically constrained encoder-decoder deep learning networks, it is incorporated as part of the model training loss function, and The loss function and the SSIM function are linearly combined to obtain the total loss function. The specific formula is as follows: in and These are weight parameters, representing the proportion of each function in the overall objective function, and the loss due to physical constraints. Used to quantify the discrepancy between the enhanced image and the underwater physical model.
7. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the underwater image enhancement method with physical constraints as described in any one of claims 1-6.
8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by the processor, they implement the steps of the underwater image enhancement method with physical constraints as described in any one of claims 1-6.