An underwater image enhancement method based on deep guided state space model

Abstract

This invention provides an underwater image enhancement method based on a depth-guided state-space model, comprising the following steps: acquiring an original underwater degraded image and inputting it into a convolutional stem network to obtain initial image features; inputting the original underwater degraded image into a depth estimation model to obtain a depth potential field; using the depth potential field to perform a topological-preserving rearrangement of the initial image features to obtain a feature sequence, and inputting the feature sequence into a state-space model to obtain global image features; constructing a local denoising branch to obtain local image features; fusing the initial features, global features, and local features to obtain fused features; using the fused features to generate semantic residuals, and using the semantic residuals to adaptively correct the depth potential field to obtain an optical potential field; generating physical transmission modulation weights based on the optical potential field, and using the physical transmission modulation weights to modulate the fused features; using an L1 loss function for supervised training, and after completing network training, outputting an enhanced image of the underwater degraded image.

Description

Technical Field

[0001] This invention relates to the field of underwater image processing technology, and more particularly to an underwater image enhancement method based on a depth-guided state space model. Background Technology

[0002] With the rapid development of technologies such as marine resource development, underwater exploration, and underwater robots, underwater image acquisition and analysis are of great significance in fields such as marine engineering, environmental monitoring, and underwater visual perception. However, due to the absorption and scattering of light by water, underwater images often suffer from severe color casts, reduced contrast, and loss of detail.

[0003] To address underwater image degradation, various image enhancement methods have been proposed in existing technologies, including physically-based methods, non-physically-based methods, and deep learning-based methods. While physically-based methods perform well under specific water quality conditions, their effectiveness significantly decreases under complex and non-stationary water conditions. Non-physically-based methods, although simple to implement and capable of real-time operation, are prone to color casts or artifacts. Deep learning-based methods, while showing potential in long-range dependency modeling due to their use of state-space models, are limited by their neglect of physical geometry, making it difficult to accurately simulate depth-dependent light attenuation and backscattering noise. Summary of the Invention

[0004] In response to the technical problems mentioned in the background section, this invention provides an underwater image enhancement method based on a contour-guided evolutionary state-space model. This invention deeply integrates the three-dimensional physical geometric constraints of underwater light propagation with a state-space model. It guides the sequence to evolve along the physical depth gradient through a topology-preserving contour scanning mechanism, and simultaneously introduces potential field evolution and physical transport modulation gates to explicitly simulate the Jaffe-McGlamery optical model, achieving high-quality image restoration in complex aquatic environments.

[0005] The technical means employed in this invention are as follows:

[0006] An underwater image enhancement method based on a depth-guided state-space model includes the following steps: Step 1: Obtain the original underwater degraded image and input it into a convolutional feature extraction network to obtain the initial features of the original underwater degraded image. ; Step 2: Input the original underwater degraded image into the depth estimation model Depth Anything V2 to obtain the depth potential field of the original underwater degraded image. ; Step 3: Set the initial features With the depth potential field The global features of the original underwater degradation image are obtained by inputting contour scanning and state space evolution modules. ; Step 4: Set the initial features Parallel input and parallel denoising branch are used to obtain local features of the original underwater degraded image. ; Step 5: Set the initial features The global features and the local features The images are fused to obtain the fused features of the original underwater degraded images. ; Step 6: Merge the features Inputting the data into a convolutional network for feature mapping yields semantic residuals. The semantic residual With the depth potential field Residual fusion is performed to obtain the optical potential field. ; Step 7: Utilize the optical potential field Generate physical transmission modulation weights and utilize the physical transmission modulation weights Modulate the fusion feature Obtain the final features ; Step 8: Supervised training of the network is performed using the L1 loss function. After the network training is completed, the enhanced image of the underwater degradation image is obtained using the trained network. .

[0007] Furthermore, step 3 includes the following steps: Step 31: Set the initial features Input a deep convolutional layer, output enhanced features ; Step 32: Add the enhanced features Decomposed into horizontal branch features according to the channel dimension and vertical branching features : ; Step 33: Apply the depth potential field Quantized into K levels, forming a quantized deep potential field. : ; Where K represents the number of quantification levels; Step 34: The horizontal branch feature With the quantized depth potential field Evolution is performed by inputting a transverse state-space model: ; ; ; ; in, Indicates flattening; Indicates sorting; This indicates that the sorting algorithm is a stable sorting algorithm; Indicates the index of the horizontal branches; This indicates that contour line scanning and rearrangement are being performed; This indicates a lateral state-space evolution; This represents the intermediate features of the lateral branches after evolution through the state space; This represents the inverse mapping of the horizontal branching index; This indicates that spatial domain recovery is being performed; This represents the characteristics of the lateral branches after they have been restored to the spatial domain; Step 35: The vertical branch feature With the quantized depth potential field transpose Evolution is performed by inputting a longitudinal state-space model: ; ; ; ; in, Indicates flattening; Indicates sorting; This indicates that the sorting algorithm is a stable sorting algorithm; Indicates the sorting index of the vertical branches; This indicates that contour line scanning and rearrangement are being performed; This indicates a longitudinal state-space evolution; This represents the intermediate features of the vertical branch after evolution through the state space; This represents the inverse mapping of the vertical branching index; This indicates that spatial domain recovery is being performed; This represents the features of the vertical branches after they have been restored to the spatial domain; Step 36: Fuse the lateral features according to the channel dimension. With the aforementioned longitudinal features To obtain global features : .

[0008] Furthermore, in step 4, the local features are obtained from the parallel denoising branch. The calculation method is as follows: ; in, This represents a 3×3 depth convolution operation; Indicates global average pooling; This represents the Sigmoid activation function; Indicates a gating mechanism; This indicates element-wise multiplication.

[0009] Furthermore, in step 5, the features are fused. The calculation method is as follows: ; in, This represents a feedforward neural network; This indicates the learning scaling parameter.

[0010] Furthermore, step 6 includes the following steps: Step 61, from the fusion features Extract semantic residuals : ; in, Indicates the activation function; Step 62: Convert the semantic residual With the depth potential field Residual fusion is performed to obtain the optical potential field. : .

[0011] Furthermore, step 7 includes the following steps: Step 71: Utilize the optical potential field Generate physical transmission modulation weights : ; in, This represents the Sigmoid activation function.

[0012] Step 72: Utilize the physical transmission modulation weights Modulate the fusion feature The final feature is obtained : ; in, This indicates element-wise multiplication.

[0013] Furthermore, the loss function in step 8 is the L1 loss function: ; in, Indicates the total number of pixels in the image; This refers to the enhanced image; This refers to a real image.

[0014] Compared with the prior art, the present invention has the following advantages: This invention provides an underwater image enhancement method based on a depth-guided state-space model. It integrates a depth potential field-based scanning mechanism into the state-space model, avoiding the disruption of physical continuity caused by traditional grid scanning and achieving depth-dependent physical consistency modeling. By combining potential field evolution and physical transmission modulation mechanisms, it effectively eliminates the neighborhood difference between the initial depth potential field and the actual underwater optical path. Furthermore, it achieves adaptive depth field calibration using only reconstruction loss, without any depth ground truth supervision. The fusion of parallel local denoising branches effectively suppresses far-field backscattering noise, restores the true colors of the near-field, and maintains clear texture details. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic flowchart illustrating the principle of the present invention.

[0017] Figure 2 The figures show a comparison of the enhancement effects of the present invention with other algorithms on the UIEB dataset. Among them, (a) represents the effect of the Ucolor algorithm, (b) represents the effect of the VQCNIR algorithm, (c) represents the effect of the FMambaIR algorithm, (d) represents the effect of the UDNet algorithm, (e) represents the effect of the CDF-UIE algorithm, and (f) represents the effect of the method of the present invention.

[0018] Figure 3The images show a comparison of the magnified local effects of the present invention and other algorithms on publicly available underwater datasets. In the images, (a) represents the magnified local effect of the Ucolor algorithm, (b) represents the magnified local effect of the VQCNIR algorithm, (c) represents the magnified local effect of the FMambaIR algorithm, (d) represents the magnified local effect of the UDNet algorithm, (e) represents the magnified local effect of the CDF-UIE algorithm, and (f) represents the magnified local effect of the method of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] like Figure 1 As shown, this invention provides an underwater image enhancement method based on a depth-guided state-space model, comprising the following steps: Step 1: Obtain the original underwater degraded image and input it into a convolutional feature extraction network to obtain the initial features of the original underwater degraded image. ; Step 2: Input the original underwater degraded image into the depth estimation model Depth Anything V2 to obtain the depth potential field of the original underwater degraded image. ; Step 3: Set the initial features With the depth potential field The global features of the original underwater degradation image are obtained by inputting contour scanning and state space evolution modules. Step 3 includes the following steps: Step 31: Set the initial features Input a deep convolutional layer, output enhanced features ; Step 32: Add the enhanced features Decomposed into horizontal branch features according to the channel dimension and vertical branching features : ; Step 33: Apply the depth potential field Quantized into K levels, forming a quantized deep potential field. : ; Where K represents the number of quantification levels; Step 34: The horizontal branch feature With the quantized depth potential field Evolution is performed by inputting a transverse state-space model: ; ; ; ; in, Indicates flattening; Indicates sorting; This indicates that the sorting algorithm is a stable sorting algorithm; Indicates the index of the horizontal branches; This indicates that contour line scanning and rearrangement are being performed; This indicates a lateral state-space evolution; This represents the intermediate features of the lateral branches after evolution through the state space; This represents the inverse mapping of the horizontal branching index; This indicates that spatial domain recovery is being performed; This represents the characteristics of the lateral branches after they have been restored to the spatial domain; Step 35: The vertical branch feature With the quantized depth potential field transpose Evolution is performed by inputting a longitudinal state-space model: ; ; ; ; in, Indicates flattening; Indicates sorting; This indicates that the sorting algorithm is a stable sorting algorithm; Indicates the sorting index of the vertical branches; This indicates that contour line scanning and rearrangement are being performed; This indicates a longitudinal state-space evolution; This represents the intermediate features of the vertical branch after evolution through the state space; This represents the inverse mapping of the vertical branching index; This indicates that spatial domain recovery is being performed; This represents the features of the vertical branches after they have been restored to the spatial domain; Step 36: Fuse the lateral features according to the channel dimension. With the aforementioned longitudinal features To obtain global features : .

[0022] Step 4: Set the initial features Parallel input and parallel denoising branch are used to obtain local features of the original underwater degraded image. In step 4, the local features are obtained from the parallel denoising branch. The calculation method is as follows: ; in, This represents a 3×3 depth convolution operation; Indicates global average pooling; This represents the Sigmoid activation function; Indicates a gating mechanism; This indicates element-wise multiplication.

[0023] Step 5: Set the initial features The global features and the local features The images are fused to obtain the fused features of the original underwater degraded images. In step 5, the features are fused. The calculation method is as follows: ; in, This represents a feedforward neural network; This indicates the learning scaling parameter.

[0024] Step 6: Merge the features Inputting the data into a convolutional network for feature mapping yields semantic residuals. The semantic residual With the depth potential field Residual fusion is performed to obtain the optical potential field. Step 6 includes the following steps: Step 61, from the fusion features Extract semantic residuals : ; in, Indicates the activation function; Step 62: Convert the semantic residual With the depth potential field Residual fusion is performed to obtain the optical potential field. : .

[0025] Step 7: Utilize the optical potential field Generate physical transmission modulation weights and utilize the physical transmission modulation weights Modulate the fusion feature Obtain the final features Step 7 includes the following steps: Step 71: Utilize the optical potential field Generate physical transmission modulation weights : ; in, This represents the Sigmoid activation function.

[0026] Step 72: Utilize the physical transmission modulation weights Modulate the fusion feature The final feature is obtained : ; in, This indicates element-wise multiplication.

[0027] Step 8: Supervised training of the network is performed using the L1 loss function. After the network training is completed, the enhanced image of the underwater degradation image is obtained using the trained network. The loss function in step 8 is the L1 loss function: ; in, Indicates the total number of pixels in the image; This refers to the enhanced image; This refers to a real image.

[0028] Example 1 like Figure 2 As shown, this embodiment provides a comparison of the enhancement effects of various existing algorithms on underwater scene images on the UIEB dataset. (a) The Ucolor algorithm can improve image brightness and color saturation to a certain extent, but some areas exhibit overly strong color enhancement, leading to local color shifts. (b) The VQCNIR algorithm has a certain ability to restore the overall image tone, but still suffers from insufficient detail restoration in low-light areas. (c) The FMambaIR algorithm can enhance image structural information, but some areas still exhibit slight edge transitions that are not smooth enough. (d) The UDNet algorithm improves image contrast, but still falls short in detail texture restoration. (e) The CDF-UIE algorithm has a certain optimization effect on overall brightness and color balance, but exhibits some smoothing in high-frequency texture areas. In contrast, (f) The method described in this embodiment can better restore color information and structural details in underwater images, resulting in a better overall visual effect.

[0029] like Figure 3 As shown, this embodiment provides a comparison of the local magnification enhancement effects with other algorithms on a publicly available underwater dataset. (a) The Ucolor algorithm exhibits insufficient edge sharpness and overly strong texture enhancement in the magnified local area; (b) The VQCNIR algorithm still suffers from insufficient texture representation during local detail restoration; (c) The FMambaIR algorithm shows slight edge transition discontinuities during local structure restoration; (d) The UDNet algorithm has limited ability to restore detailed textures in local areas, with some areas still exhibiting blurriness; (e) While the CDF-UIE algorithm enhances local contrast, it shows a smoothing trend in some high-frequency areas. In contrast, (f) The method described in this embodiment maintains clearer edges and texture structures in the magnified local area, demonstrating superior local detail restoration capabilities.

[0030] Example 2 This embodiment compares and analyzes the experimental results of different underwater image enhancement algorithms using three evaluation metrics: PSNR, URanker, and UCIQE. PSNR measures the ratio of the maximum possible power of the signal between the enhanced image and the reference image to the reconstruction error noise power, reflecting the overall reconstruction quality of the image. URanker is a learning-based no-reference image quality assessment method that ranks and evaluates the quality of enhanced images by simulating human visual perception, providing a more comprehensive reflection of the subjective visual effect of the enhancement results. UCIQE is a no-reference evaluation metric for underwater images that assesses the visual quality of the image by comprehensively considering factors such as chroma, saturation, and contrast. Table 1 shows that the method of this invention outperforms the compared algorithms in all three objective evaluation metrics.

[0031] This invention integrates a depth potential field-based scanning mechanism into a state-space model, achieving depth-dependent physical consistency modeling. By combining potential field evolution and physical transmission modulation mechanisms, it effectively eliminates the domain difference between the initial depth potential field and the actual underwater optical path. The fusion of parallel local denoising branches effectively suppresses far-field backscattering noise, restores the true colors of near-field objects, and maintains clear texture details. This results in significantly improved PSNR, URanker, and UCIQE metrics for the generated enhanced image, outperforming existing underwater image enhancement methods overall.

[0032] Table 1. Comparison of the results of the model of this invention with other advanced algorithms on the UIEB-V90 dataset.

[0033] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the above embodiments of the present invention, the descriptions of each embodiment have their own emphasis; parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. It should be understood that the disclosed technical content in the several embodiments provided in this application can be implemented in other ways.

[0034] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater image enhancement method based on deep guided state-space model, characterized in that, Includes the following steps: Step 1, obtaining an original underwater degraded image, inputting the original underwater degraded image into a convolution feature extraction network to obtain initial features of the original underwater degraded image ; Step 2, input the original underwater degraded image into a depth estimation model Depth Anything V2 to obtain a depth potential field of the original underwater degraded image ; Step 3, the initial features are obtained by inputting the contour scanning and state space evolution module with the deep potential field input contour scanning and state space evolution module, to obtain the global features of the original underwater degraded image ; Step 4, the initial feature Parallel input and parallel denoising branch, to obtain the local feature of the original underwater degradation image ; Step 5, fusing the initial features , the global features , and the local features to obtain fused features of the original underwater degraded image ; Step 6: Merge the features Inputting the data into a convolutional network for feature mapping yields semantic residuals. The semantic residual With the depth potential field Residual fusion is performed to obtain the optical potential field. ; Step 7: Utilize the optical potential field Generate physical transmission modulation weights and utilize the physical transmission modulation weights Modulate the fusion feature Obtain the final features ; Step 8: Supervised training of the network is performed using the L1 loss function. After the network training is completed, the enhanced image of the underwater degradation image is obtained using the trained network. .

2. The underwater image enhancement method based on a depth-guided state-space model according to claim 1, characterized in that, Step 3 includes the following steps: Step 31: Set the initial features Input a deep convolutional layer, output enhanced features ; Step 32: Add the enhanced features Decomposed into horizontal branch features according to the channel dimension and vertical branching features : ； Step 33: Apply the depth potential field Quantized into K levels, forming a quantized deep potential field. : ； Where K represents the number of quantification levels; Step 34: The horizontal branch feature With the quantized depth potential field Evolution is performed by inputting a transverse state-space model: ； ； ； ； in, Indicates flattening; Indicates sorting; This indicates that the sorting algorithm is a stable sorting algorithm; Indicates the index of the horizontal branches; This indicates that contour line scanning and rearrangement are being performed; This indicates a lateral state-space evolution; This represents the intermediate features of the lateral branches after evolution through the state space; This represents the inverse mapping of the horizontal branching index; This indicates that spatial domain recovery is being performed; This represents the characteristics of the lateral branches after they have been restored to the spatial domain; Step 35: The vertical branch feature With the quantized depth potential field transpose Evolution is performed by inputting a longitudinal state-space model: ； ； ； ； in, Indicates flattening; Indicates sorting; This indicates that the sorting algorithm is a stable sorting algorithm; Indicates the sorting index of the vertical branches; This indicates that contour line scanning and rearrangement are being performed; This indicates a longitudinal state-space evolution; This represents the intermediate features of the vertical branch after evolution through the state space; This represents the inverse mapping of the vertical branching index; This indicates that spatial domain recovery is being performed; This represents the features of the vertical branches after they have been restored to the spatial domain; Step 36: Fuse the lateral features according to the channel dimension. With the aforementioned longitudinal features To obtain global features : 。 3. The underwater image enhancement method based on a depth-guided state-space model according to claim 1, characterized in that, In step 4, the local features are obtained by the parallel denoising branch. The calculation method is as follows: ； in, This represents a 3×3 depth convolution operation; Indicates global average pooling; This represents the Sigmoid activation function; Indicates a gating mechanism; This indicates element-wise multiplication.

4. The underwater image enhancement method based on a depth-guided state-space model according to claim 1, characterized in that, In step 5, the features are fused. The calculation method is as follows: ； in, This represents a feedforward neural network; This indicates the learning scaling parameter.

5. The underwater image enhancement method based on a depth-guided state-space model according to claim 1, characterized in that, Step 6 includes the following steps: Step 61, from the fusion features Extract semantic residuals : ； in, Indicates the activation function; Step 62: Convert the semantic residual With the depth potential field Residual fusion is performed to obtain the optical potential field. : 。 6. The underwater image enhancement method based on a depth-guided state-space model according to claim 1, characterized in that, Step 7 includes the following steps: Step 71: Utilize the optical potential field Generate physical transmission modulation weights : ； in, This represents the Sigmoid activation function. Step 72: Utilize the physical transmission modulation weights Modulate the fusion feature The final feature is obtained : ； in, This indicates element-wise multiplication.

7. The underwater image enhancement method based on a depth-guided state-space model according to claim 1, characterized in that, The loss function in step 8 is the L1 loss function: ； in, Indicates the total number of pixels in the image; This refers to the enhanced image; This refers to a real image.

Images