An underwater image enhancement method based on semantic guided sequence modeling

Abstract

The application provides an underwater image enhancement method based on semantic guidance sequence modeling, and relates to the technical field of underwater image processing, and comprises the following steps: obtaining an underwater degraded image and preprocessing, extracting semantic features through a pre-trained visual feature extraction network; performing spatial domain detail stabilization processing on the image, integrating local and global features through dynamic adaptive fusion; constructing a spatial semantic response mapping according to the semantic features, converting two-dimensional enhancement features into a one-dimensional sequence; performing forward and reverse recombination on the feature sequence, constructing a bidirectional sequence input; sending the bidirectional sequence into corresponding modeling units respectively, capturing bidirectional context dependency; fusing bidirectional modeling results to obtain a unified sequence representation, restoring to a two-dimensional spatial structure to obtain an enhancement feature mapping; generating pixel prediction values based on the feature mapping, and outputting an underwater enhanced image; aligning the enhanced image with a reference image to construct a reconstruction loss, and iteratively optimizing the model to convergence; and using the trained model to realize underwater image enhancement.

Description

Technical Field

[0001] This invention relates to the field of underwater image processing technology, and more particularly to an underwater image enhancement method based on semantically guided sequence modeling. Background Technology

[0002] The underwater environment possesses complex optical propagation characteristics, and the absorption and scattering of light by water significantly degrades image quality. Due to the combined effects of wavelength-dependent absorption and forward and backscattering, underwater images typically suffer from color distortion, reduced contrast, and blurred details such as edges and textures. This degradation not only affects human visual perception but also adversely impacts automated processing in applications such as underwater target recognition, tracking, and environmental awareness.

[0003] To improve underwater image quality, various image enhancement and restoration methods have been proposed in existing technologies, including physical model-based compensation methods, prior-based image enhancement methods, and end-to-end enhancement methods based on deep learning. Although some methods have achieved certain results in terms of local details or overall brightness, it is still difficult to balance global consistency and local structural stability in complex underwater scenes. In particular, when converting two-dimensional image features into a sequence for modeling, the spatial semantic structure is easily destroyed, leading to unstable enhancement results. Therefore, how to effectively introduce semantic information and achieve reasonable organization of features in the underwater image enhancement process still needs further research. Summary of the Invention

[0004] To address the aforementioned technical problem of the adverse effects of the underwater environment on image quality, an underwater image enhancement method based on semantically guided sequence modeling is provided.

[0005] The technical means employed in this invention are as follows:

[0006] An underwater image enhancement method based on semantically guided sequence modeling includes the following steps: S1. Acquire underwater degraded images and preprocess them. Simultaneously, extract semantic features of the underwater degraded images using a pre-trained visual feature extraction network. ; S2. Perform spatial domain detail stabilization processing on the underwater degraded image; the spatial domain detail stabilization processing includes: dynamic adaptive fusion operation and explicit high-frequency compensation operation; S3. Based on the semantic features of underwater degraded images Construct a semantic response mapping related to spatial location, and reorganize the enhanced feature mapping obtained after processing the S2 spatial domain into a one-dimensional sequence; S4. Perform forward sequence recombination and reverse sequence recombination on the one-dimensional feature sequence to obtain forward sequence input and reverse sequence input; S5. The forward sequence input and the reverse sequence input are respectively sent to the forward sequence modeling unit and the reverse sequence modeling unit to perform bidirectional sequence modeling, and the forward sequence modeling result and the reverse sequence modeling result, i.e., the bidirectional context dependency relationship, are obtained. S6. The forward sequence modeling result obtained in S5 is fused with the reverse sequence modeling result to obtain a unified sequence representation, and the fused sequence features are restored to a two-dimensional spatial structure to obtain an enhanced feature mapping. S7. Generate pixel prediction values ​​for the underwater image based on the enhanced feature map, and output the enhanced underwater image; S8. Align the generated underwater enhanced image with the reference image, construct the reconstruction loss, and iteratively optimize the model parameters based on the reconstruction loss until the loss function converges to obtain the final underwater image enhancement model. S9. Obtain the enhanced image of the underwater image to be enhanced using the final underwater image enhancement model.

[0007] Furthermore, in S1, the pre-trained visual feature extraction network is a DINOv2 network with frozen parameters, and its semantic features are calculated as follows: ; in, This represents the input underwater degraded image. This represents the semantic feature mapping extracted by the DINOv2 network.

[0008] Furthermore, in S2, the spatial domain detail stabilization processing method is as follows: ; in, This represents the features of the input underwater image. This represents high-frequency residual information extracted based on convolution operations. This represents the learnable adjustment coefficient. This represents the enhanced features after spatial domain detail stabilization.

[0009] Furthermore, in S3, the calculation method for constructing the spatial semantic response mapping based on semantic features is as follows: ; in, This represents the L2 norm operation on semantic features along the channel dimension. This represents a normalization operation in the spatial dimension. This represents a semantic response map used for sequence recombination.

[0010] Furthermore, in S3, the calculation method for recombining two-dimensional spatial features into a one-dimensional sequence is as follows: ; in, This represents the feature map after spatial domain detail stabilization. This represents a one-dimensional sequence representation obtained by flattening features in a two-dimensional space.

[0011] Furthermore, in S4, the forward and reverse sequences are constructed in the following ways: ; ; in, This indicates an operation to reorder sequences based on semantic response mappings. This represents a positive semantic recombination sequence. This represents a reverse semantic recombination sequence.

[0012] Furthermore, in S5, the bidirectional modeling method is as follows: ; ; in, This represents a sequence modeling unit used to capture contextual dependencies in a sequence.

[0013] Furthermore, in S6, the sequence fusion and spatial recovery method is as follows: ; in, This represents the operation of restoring a one-dimensional sequence feature to a two-dimensional spatial structure.

[0014] Furthermore, in S7 and S8, the model training and optimization methods are as follows: ; in, This represents the generated underwater enhanced image. Indicates a reference image. This represents the reconstruction loss function used for model parameter optimization.

[0015] Compared with the prior art, the present invention has the following advantages: To address the issues of semantic information confounding, local structural degradation, and unstable enhancement results caused by light absorption and scattering effects during underwater image enhancement, this invention introduces a semantically guided sequence modeling method based on a pre-trained visual model. By extracting high-level semantic features from underwater images and constructing semantic response mappings related to spatial location, semantically consistent sequence recombination of two-dimensional features is performed. This allows semantically similar or informationally similar regions to form a continuous structure within the sequence, thereby improving the coherence and stability of sequence modeling. Simultaneously, a spatial domain detail stabilization mechanism is introduced before sequence modeling. Through a combination of dynamic adaptive fusion and explicit high-frequency compensation, high-frequency structural information such as edges and textures is effectively enhanced, avoiding excessive attenuation of detail information during sequence modeling. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the model of the present invention.

[0018] Figure 2 The images show a comparison of the reconstruction results of this invention with other algorithms on the UIEB dataset. In the images, (a) represents the reconstruction result using the Ushape algorithm, (b) represents the reconstruction result using the AST algorithm, (c) represents the reconstruction result using the CDF-UIE algorithm, (d) represents the reconstruction result using the FMambaIR algorithm, (e) represents the reconstruction result using the UDNet algorithm, and (f) represents the reconstruction result using this invention.

[0019] Figure 3 The images show a comparison of the local magnification results of the present invention and other algorithms on publicly available underwater datasets. In the images, (a) represents the magnified reconstruction result using the Ushape algorithm, (b) represents the magnified reconstruction result using the AST algorithm, (c) represents the magnified reconstruction result using the CDF-UIE algorithm, (d) represents the magnified reconstruction result using the FMambaIR algorithm, (e) represents the magnified reconstruction result using the UDNet algorithm, and (f) represents the magnified reconstruction result using the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] like Figure 1 As shown, this invention provides an underwater image enhancement method based on semantically guided sequence modeling, characterized by the following steps: S1. Acquire underwater degraded images and preprocess them. Simultaneously, extract semantic features of the underwater degraded images using a pre-trained visual feature extraction network. In S1, the pre-trained visual feature extraction network is a DINOv2 network with frozen parameters, and its semantic features are calculated as follows: ; in, This represents the input underwater degraded image. This represents the semantic feature mapping extracted by the DINOv2 network.

[0023] S2. Perform spatial domain detail stabilization processing on the underwater degraded image; the spatial domain detail stabilization processing includes: dynamic adaptive fusion operation and explicit high-frequency compensation operation; in S2, the spatial domain detail stabilization processing method is as follows: ; in, This represents the features of the input underwater image. This represents high-frequency residual information extracted based on convolution operations. This represents the learnable adjustment coefficient. This represents the enhanced features after spatial domain detail stabilization.

[0024] S3. Based on the semantic features of the underwater degraded image Construct a semantic response map related to spatial location, and reassemble the enhanced feature map obtained after spatial domain processing in S2 into a one-dimensional sequence; in S3, the calculation method for constructing the spatial semantic response map based on semantic features is as follows: ; in, This represents the L2 norm operation on semantic features along the channel dimension. This represents a normalization operation in the spatial dimension. This represents a semantic response map used for sequence recombination.

[0025] S4. Perform forward sequence recombination and reverse sequence recombination on the one-dimensional feature sequence to obtain forward sequence input and reverse sequence input; in S4, the construction methods of the forward sequence and the reverse sequence are as follows: ; ; in, This indicates an operation to reorder sequences based on semantic response mappings. This represents a positive semantic recombination sequence. This represents a reverse semantic recombination sequence.

[0026] S5. The forward sequence input and the reverse sequence input are respectively fed into the forward sequence modeling unit and the reverse sequence modeling unit for bidirectional sequence modeling, to obtain the forward sequence modeling result and the reverse sequence modeling result, i.e., the bidirectional context dependency; in S5, the bidirectional modeling method is as follows: ; ; in, This represents a sequence modeling unit used to capture contextual dependencies in a sequence.

[0027] S6. The forward sequence modeling result obtained in S5 is fused with the reverse sequence modeling result to obtain a unified sequence representation, and the fused sequence features are restored to a two-dimensional spatial structure to obtain the enhanced feature mapping; the sequence fusion and spatial restoration method in S6 is as follows: ; in, This represents the operation of restoring a one-dimensional sequence feature to a two-dimensional spatial structure.

[0028] S7. Generate pixel prediction values ​​for the underwater image based on the enhanced feature map, and output the enhanced underwater image; S8. Align the generated underwater enhanced image with the reference image, construct a reconstruction loss, and iteratively optimize the model parameters based on the reconstruction loss until the loss function converges to obtain the final underwater image enhancement model; the model training and optimization method is as follows: ; in, This represents the generated underwater enhanced image. Indicates a reference image. This represents the reconstruction loss function used for model parameter optimization.

[0029] S9. Obtain the enhanced image of the underwater image to be enhanced using the final underwater image enhancement model.

[0030] To verify the generalization ability of this invention to complex underwater scenarios, underwater scenarios of different types were selected as test sets. At the same time, the experimental results were compared and analyzed qualitatively and quantitatively with those of the Ushape algorithm, AST algorithm, CDF-UIE algorithm, FMambaIR algorithm, and UDNet algorithm.

[0031] Example 1 like Figure 2 As shown, this invention provides a comparison of reconstruction results with various existing algorithms on the UIEB dataset for underwater scene images. (a) The Ushape algorithm shown has limited color correction capabilities for underwater scenes during reconstruction, resulting in noticeable color shifts in the overall image and blurred details in the background area. (b) The AST algorithm shown enhances image brightness to some extent but fails to effectively suppress color distortion caused by underwater scattering, leading to unnatural colors in local areas of the reconstruction result. (c) The CDF-UIE algorithm shown improves local contrast but is prone to over-enhancement in structurally complex areas, causing inconsistent texture details. (d) The FMambaIR algorithm shown has certain advantages in global modeling but is insufficient in restoring local high-frequency details in underwater scenes, resulting in blurred edges in some areas. (e) The UDNet algorithm shown is prone to color oversaturation during enhancement, leading to unstable overall visual effects in the reconstructed image. In contrast, (f) The method of this invention shown can effectively restore the edges and texture details of foreground targets while maintaining the overall structural consistency of the underwater scene, and suppress color distortion in the background area, thus obtaining a more natural and stable underwater image reconstruction effect.

[0032] like Figure 3As shown, this invention provides a comparison of the local magnification reconstruction effects with other algorithms on a publicly available underwater dataset. (a) The Ushape algorithm exhibits significant detail blurring in the magnified area, with unclear texture structures and difficulty in identifying target edge features. (b) The AST algorithm introduces discontinuities during local structure restoration, resulting in broken texture details and affecting overall visual consistency. (c) The CDF-UIE algorithm is prone to color shifts when enhancing local details, leading to inconsistencies in color representation between the magnified area and surrounding areas. (d) While the FMambaIR algorithm shows some improvement in global enhancement, it still suffers from detail blurring in local high-frequency structure restoration, with insufficient clarity in edge regions. (e) The UDNet algorithm exhibits over-enhancement in the magnified area, causing texture distortion and affecting the realistic representation of the target structure. (f) The method of this invention effectively preserves the edge and texture information of the target structure in the magnified area, with natural detail transitions and no obvious blurring or artifacts, thus achieving a more stable and clearer local detail reconstruction effect during underwater image enhancement.

[0033] This embodiment compares and analyzes the experimental results of different underwater image enhancement algorithms using three objective evaluation metrics: PSNR, SSIM, and MSE. PSNR measures the ratio of the maximum possible power of the signal between the enhanced image and the reference image to the reconstruction error noise power, reflecting the overall reconstruction quality. SSIM measures the structural similarity between two images in terms of brightness, contrast, and structural information, better aligning with human perception of image quality. MSE represents the mean square error at the pixel level between the enhanced image and the reference image, reflecting the magnitude of the reconstruction error. Table 1 shows that the method of this invention outperforms the compared algorithms in all three objective evaluation metrics. Compared to other methods, the underwater enhanced images generated by this invention effectively reduce reconstruction errors and improve overall image quality while maintaining structural consistency.

[0034] This invention introduces a semantically guided sequence modeling mechanism to effectively aggregate and model key structural regions in degraded underwater images. Before sequence modeling, spatial domain detail stabilization processing is performed, effectively enhancing high-frequency information such as edges and textures. Furthermore, bidirectional sequence modeling is used to jointly model global and local information, resulting in significantly improved PSNR, SSIM, and MSE metrics for the generated enhanced image. Overall, its performance surpasses existing underwater image enhancement methods.

[0035] Table 1. Comparison of PSNR, SSIM, and MSE results of the model of this invention with other advanced algorithms on the UIEB dataset.

[0036] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the above embodiments of the present invention, the descriptions of each embodiment have their own emphasis; parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. It should be understood that the disclosed technical content in the several embodiments provided in this application can be implemented in other ways.

[0037] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater image enhancement method based on semantically guided sequence modeling, characterized in that, Includes the following steps: S1. Acquire underwater degraded images and preprocess them. Simultaneously, extract semantic features of the underwater degraded images using a pre-trained visual feature extraction network. ; S2. Perform spatial domain detail stabilization processing on the underwater degraded image; the spatial domain detail stabilization processing includes: dynamic adaptive fusion operation and explicit high-frequency compensation operation; S3. Based on the semantic features of the underwater degraded image Construct a semantic response mapping related to spatial location, and reorganize the enhanced feature mapping obtained after processing the S2 spatial domain into a one-dimensional sequence; S4. Perform forward sequence recombination and reverse sequence recombination on the one-dimensional feature sequence to obtain forward sequence input and reverse sequence input; S5. The forward sequence input and the reverse sequence input are respectively sent to the forward sequence modeling unit and the reverse sequence modeling unit to perform bidirectional sequence modeling, and the forward sequence modeling result and the reverse sequence modeling result, i.e., the bidirectional context dependency relationship, are obtained. S6. The forward sequence modeling result obtained in S5 is fused with the reverse sequence modeling result to obtain a unified sequence representation, and the fused sequence features are restored to a two-dimensional spatial structure to obtain an enhanced feature mapping. S7. Generate pixel prediction values ​​for the underwater image based on the enhanced feature map, and output the enhanced underwater image; S8. Align the generated underwater enhanced image with the reference image, construct the reconstruction loss, and iteratively optimize the model parameters based on the reconstruction loss until the loss function converges to obtain the final underwater image enhancement model. S9. Obtain the enhanced image of the underwater image to be enhanced using the final underwater image enhancement model.

2. The underwater image enhancement method based on semantically guided sequence modeling according to claim 1, characterized in that, In S1, the pre-trained visual feature extraction network is a DINOv2 network with frozen parameters, and its semantic features are calculated as follows: ； in, This represents the input underwater degraded image. This represents the semantic feature mapping extracted by the DINOv2 network.

3. The underwater image enhancement method based on semantically guided sequence modeling according to claim 1, characterized in that, In S2, the spatial domain detail stabilization processing method is as follows: ； in, This represents the features of the input underwater image. This represents high-frequency residual information extracted based on convolution operations. This represents the learnable adjustment coefficient. This represents the enhanced features after spatial domain detail stabilization.

4. The underwater image enhancement method based on semantically guided sequence modeling according to claim 1, characterized in that, In S3, the calculation method for constructing the spatial semantic response mapping based on semantic features is as follows: ； in, This represents the L2 norm operation on semantic features along the channel dimension. This represents a normalization operation in the spatial dimension. This represents a semantic response map used for sequence recombination.

5. The underwater image enhancement method based on semantically guided sequence modeling according to claim 1, characterized in that, In S3, the calculation method for recombining two-dimensional spatial features into a one-dimensional sequence is as follows: ； in, This represents the feature map after spatial domain detail stabilization. This represents a one-dimensional sequence representation obtained by flattening features in a two-dimensional space.

6. The underwater image enhancement method based on semantically guided sequence modeling according to claim 1, characterized in that, In S4, the forward and reverse sequences are constructed in the following ways: ； ； in, This indicates an operation to reorder sequences based on semantic response mappings. This represents a forward semantic recombination sequence. This represents a reverse semantic recombination sequence.

7. The underwater image enhancement method based on semantically guided sequence modeling according to claim 1, characterized in that, In S5, the bidirectional modeling method is as follows: ； ； in, This represents a sequence modeling unit used to capture contextual dependencies in a sequence.

8. The underwater image enhancement method based on semantically guided sequence modeling according to claim 1, characterized in that, In step S6, the sequence fusion and spatial recovery method is as follows: ； in, This represents the operation of restoring a one-dimensional sequence feature to a two-dimensional spatial structure.

9. The underwater image enhancement method based on semantically guided sequence modeling according to claim 1, characterized in that, In S7 and S8, the model training and optimization methods are as follows: ； in, This represents the generated underwater enhanced image. Indicates a reference image. This represents the reconstruction loss function used for model parameter optimization.

Images