Lightweight underwater image enhancement method based on leapfrogging sampling diffusion model

Abstract

The invention discloses a lightweight underwater image enhancement method based on a leapfrog sampling diffusion model, and the method achieves the independent parallel processing of time step coding and color information through the design of a parallel structure of an attention-driven Transformer (AP-Trans) module, introduces a space attention mechanism to enhance the detail recovery capability, and improves the image quality. A global channel interaction module is adopted to maintain color fidelity, high-quality enhancement of underwater images is achieved, a large-parameter self-attention module in a traditional diffusion model is further replaced with a lightweight channel attention mechanism, the model parameter quantity is remarkably reduced, and a dynamic leapfrogging sampling strategy is adopted, so that the real-time performance of the underwater images is improved. According to the method, the sampling process of 20-50 steps of a traditional leapfrog diffusion model is reduced to 5 steps, the enhancement effect is remarkably improved while the excellent generation capacity of the diffusion model is kept, the problem of interactive interference of time step coding and color information is effectively solved, and recovery of underwater image colors and details is balanced.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and artificial intelligence, and in particular to underwater image quality enhancement technology, and specifically to a lightweight underwater image enhancement method based on a skip sampling diffusion model. Background Art

[0002] With the continuous deepening of ocean exploration, autonomous underwater vehicles (AUVs) are playing an increasingly important role in fields such as marine resource development and environmental monitoring. However, the underwater environment poses significant challenges to the visual system of AUVs, mainly due to two optical degradation effects: (1) color distortion caused by the selective absorption of light of different wavelengths underwater, which usually appears as a blue or green hue; (2) scattering of light by suspended particles, resulting in image blur and reduced visibility of details. These effects seriously affect the quality of underwater images, including image contrast and clarity, and limit the application scenarios of AUVs.

[0003] To solve the above optical degradation effects, traditional image processing techniques are divided into two categories: physical model-based and non-physical model-based methods, namely, underwater image enhancement based on physical models and underwater image enhancement based on multi-color space and channel frequency division. However, these methods often require specific adjustments for different underwater conditions, and have high computational complexity, making it difficult to meet real-time processing requirements. In recent years, deep learning technology has made significant progress in the field of underwater image enhancement. Methods based on convolutional neural networks (CNNs), such as Ucolor and UIEC^2, have shown good potential in color restoration. However, these methods mainly focus on color correction and do not adequately handle the blurring effect caused by scattering. In addition, due to the difficulty in obtaining standard images of real underwater scenes, the dependence on training data further limits their performance. Existing systems mainly focus on color correction and contrast stretching, and have not effectively solved the scattering blurring problem of underwater images. Although the DM-water method based on the diffusion model has made progress in color restoration, its Transformer module mainly relies on the channel attention mechanism and lacks a special structure to restore blurred details, and still has limitations in detail preservation. Therefore, the color distortion and scattering blurring problems of underwater images need to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a lightweight underwater image enhancement method based on a skip sampling diffusion model. By designing an attention-driven Transformer (AP-Trans) module, independent parallel processing of time step encoding and color information is realized. At the same time, a spatial attention mechanism is introduced to enhance the detail recovery capability, and a global channel interaction module is used to maintain color fidelity, so as to solve the problem that due to the complexity of the underwater environment, the light is selectively absorbed and scattered during the propagation process, resulting in serious color distortion and blurred details in the acquired image.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a lightweight underwater image enhancement method based on a skip sampling diffusion model, the method mainly includes two processes: training and reasoning:

[0006] Step S01: The training process is a Markov random process, that is, the result of the next step is determined only by the result of the current step and has nothing to do with the previous step. A fixed decreasing sequence from large to small is set as the signal retention coefficient α t , taking the clear underwater image label as the ideal target image x0;

[0007] Furthermore, to ensure that the model is not sensitive to the fixed number of steps and to increase the model's step robustness, a positive integer t is randomly selected from the uniform distribution U(1,T), (T=2000) as the number of steps to be learned in each training. According to the Markov chain, the result of adding noise to the label x0 after t steps is:

[0008]

[0009] in, is the cumulative product of the predetermined signal holding coefficients, That is, all α from step 0 to t t The continuous multiplication of; ∈ is the input Gaussian noise, and x0 is the ideal target image.

[0010] Step S02: Take the input degraded underwater image raw as the conditional variable c, combined with the time step t and its corresponding denoised image x t The diffusion model of the U-shaped network structure is input together, where the conditional variable c is used to guide the denoising process to ensure that the generated result is consistent with the input image content, and the time step t is used to control the denoising intensity to achieve progressive recovery from noise to clear image;

[0011] Furthermore, the diffusion model is optimized using an L1 loss function during training, which compares the difference between the noise predicted by the model and the Gaussian noise actually added, expressed as:

[0012]

[0013] Among them, ∈ is the input Gaussian noise, ∈ θ is the noise predicted by the model, α t is the predefined signal holding factor, is the cumulative product of the predetermined coefficients, expressed as x0 is the ideal target image, ground truth (GT) image, that is, the image with the desired ideal enhancement effect, c is the conditional variable, that is, the input degraded underwater image, t is the time step, and the training method based on noise prediction enables the diffusion model to learn the degradation distribution of the image, that is, the diffusion model realizes image restoration during sampling;

[0014] Furthermore, the diffusion model achieves image restoration during sampling by adopting a deterministic non-Markov sampling strategy to obtain random noise x from a standard Gaussian distribution. T Starting from ~N(0,I), skip sampling denoising is performed at (2000, 1500, 1000, 500) steps according to the following formula:

[0015]

[0016] Among them, t is the time step before skip sampling, t′ is the target step number of skip sampling, and the formula predicts the noise ∈ θ Guide the denoising process of each jump time step to achieve progressive recovery from noise to clear image. For the last jump step (500, 0), no additional noise needs to be added. The formula of this step is expressed as:

[0017]

[0018] Step S03: Replace the large-parameter self-attention model in the traditional diffusion model with a channel attention mechanism with low parameters and good color restoration effect, and construct a parallel dual-branch structure in the AP-Trans module, which includes a time step encoding branch and a color channel processing branch. The time step encoding branch uses two series-connected GSC (GroupNorm-Swish-Conv) modules to process the time information to generate the time step feature F t The color channel processing branch calculates the channel weights through global average pooling and convolution to generate channel enhancement features F c .

[0019] Furthermore, the specific structure of the AP-Trans module includes an input layer, parallel branches and a fusion layer. The input features are first pre-processed by convolution and then divided into two independent branches: time step encoding and channel enhancement. The time step encoding branch adopts two series-connected GSC modules, each of which contains group normalization, Swish activation function and convolution layer in sequence to extract time step features. The channel enhancement branch first performs global average pooling on the input features and then generates channel attention weights through convolution. The output features of the two branches are adaptively fused through the global color feature fusion module to achieve an effective combination of time step information and color information.

[0020] Step S04: A global color feature fusion module is used to fuse y and z. The module first extracts global features from the input features, then generates fusion weights through two convolution and normalization operations, and finally outputs fusion features. This design can adaptively adjust the weight of color information according to different time steps.

[0021] Furthermore, the global color feature fusion module adopts a fusion strategy guided by global color features. First, the input time step feature F t and channel enhancement feature F c Perform global average pooling,

[0022] The process of obtaining global context information can be expressed as:

[0023]

[0024] Where I represents the input of the module, X represents the features obtained after the input I is processed by the Conv3 convolution layer, and G N represents the channel attention feature of the Nth group, GAP represents global pooling, Conv(n) represents convolution with kernel n, σ represents the sigmoid activation function, Φ θ represents the GSC module;

[0025] After two 1x1 convolutions and layer normalization operations, the fusion weight α is generated through the sigmoid activation function to achieve F t and F c The adaptive weighted fusion process can be expressed as:

[0026]

[0027] Among them, f GCF ReLU represents the global color feature extraction function, ReLU represents the Relu activation function, and LN represents layer normalization. The design of this module allows the model to dynamically adjust the importance of color information according to different time steps and image content, and then output the fusion feature F f , adaptively adjust the weight of color information according to different time steps;

[0028] Step S05: To further enhance the detail recovery capability of the model, the present invention innovatively adopts a spatial attention mechanism in the diffusion model to f The spatial attention module is used to enhance the details. The spatial attention captures spatial information of different scales through parallel maximum pooling and average pooling operations, and generates a detail attention map to enhance the image details blurred by the scattering effect. The spatial information enhancement feature F is obtained through the spatial attention mechanism. eAfter that, it is sent to the multi-layer vector machine for calculation and information processing. The above steps constitute a complete attention-driven Trans encoding / decoding block;

[0029] Furthermore, the spatial attention module uses a dual-path parallel pooling strategy to capture spatial features. The module performs maximum pooling and average pooling operations simultaneously, where maximum pooling helps capture salient target features and average pooling helps obtain global spatial information. The results of the two poolings are concatenated, and the spatial feature pattern is extracted through a 7x7 convolution. Finally, a spatial attention map is generated through a sigmoid activation function. The attention map is used to enhance the key areas in the feature map, especially the detail areas affected by the scattering effect.

[0030] Step S06: bidirectional interaction between encoder and decoder features is achieved through a global channel interaction module, which sets a dual global channel feature collection head at the jump connection of the U-shaped network to achieve bidirectional selective transmission of features, and finally generate a high-quality enhanced image;

[0031] Furthermore, the global channel interaction module includes a dual global channel feature collection head, which processes the encoder feature F e and decoder features F d , each feature collection head first performs a global average pooling operation to extract global statistical information at the channel level, and then passes through two groups of processing units in sequence, each group contains 1×1 convolution, layer normalization and ReLU activation function, and finally passes through another group of 1×1 convolution and layer normalization, and uses sigmoid activation function to generate channel weights. The process can be expressed by the formula:

[0032] Y = Conv3[f GCF (X en )×X de , f GCF (X de )×X en ]

[0033] Among them, X en represents the encoder feature, X de Represents the decoder features, and realizes bidirectional selective transfer of features through cross-feature weighting to generate enhanced images.

[0034] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0035] (1) The present invention realizes the unique and effective processing of time step encoding and color information through the parallel structure design of AP-Trans module, and improves the modeling ability of the model for underwater image degradation;

[0036] (2) The introduction of the spatial attention mechanism significantly enhances the model's ability to recover blurred details caused by scattering; and the designed global channel interaction module successfully solves the problem of color fidelity degradation introduced by spatial attention;

[0037] (3) By replacing the large-parameter self-attention module in the traditional diffusion model with a lightweight channel attention mechanism, the number of model parameters is significantly reduced and the inference speed is improved. At the same time, a dynamic skip sampling strategy is adopted to reduce the 20-50-step sampling process of the traditional skip diffusion model to 5 steps, further improving the model inference efficiency.

[0038] (4) Experiments on public datasets show that the present invention achieves significant improvements in all reference indicators compared with the prior art, especially in severely degraded underwater images. The present invention provides a stable, reliable and real-time image enhancement solution for the underwater robot vision system. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0040] Figure 1 is a model network structure diagram of an embodiment of the present invention;

[0041] Figure 2 is a specific structural diagram of the global channel interconnection module in an embodiment of the present invention;

[0042] Figure 3 is a single-step training process diagram in an embodiment of the present invention;

[0043] Figure 4 It is the skip-step reasoning sampling process in the embodiment of the present invention;

[0044] Figure 5 It is a comparison chart of the experimental results of the present invention and the results of the existing method. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. In the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] See also Figures 1 to 4 , combined with Figures 1 to 4 The embodiment of the present invention provides the following technical solution: a lightweight underwater image enhancement method based on a skip sampling diffusion model. The present invention is characterized in that it effectively solves the interactive interference problem between time step encoding and color information, and balances the restoration of underwater image color and details. The technical solution mainly includes two processes: training and reasoning. The method specifically includes the following steps:

[0047] Step S01: Set a fixed decreasing sequence from large to small as the signal retention coefficient α t , taking the clear underwater image label as the ideal target image x0, training is performed based on the random process model;

[0048] In this embodiment, the training process is a Markov random process, that is, the result of the next step is determined only by the result of the current step and has nothing to do with the previous step. To ensure that the model is not sensitive to a fixed number of steps and to increase the robustness of the model's number of steps, each training randomly extracts a positive integer t from the uniform distribution U(1,T), (T=2000) as the number of steps to be learned. According to the Markov chain, the result of adding noise to the label x0 after t steps is:

[0049]

[0050] in, is the cumulative product of the predetermined coefficients, That is, all α from step 0 to t t The continuous multiplication of; ∈ is the input Gaussian noise, x0 is the ideal target image;

[0051] For example, a single-step training process is Figure 3 shown.

[0052] Step S02: Take the degraded underwater image raw as the conditional variable c and the random noise x t Enter the diffusion model together;

[0053] In this embodiment, the diffusion model is optimized during training using the L1 loss function, which compares the difference between the noise predicted by the model and the Gaussian noise actually added, expressed as:

[0054]

[0055] Among them, ∈ is the input Gaussian noise, ∈ θ is the noise predicted by the model, α t is the predefined signal holding factor, is the cumulative product of the predetermined coefficients, expressed as x0 is the ideal target image, ground truth (GT) image, that is, the image with the desired ideal enhancement effect, c is the conditional variable, that is, the input degraded underwater image, t is the time step, and the training method based on noise prediction enables the model to learn the degradation distribution of the image, thereby achieving high-quality image restoration during sampling;

[0056] Furthermore, in order to improve the sampling efficiency during the sampling process of the diffusion model, the present invention adopts the following Figure 4 The deterministic non-Markov sampling strategy shown above obtains random noise x by sampling from a standard Gaussian distribution using a larger sampling step size without affecting the generation quality. T Starting from ~N(0,I), skip sampling denoising is performed at (2000, 1500, 1000, 500) steps according to the following formula:

[0057]

[0058] Among them, t is the time step before skip sampling, t′ is the target step number of skip sampling, and the formula predicts the noise ∈ θ Guide the denoising process of each jump time step to achieve progressive recovery from noise to clear image. For the last jump step (500, 0), no additional noise is needed. The formula of this step is expressed as:

[0059]

[0060] For example, the specific implementation method of the diffusion model during the training process is as follows:

[0061] 1) Sample noise ∈ from a standard Gaussian distribution;

[0062] 2) Adjust α according to the predefined variance t Calculate the cumulative coefficient

[0063] 3) Generate a noisy image x using the reparameterization technique t ;

[0064] 4) Input conditional variable c (degraded underwater image) and time step t;

[0065] 5) Calculate the noise ∈ predicted by the model θ The L1 loss between ∈ and the actual noise ∈;

[0066] 6) Optimize model parameters through back-propagation.

[0067] Exemplarily, the specific implementation of the diffusion model in the sampling process is as follows:

[0068] 1) Sample the initial noise x from a standard Gaussian distributionT ;

[0069] 2) The noise x T , conditional variables c and time step t are input into the model;

[0070] 3) According to the predicted noise ∈ θ Calculate the previous step noisy image x t-1 ;

[0071] 4) Gradually reduce the noise level until the final enhanced image is obtained.

[0072] Exemplarily, the method of the present invention has no strict restrictions on the size of the input image. For large-size images that exceed the GPU memory limit, a sliding window strategy can be used for block processing. The recommended size of each image block is 256x256 pixels, and a certain overlap area is maintained between adjacent image blocks (the recommended overlap rate is 0.1) to ensure a smooth transition at the splicing point. Finally, the processed image blocks are spliced ​​into a complete enhanced image by weighted averaging to improve the efficiency of processing large-size images.

[0073] Exemplarily, the present invention adopts the following sliding window strategy:

[0074] 1) Divide the input image into multiple image blocks of size 256x256;

[0075] 2) Keep 10% overlap between adjacent image blocks;

[0076] 3) Process each image block independently;

[0077] 4) Use weighted averaging to stitch the processed image blocks into a complete image

[0078] Step S03: Construct parallel time step encoding and channel enhancement branches in the AP-Trans module;

[0079] In this embodiment, combined Figure 2 , Figure 2 : is a specific structural diagram of the global channel interconnection module in an embodiment of the present invention. The specific structure of the AP-Trans module includes an input layer, a parallel branch and a fusion layer. The input feature is first preprocessed by 3×3 convolution, and then divided into two independent branches: time step encoding and channel enhancement. The time step encoding branch uses two series-connected GSC modules, each of which sequentially includes group normalization, Swish activation function and 3×3 convolution layer to extract time step features. The channel enhancement branch first performs global average pooling on the input features, and then generates channel attention weights through 1×1 convolution. The output features of the two branches are adaptively fused through the global color feature fusion module to achieve an effective combination of time step information and color information.

[0080] Exemplarily, the detailed structure of the AP-Trans module includes:

[0081] 1) Input layer: perform 3×3 convolution preprocessing on the input features;

[0082] 2) Parallel branches: including time step encoding branch and channel enhancement branch;

[0083] 3) Time step encoding branch: consists of two serially connected GSC modules;

[0084] 4) Channel enhancement branch: global average pooling and 1x1 convolution are used;

[0085] 5) Global color feature fusion module: realizes the adaptive fusion of two branch features;

[0086] 6) Spatial attention module: enhances the detail information in the feature map.

[0087] In this embodiment, in order to solve the real-time technical problem of underwater image enhancement tasks, the present invention replaces the large-parameter self-attention model in the traditional diffusion model with a channel attention mechanism with low parameters and good color restoration effect. In view of the global pooling characteristics of the channel attention, which leads to its lack of fine-grained resolution ability for time step information, the present invention constructs a parallel dual-branch structure in the AP-Trans module. The dual-branch structure includes a time step encoding branch and a color channel processing branch, wherein the time step encoding branch uses two series-connected GSC (GroupNorm-Swish-Conv) modules to process the time information for generating the time step feature F t ; The color channel processing branch calculates the channel weight through global average pooling and convolution to generate the channel enhancement feature F c ;

[0088] Exemplarily, in the two GSC modules included in the time step encoding branch, each GSC module consists of group normalization, Swish activation function and 3×3 convolution layer, and the channel enhancement branch uses global average pooling and 1×1 convolution to calculate channel weights; the outputs of the two branches are fused by the global color feature fusion module;

[0089] Exemplarily, the U-type network adopts a symmetrical encoder-decoder structure, including 4 downsampling and upsampling stages, each encoding stage includes an AP-Trans module and a downsampling layer with a step size of 2, and the number of feature channels is set to 32, 64, 128, and 256, respectively. Correspondingly, each decoding stage includes an upsampling layer and an AP-Trans module, and the number of feature channels is 256, 128, 64, and 32, respectively; a global channel interaction module is set between the encoder and decoder features of the same scale to realize bidirectional selective transmission of features, and the conditional variable c is used to guide the denoising process to ensure that the generated results are consistent with The content of the input image remains consistent, and the time step t is used to control the denoising intensity to achieve progressive recovery from noise to a clear image. This symmetrical design ensures the gradual abstraction and fine reconstruction of features. The decoder adopts a convolutional neural network structure, and its last layer outputs a 3×H×W tensor, where 3 represents the three RGB channels, and H and W are the height and width of the input image, respectively. The output tensor is processed by the tanh activation function to map the pixel value range to [-1, 1], and then adjusted to the [0, 1] range through a linear transformation, representing the normalized RGB color value. This design ensures that the output image has a reasonable color distribution.

[0090] Step S04: Use the global color feature fusion module to fusion the time step feature F t and channel enhancement feature F c to integrate;

[0091] In this embodiment, the global color feature fusion module adopts a fusion strategy guided by global color features. Specifically, firstly, the input time step feature F t and channel enhancement feature F c Perform global average pooling,

[0092] The process of obtaining global context information can be expressed as:

[0093]

[0094] Where I represents the input of the module, X represents the features obtained after the input I is processed by the Conv3 convolution layer, and G N represents the channel attention feature of the Nth group, GAP represents global pooling, Conv(n) represents convolution with kernel n, σ represents the sigmoid activation function, Φ θ represents the GSC module;

[0095] Furthermore, after two 1x1 convolutions and layer normalization operations, the fusion weight α is generated through the sigmoid activation function to achieve F t and F c The adaptive weighted fusion process can be expressed as:

[0096]

[0097] Among them, f GCF ReLU represents the global color feature extraction function, ReLU represents the Relu activation function, and LN represents layer normalization. The design of this module allows the model to dynamically adjust the importance of color information according to different time steps and image content, and then output the fusion feature F f ,This design can adaptively adjust the weight of color information according to different time steps;

[0098] Exemplarily, the specific processing process of the global color feature fusion module is as follows:

[0099] 1) Perform global average pooling on the input features to obtain global statistical information in the channel dimension;

[0100] 2) After two sets of 1x1 convolution and layer normalization operations;

[0101] 3) Generate fusion weight α through sigmoid activation function;

[0102] 4) Realize the weighted fusion of time step features and channel enhancement features.

[0103] Step S05: fusion feature F f The spatial attention module performs the maximum pooling and average pooling operations at the same time. The pooling result is processed by 7x7 convolution and sigmoid activation function to generate a spatial attention map, which is used to enhance the detail information in the feature and output the enhanced feature F. e ;

[0104] In this embodiment, in order to further enhance the detail recovery capability of the model, the present invention innovatively adopts a spatial attention mechanism in the diffusion model to f The spatial attention module is sent to the detail enhancement module. The spatial attention captures spatial information of different scales through parallel maximum pooling and average pooling operations to generate a detail attention map for enhancing image details blurred by the scattering effect. The spatial information enhancement feature F is obtained through the spatial attention mechanism. e After that, it is sent to the multi-layer vector machine for calculation and information processing. The above steps constitute a complete attention-driven Trans encoding / decoding block;

[0105] Exemplarily, the specific implementation process of the spatial attention module is as follows:

[0106] 1) Perform maximum pooling and average pooling operations on the input features at the same time to obtain the maximum value feature and average value feature of size 1x1x c respectively;

[0107] 2) The two pooling results are concatenated in the channel dimension and the spatial feature patterns are extracted through 7x7 convolution;

[0108] 3) Generate spatial attention map through sigmoid activation function;

[0109] 4) Multiply the attention map with the input features to enhance the important areas;

[0110] Specifically, maximum pooling helps to capture salient target features, while average pooling helps to obtain global spatial information. The spatial attention map generated by splicing the results of the two pooling methods is used to enhance the key areas in the feature map, especially the detail areas affected by the scattering effect.

[0111] Step S06: realizing bidirectional interaction between encoder and decoder features through a global channel interaction module;

[0112] In this embodiment, the global channel interaction module includes a dual global channel feature collection head, which processes the encoder feature F e and decoder features F d , each feature collection head first performs a global average pooling operation to extract global statistical information at the channel level, and then passes through two groups of processing units in sequence, each group contains 1×1 convolution, layer normalization and ReLU activation function, and finally passes through another group of 1×1 convolution and layer normalization, and uses sigmoid activation function to generate channel weights. The process can be expressed by the formula:

[0113] Y = Conv3[f GCF (X en )×X de , f GCF (X de )×X en ]

[0114] Among them, X en represents the encoder feature, X de Represents decoder features, and realizes bidirectional selective transmission of features through cross-feature weighting, which can effectively transmit detail information while maintaining color fidelity, and finally generate high-quality enhanced images;

[0115] Exemplarily, the global channel interaction module implements feature interaction at the jump connection of the U-shaped network, and the specific processing process is as follows:

[0116] 1) Encoder feature F e and decoder features F d Perform global average pooling respectively;

[0117] 2) The pooled features are processed in two groups of processing units in sequence, each of which contains 1×1 convolution, layer normalization and ReLU activation function;

[0118] 3) After a set of 1×1 convolutions and layer normalization, the sigmoid activation function is used to generate channel weights;

[0119] 4) Implement F e and F d The cross-features are weighted to generate fusion features.

[0120] In this embodiment, the experimental result indicators of the present invention and other methods are compared, and the following data are obtained ("↑" and "↓" respectively indicate that the larger the value, the better and the smaller the value, the better):

[0121] MSE↓ PSNR↑ SSIM↑ Semi-UIR 313.64 25.09 0.93 FiveA+ 531.93 22.34 0.90 UIEC^2 25.1749 23.8125 0.91 DM-water 288.19 26.85 0.94 The present invention 244.56 28.17 0.94

[0122] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

[0123] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0124] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A lightweight underwater image enhancement method based on a skip sampling diffusion model, characterized by: The following steps are involved: Step S01: Set a fixed decreasing sequence from large to small as the signal retention coefficient α t , taking the clear underwater image label as the ideal target image x0, training is performed based on the random process model; Step S02: Take the degraded underwater image raw as the conditional variable c, combined with the time step t and its corresponding denoised image x t The diffusion model is input together, and the diffusion model adopts a spatial attention mechanism and has a U-shaped network structure; Step S03: replacing the large-parameter self-attention model in the diffusion model with a low-parameter channel attention mechanism model, constructing an attention-driven AP-Trans module, and constructing parallel time-step encoding and channel enhancement branches in the AP-Trans module; Step S04: Construct a global color feature fusion module to fuse the time step feature and the channel enhancement feature to obtain the fusion feature F f ; Step S05: the fusion feature F f The spatial attention module adopts the spatial attention mechanism, performs maximum pooling and average pooling operations, generates a spatial attention map based on the pooling results, and outputs the enhanced feature F e ; Step S06: Finally, the two-way interaction between the encoder and decoder features is realized through the global channel interaction module.

2. The lightweight underwater image enhancement method based on the skip sampling diffusion model according to claim 1 is characterized in that: The training based on the random process model includes a training process and an inference process. The training process is a Markov random process with a certain forward and reverse transfer probability distribution. The state transfer of each step depends only on the current state.

3. The lightweight underwater image enhancement method based on the skip sampling diffusion model according to claim 1 is characterized in that: The diffusion model is optimized using the L1 loss function during training. The loss function compares the difference between the noise predicted by the model and the actual Gaussian noise added, expressed as: Among them, ∈ is the input Gaussian noise, ∈ θ is the noise predicted by the model, α t is the predefined signal holding factor, is the cumulative product of the predetermined coefficients, expressed as x0 is the ideal target image, ground truth (GT) image, that is, the image with the desired ideal enhancement effect, c is the conditional variable, that is, the input degraded underwater image, t is the time step, and the training method based on noise prediction enables the diffusion model to learn the degradation distribution of the image, that is, the diffusion model realizes image restoration during sampling.

4. The lightweight underwater image enhancement method based on the skip sampling diffusion model according to claim 3 is characterized by: The diffusion model achieves image restoration during sampling by adopting a deterministic non-Markov sampling strategy to obtain random noise x from a standard Gaussian distribution. T Starting from ~N(0,I), skip sampling denoising is performed at (2000, 1500, 1000, 500) steps according to the following formula: Among them, t is the time step before skip sampling, t′ is the target step number of skip sampling, and the formula predicts the noise ∈ θ Guide the denoising process of each jump time step to achieve progressive recovery from noise to clear image. For the last jump step (500, 0), no additional noise needs to be added. The formula of this step is expressed as: Among them, the specific implementation method of the diffusion model during the training process is as follows: 1) Sample noise ∈ from a standard Gaussian distribution; 2) Adjust α according to the predefined variance t Calculate the cumulative coefficient 3) Generate a noisy image x using the reparameterization technique t ; 4) Input conditional variable c (degraded underwater image) and time step t; 5) Calculate the noise ∈ predicted by the model θ The L1 loss between ∈ and the actual noise ∈; 6) Optimize model parameters through back propagation; Among them, the specific implementation of the diffusion model in the sampling process is as follows: 1) Sample the initial noise x from a standard Gaussian distribution T ; 2) The noise x T , conditional variables c and time step t are input into the model; 3) According to the predicted noise ∈ θ Calculate the previous step noisy image x t-1 ; 4) Gradually reduce the noise level until the final enhanced image is obtained.

5. The lightweight underwater image enhancement method based on the skip sampling diffusion model according to claim 1, characterized in that: The detailed structure of the attention-driven AP-Trans module includes: Input layer: perform 3×3 convolution preprocessing on the input features; Parallel branches: including time step encoding branch and channel enhancement branch; Time step encoding branch: It consists of two serially connected GSC modules, which consist of group normalization, Swish activation function and 3×3 convolutional layer; The time step encoding branch uses two serially connected GSC modules to process the time information to generate the time step feature F t ; Channel enhancement branch: global average pooling and 1x1 convolution are used; The channel enhancement branch calculates the channel weights through global average pooling and convolution to generate the channel enhancement feature F c ; Global color feature fusion module: realizes adaptive fusion of two branch features; The outputs of the time step encoding branch and the channel enhancement branch are fused by the global color feature fusion module; Spatial attention module: enhances detail information in feature maps.

6. The lightweight underwater image enhancement method based on the skip sampling diffusion model according to claim 5 is characterized in that: The U-type network in the diffusion model adopts a symmetrical encoder-decoder structure, including 4 downsampling and upsampling stages, each encoding stage includes an AP-Trans module and a downsampling layer with a step size of 2, and the number of feature channels is set to 32, 64, 128, and 256, respectively. Correspondingly, each decoding stage includes an upsampling layer and an AP-Trans module, and the number of feature channels is 256, 128, 64, and 32, respectively. A global channel interaction module is set between the encoder and decoder features of the same scale to perform bidirectional selective transmission of features. The denoising process is guided based on the conditional variable c, and the denoising intensity is controlled based on the time step t. The decoder adopts a convolutional neural network structure, and the last layer outputs a 3×H×W tensor, where 3 represents the three RGB channels, H and W are the height and width of the input image, respectively. The output tensor is processed by the tanh activation function to map the pixel value range to [-1, 1], and finally adjusted to the [0, 1] range through linear transformation, representing the normalized RGB color value, and outputting the image.

7. The lightweight underwater image enhancement method based on the skip sampling diffusion model according to claim 6 is characterized in that: The global color feature fusion module adopts a fusion strategy guided by global color features to input the time step feature F t and channel enhancement feature F c Perform global average pooling; The process of obtaining global context information can be expressed as: Where I represents the input of the module, X represents the features obtained after the input I is processed by the Conv3 convolution layer, and G N represents the channel attention feature of the Nth group, GAP represents global pooling, Conv(n) represents convolution with kernel n, σ represents the sigmoid activation function, Φ θ represents the GSC module; After two 1x1 convolutions and layer normalization operations, the fusion weight α is generated through the sigmoid activation function to achieve F t and F c The adaptive weighted fusion process can be expressed as: Among them, f GCF ReLU represents the global color feature extraction function, ReLU represents the Relu activation function, and LN represents layer normalization. The design of this module allows the model to dynamically adjust the importance of color information according to different time steps and image content, and then output the fusion feature F f , adaptively adjust the weight of color information according to different time steps.

8. The lightweight underwater image enhancement method based on the skip sampling diffusion model according to claim 1, characterized in that: The diffusion model adopts a spatial attention mechanism to f The image is sent to a spatial attention module for detail enhancement. The spatial attention module captures spatial information of different scales through parallel maximum pooling and average pooling operations, generates a detail attention map, and enhances image details blurred by scattering effects. The spatial information enhancement feature F is obtained through the spatial attention mechanism e Afterwards, it is sent to the multi-layer vector machine for calculation and information processing to form a complete attention-driven Trans encoding / decoding block; The specific implementation method of the spatial attention module is as follows: 1) Perform maximum pooling and average pooling operations on the input features at the same time to obtain the maximum value feature and average value feature of size 1x1xc respectively; 2) The two pooling results are concatenated in the channel dimension and the spatial feature patterns are extracted through 7x7 convolution; 3) Generate spatial attention map through sigmoid activation function; 4) Multiply the attention map with the input features to enhance the important areas.

9. The lightweight underwater image enhancement method based on the skip sampling diffusion model according to claim 1, characterized in that: The global channel interaction module includes a dual global channel feature collection head to process the encoder feature F e and decoder features F d , each feature collection head first performs a global average pooling operation to extract global statistical information at the channel level, and then passes through two groups of processing units in sequence, each group contains 1×1 convolution, layer normalization and ReLU activation function, and finally passes through another group of 1×1 convolution and layer normalization, and uses sigmoid activation function to generate channel weights. The process can be expressed by the formula: Y=Conv3[f G3F (X en )×X de ,f GCF (X de )×X en ] Among them, X en represents the encoder feature, X de Represents the decoder features, and realizes bidirectional selective transfer of features through cross-feature weighting to generate enhanced images.

10. The lightweight underwater image enhancement method based on skip sampling diffusion model according to claim 9, characterized in that: The global channel interaction module realizes feature interaction at the jump connection in the U-shaped network structure. The specific processing process is as follows: 1) Encoder feature F e and decoder features F d Perform global average pooling respectively; 2) The pooled features are processed in two groups of processing units in sequence, each of which contains 1×1 convolution, layer normalization and ReLU activation function; 3) After a set of 1×1 convolutions and layer normalization, the sigmoid activation function is used to generate channel weights; 4) Implement F e and F d The cross-features are weighted to generate fusion features.

Images