Decoupling cooperative enhancement method for underwater image degradation factor

Abstract

The invention discloses a decoupling cooperative enhancement method for an underwater image degradation factor, and belongs to the technical field of underwater image enhancement. Aiming at the problem that an underwater imaging environment is complex and changeable, and light propagation is influenced by wavelength selective absorption and suspended particle scattering to cause image degradation, a degradation factor is approximately decomposed into relatively independent scattering degradation and color cast degradation, and a binarization strategy is introduced to compress the storage and calculation amount of data, so that the image degradation is realized. And global information is compensated in back propagation through decoupling learning, and data compression precision loss is recovered. According to the decoupling collaborative optimization method, scattering and color problems are distributed to a physical compensation module and a spectrum compensation module, learning information is isolated through a gating mechanism, and degeneration decoupling is achieved; the loss function is combined with the subtasks and the global loss to compensate the binarization precision loss of the submodules, and finally, the modules are cascaded to process and recover the degraded image.

Description

Technical Field

[0001] This invention belongs to the field of underwater image enhancement technology, specifically relating to a decoupled collaborative enhancement method for underwater image degradation factors. Background Technology

[0002] Underwater image enhancement has demonstrated interdisciplinary application potential in marine engineering, environmental monitoring, and smart devices. This technology significantly improves image recognizability and information integrity by compensating for the physical constraints of the underwater medium during the imaging process, thus providing a reliable perception foundation for tasks such as underwater autonomous navigation systems.

[0003] Underwater imaging environments are extremely complex and variable, posing a far greater challenge to underwater image enhancement than other natural image processing tasks. Image degradation occurs due to the dual effects of color shift and scattering blurring as light propagates in water, and these two degradation factors exhibit relative independence. This unique property necessitates treating them as separate phenomena when simulating underwater image degradation. From a physical mechanism perspective, they essentially correspond to scattering and absorption phenomena, respectively. Scattering degradation is primarily caused by the light scattering effect of suspended particles in water, following Mie or Rayleigh scattering theories. The degree of degradation is related to particle concentration, particle size distribution, and light wavelength. Color shift degradation is due to the selective absorption of light of different wavelengths by water, following the Beer-Lambert law, with its attenuation coefficient changing exponentially with wavelength. This spectral attenuation difference is a key factor leading to observed color distortion and loss of visual fidelity. Traditional mainstream paradigms employ end-to-end learning, abstractly modeling scattering and color shift in a high-dimensional feature space to achieve a nonlinear mapping from degraded to clear images. This method is based on probabilistic modeling. However, coupling scattering and color deviation into Ignoring their relative independence, from an optical imaging perspective, scattering follows Mie or Rayleigh theory, primarily affecting high-frequency spatial information; color deviation follows the Beer-Lambert law, affecting spectral channel relationships. Traditional transformation matrix gradient entanglement makes it difficult to distinguish degradation contributions. However, decoupling alone is insufficient for practical needs. Traditional methods improve performance by increasing parameters and computational complexity, but this exacerbates deployment challenges.

[0004] The current mainstream research paradigm involves abstractly modeling the degradation factors of underwater images and using deep learning methods for end-to-end learning. This learning approach confounds the individual effects of color cast and scattering, resulting in a mixed set of degradation factor restoration schemes and thus poor restoration performance. Furthermore, traditional deep learning methods require substantial computational resources, further limiting the model's potential application scope. Summary of the Invention

[0005] To address the problem of image degradation caused by the complex and variable underwater imaging environment, where light propagation is affected by wavelength-selective absorption and scattering by suspended particles, this invention provides a decoupled synergistic enhancement method for underwater image degradation factors.

[0006] To achieve the above objectives, the present invention employs the following technical solutions:

[0007] A decoupled collaborative enhancement method for underwater image degradation factors, the method comprising the following steps:

[0008] Step 1: Design a decoupled collaborative enhancement network model for underwater image degradation factors. The decoupled collaborative enhancement network model includes a decoupling gate, a spectral compensation module, and a physical compensation module. Train the spectral compensation module and the physical compensation module.

[0009] The specific operation of step 1 is as follows:

[0010] S11: Input the preset training threshold of the current target into the decoupling gating. When the training reaches the threshold, output the activation signal of the next target. This gating is the core of realizing decoupling learning of degradation factors. The formula is:

[0011]

[0012] in, To decouple the gating, It is a heaviside transition function; it outputs 1 when the input is not less than 0, and 0 otherwise. A training threshold is preset for the task objective; a larger threshold indicates a longer learning cycle for the model throughout the training process. Setting unequal training thresholds for the two modules achieves an asymmetric learning approach, indicating a focus on learning a specific degradation factor; conversely, setting equal training thresholds for both modules achieves a symmetric learning approach. For training steps, when training steps Decoupling gate when the value exceeds the threshold. It is in a closed state, and in each training session, the decoupled gating output is used to determine the next target. The activation signal is given by the formula:

[0013]

[0014] The output of the decoupling gate is either 0 or 1. When the output of the decoupling gate is 1, the decoupling gate is activated, and the activation signal is... Physical compensation module Training begins. When the output of the decoupling gate is 0, the decoupling gate is closed. The activation signal is... Spectral compensation module Begin training;

[0015] S12: Training of Spectral and Physical Compensation Modules; The activation signals of the target modules and training image data are input into the corresponding target modules to train them, resulting in enhanced spectral compensation and enhanced physical compensation modules. This stage aims to compress the input image data and learn the two types of degradation compensation modules. Before training begins, the activation signals of the training targets are output through an external gating mechanism to control the training of the two types of compensation modules. The backpropagation gradient flow during the learning process of each module only updates the corresponding degradation module. This learning strategy explicitly decouples the learning of physical degradation and spectral degradation in the parameter space, resolving the entanglement phenomenon that occurs during the learning of different modules.

[0016] The training image data is input into the network and flows to two branches. First, branch one performs preliminary processing by standardizing the information distribution of the RGB color space of the input image data using Z-Score.

[0017]

[0018] in, Channels representing input image data This indicates the adjusted result. For the average value, For the calculation of standard deviation;

[0019] Branch two performs binarization compression on the image data and extracts pixel and channel features. The image data compression process is an approximate calculation process from bit-level image data to full-precision image data, and the formula is:

[0020]

[0021] in, For traditional convolution operations, For binary convolution operations, the process of binarizing image data is handled by a sign function. This means that input image data greater than 0 is compressed to 1, and input image data less than 0 is compressed to 0. This method compresses 32-bit full-precision image data into 1-bit image data, thereby minimizing image data storage space and computational complexity. This is the scaling factor for the weighting term, which recovers information lost due to compression precision to some extent. Finally, these two branches are input to the underwater scene pixel distribution curve for image data recovery, using the following formula:

[0022]

[0023] in, and The scaling and bias factors are learned by the degradation compensation module. The output of the module is represented by the degradation compensation module, which consists of a physical compensation module and a spectral compensation module. The corresponding loss consists of a global loss and a module-specific loss. The global loss calculates the overall structural difference between the predicted image data and the ground truth image data, using the following formula:

[0024]

[0025] in, Calculate the squared average similarity error between the predicted image and the ground truth image. Image similarity is measured from three dimensions: brightness, contrast, and structure.

[0026] The formula for the specificity loss of the spectral compensation module is as follows:

[0027]

[0028] in, Constrain the color space of the image. Supervise the frequency domain of the image;

[0029] The specific loss formula for the physical compensation module is as follows:

[0030]

[0031] in, Monitor the dark channel of image data. The transmission graph of image data is supervised; the specific losses of each module are used to calculate the differences in spectral information and physical transmittance between the predicted image data and the ground truth (GT) data. The global loss avoids the module only learning its own task content after decoupling, thus providing some information supplementation. The spectral compensation module and the physical compensation module have the following advantages in this decoupled and collaborative learning approach: firstly, due to module decoupling, they learn more specific degradation recovery capabilities; secondly, due to alternating collaboration, the information from the two modules complements each other, mitigating the information loss problem caused by data binarization.

[0032] S13: Repeated iterative training and learning process. During the iteration, the spectral compensation module and the physical compensation module alternately and collaboratively optimize according to the preset training threshold until the loss function converges, and the final task-specific degradation compensation module is obtained.

[0033] Step 2: Input the degraded image into the decoupled collaborative enhancement network model and restore the image by cascading modules. Output the enhanced image. The degraded image is processed by the cascaded spectral compensation module and physical compensation module. In this stage, the degradation compensation module has already undergone decoupled collaborative optimization, so the participation of decoupled gating is not required.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] This invention decouples and collaboratively learns the main underwater image degradation factors, allowing each module to focus on its target sub-task. Theoretical and experimental results verify that the degradation compensation module obtained through this method learns cleaner task processing information and more specialized task processing capabilities. Furthermore, the invention's concise network structure and effective data compression reduce model storage and computational overhead, and the alternating learning process between degradation modules compensates to some extent for the accuracy loss caused by binarization compression. Experiments on public datasets demonstrate that this invention outperforms mainstream inventions in both quality and efficiency, achieving high performance at minimal cost and showing great potential in resource-constrained devices. Attached Figure Description

[0036] Figure 1 This is a framework diagram of a decoupled collaborative enhancement network model for underwater image degradation factors. Detailed Implementation

[0037] To gain a deeper understanding of this invention, we will provide a comprehensive and detailed description. However, this invention has various implementations and is not limited to the specific examples listed herein. These examples are presented to enhance a full understanding of the disclosure of this invention.

[0038] A decoupled collaborative enhancement method for underwater image degradation factors, the method comprising the following steps:

[0039] Step 1: Design a decoupled collaborative enhancement network model for underwater image degradation factors. The decoupled collaborative enhancement network model includes a decoupling gate, a spectral compensation module, and a physical compensation module. Train the spectral compensation module and the physical compensation module.

[0040] The specific operation of step 1 is as follows:

[0041] S11: Input the preset training threshold of the current target into the decoupling gating. When the training reaches the threshold, output the activation signal of the next target. This gating is the core of realizing decoupling learning of degradation factors. The formula is:

[0042]

[0043] in, To decouple the gating, It is a heaviside transition function; it outputs 1 when the input is not less than 0, and 0 otherwise. A training threshold is preset for the task objective; a larger threshold indicates a longer learning cycle for the model throughout the training process. Setting unequal training thresholds for the two modules achieves an asymmetric learning approach, indicating a focus on learning a specific degradation factor; conversely, setting equal training thresholds for both modules achieves a symmetric learning approach. For training steps, when training steps Decoupling gate when the value exceeds the threshold. It is in a closed state, and in each training session, the decoupled gating output is used to determine the next target. The activation signal is given by the formula:

[0044]

[0045] The output of the decoupling gate is either 0 or 1. When the output of the decoupling gate is 1, the decoupling gate is activated, and the activation signal is... Physical compensation module Training begins. When the output of the decoupling gate is 0, the decoupling gate is closed. The activation signal is... Spectral compensation module Begin training;

[0046] S12: Training of Spectral and Physical Compensation Modules; The activation signals of the target modules and training image data are input into the corresponding target modules to train them, resulting in enhanced spectral compensation and enhanced physical compensation modules. This stage aims to compress the input image data and learn the two types of degradation compensation modules. Before training begins, the activation signals of the training targets are output through an external gating mechanism to control the training of the two types of compensation modules. The backpropagation gradient flow during the learning process of each module only updates the corresponding degradation module. This learning strategy explicitly decouples the learning of physical degradation and spectral degradation in the parameter space, resolving the entanglement phenomenon that occurs during the learning of different modules.

[0047] The training image data is input into the network and flows to two branches. First, branch one performs preliminary processing by standardizing the information distribution of the RGB color space of the input image data using Z-Score.

[0048]

[0049] in, Channels representing input image data This indicates the adjusted result. For the average value, For the calculation of standard deviation;

[0050] Branch two performs binarization compression on the image data and extracts pixel and channel features. The image data compression process is an approximate calculation process from bit-level image data to full-precision image data, and the formula is:

[0051]

[0052] in, For traditional convolution operations, For binary convolution operations, the process of binarizing image data is handled by a sign function. This means that input image data greater than 0 is compressed to 1, and input image data less than 0 is compressed to 0. This method compresses 32-bit full-precision image data into 1-bit image data, thereby minimizing image data storage space and computational complexity. This is the scaling factor for the weighting term, which recovers information lost due to compression precision to some extent. Finally, these two branches are input to the underwater scene pixel distribution curve for image data recovery, using the following formula:

[0053]

[0054] in, and The scaling and bias factors are learned by the degradation compensation module. The output of the module is represented by the degradation compensation module, which consists of a physical compensation module and a spectral compensation module. The corresponding loss consists of a global loss and a module-specific loss. The global loss calculates the overall structural difference between the predicted image data and the ground truth image data, using the following formula:

[0055]

[0056] in, Calculate the squared average similarity error between the predicted image and the ground truth image. Image similarity is measured from three dimensions: brightness, contrast, and structure.

[0057] The formula for the specificity loss of the spectral compensation module is as follows:

[0058]

[0059] in, Constrain the color space of the image. Supervise the frequency domain of the image;

[0060] The specific loss formula for the physical compensation module is as follows:

[0061]

[0062] in, Monitor the dark channel of image data. The transmission graph of image data is supervised; the specific losses of each module are used to calculate the differences in spectral information and physical transmittance between the predicted image data and the ground truth (GT) data. The global loss avoids the module only learning its own task content after decoupling, thus providing some information supplementation. The spectral compensation module and the physical compensation module have the following advantages in this decoupled and collaborative learning approach: firstly, due to module decoupling, they learn more specific degradation recovery capabilities; secondly, due to alternating collaboration, the information from the two modules complements each other, mitigating the information loss problem caused by data binarization.

[0063] S13: Repeated iterative training and learning process. During the iteration, the spectral compensation module and the physical compensation module alternately and collaboratively optimize according to the preset training threshold until the loss function converges, and the final task-specific degradation compensation module is obtained.

[0064] Step 2: Input the degraded image into the decoupled collaborative enhancement network model and restore the image by cascading modules. Output the enhanced image. The degraded image is processed by the cascaded spectral compensation module and physical compensation module. In this stage, the degradation compensation module has already undergone decoupled collaborative optimization, so the participation of decoupled gating is not required.

[0065] Table 1: Performance Comparison of Mainstream Methods and the Invention

[0066]

[0067] Note: Underlined parts represent the second-best, bolded parts represent the best.

[0068] Table 1 shows that we achieved the best results in both PSNR and SSIM metrics across different datasets, indicating that the recovered images are closer to the labeled images. Experiments on public datasets demonstrate that our invention outperforms mainstream inventions in both quality and efficiency, achieving high performance at the lowest cost and showing great potential in resource-constrained devices.

[0069] Contents not described in detail in this specification are prior art known to those skilled in the art. Although illustrative specific embodiments of the invention have been described above to facilitate understanding by those skilled in the art, it should be understood that the invention is not limited to the scope of the specific embodiments. Various modifications are readily apparent to those skilled in the art as long as they fall within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of this invention are protected.

Claims

1. A decoupled collaborative enhancement method for underwater image degradation factors, characterized in that, The method includes the following steps: Step 1: Design a decoupled collaborative enhancement network model for underwater image degradation factors. The decoupled collaborative enhancement network model includes a decoupling gate, a spectral compensation module, and a physical compensation module. Train the spectral compensation module and the physical compensation module. Step 2: Input the degraded image into the decoupled collaborative enhancement network model and restore the image by cascading modules.

2. The decoupled collaborative enhancement method for underwater image degradation factors according to claim 1, characterized in that, The specific operation of step 1 is as follows: S11: Input the preset training threshold of the current target into the decoupling gating, and when the training reaches the threshold, output the activation signal of the next target, as shown in the formula: in, To decouple the gating, It is a heaviside transition function; it outputs 1 when the input is not less than 0, and 0 otherwise. The training threshold preset for the task objective. For training steps, when training steps Decoupling gate when the value exceeds the threshold. It is in a closed state, and in each training session, the decoupled gating output is used to determine the next target. The activation signal is given by the formula: The output of the decoupling gate is either 0 or 1. When the output of the decoupling gate is 1, the decoupling gate is activated, and the activation signal is... Physical compensation module Training begins. When the output of the decoupling gate is 0, the decoupling gate is closed. The activation signal is... Spectral compensation module Begin training; S12: Training and learning of the spectral and physical compensation modules; The activation signal of the target module and the training image data are input into the corresponding target module respectively to train and learn the target module, thereby obtaining the enhanced spectral compensation module and the enhanced physical compensation module. When training image data is input into the network's spectral compensation module or physical compensation module, it flows to two branches. First, branch one performs preliminary processing, using Z-Score to standardize the information distribution in the RGB color space of the input image data. in, Channels representing input image data This indicates the adjusted result. For the average value, For the calculation of standard deviation; Branch two performs binarization compression on the image data and extracts pixel and channel features. The image data compression process is a calculation process from bit image data to full-precision image data, and the formula is: in, For traditional convolution operations, For binary convolution operations, the process of binarizing image data is handled by a sign function. This means that input image data with a value greater than 0 is compressed to 1, and input image data with a value less than 0 is compressed to 0. These are the scaling factors for the weighting terms. Finally, these two branches are input to the underwater scene pixel distribution curve for image data recovery, using the following formula: in, and The scaling and bias factors are learned by the degradation compensation module. The output of the module is represented by the degradation compensation module, which consists of a physical compensation module and a spectral compensation module. The corresponding loss consists of a global loss and a module-specific loss. The global loss calculates the overall structural difference between the predicted image data and the ground truth image data, using the following formula: in, Calculate the squared average similarity error between the predicted image and the ground truth image. Image similarity is measured from three dimensions: brightness, contrast, and structure. The formula for the specificity loss of the spectral compensation module is as follows: in, Constrain the color space of the image. Supervise the frequency domain of the image; The specific loss formula for the physical compensation module is as follows: in, Monitor the dark channel of image data. Supervise the transmission graph of image data; S13: Repeated iterative training and learning process. During the iteration, the spectral compensation module and the physical compensation module alternately and collaboratively optimize according to the preset training threshold until the loss function converges, and the final task-specific degradation compensation module is obtained.

3. The decoupled collaborative enhancement method for underwater image degradation factors according to claim 2, characterized in that, The specific operation of step 2 is as follows: input the degraded image into the decoupled collaborative enhancement network model, restore the image by cascading modules, and output the enhanced image.

Images