Underwater image degradation diffusion self-supervised enhancement model and building and training method

Abstract

The invention discloses an underwater image degradation diffusion self-supervised enhancement model and a building and training method, and relates to the technical field of underwater image enhancement. The invention provides an underwater image degradation diffusion self-supervised enhancement model and a building and training method for the problem of target detection performance reduction caused by underwater image quality degradation, the model built by the method is combined with the differential thought of a Koschmieder illumination scattering degradation model and a diffusion model, the degradation process of a real underwater environment is simulated, and the target detection performance is improved. Self-supervised training data is generated, dependence of a traditional method on manual annotation data is avoided, image features are extracted through a ResNet50 backbone network, global background light and transmission rate are predicted in parallel, a clear image is reversely recovered by using a physical formula, transmission rate consistency loss, background light consistency loss and reconstruction loss are also proposed, an unsupervised optimization target is constructed, and the self-supervised training data is obtained. And the accuracy and stability of model training are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of underwater image enhancement, in particular to an underwater image degradation diffusion self-supervised enhancement model based on Koschmieder illumination scattering degradation model theory and diffusion model differential thought and a building and training method. BACKGROUND

[0002] Target detection is a key technology in the field of computer vision research, and its task is to locate and classify targets in images. Existing general target detection methods can exhibit excellent detection performance on large-scale high-quality training datasets. However, when faced with underwater datasets with poor quality and scarce annotation information caused by complex underwater environments such as light degradation and blur, the performance of general target detection methods is greatly limited.

[0003] Most existing underwater image enhancement methods based on deep learning rely on data-driven models, which require a large amount of supervised paired dataset during training. The construction process of such datasets is usually complex and costly: researchers first need to collect a large number of original underwater images, which often have typical underwater imaging problems such as color distortion, low contrast, and blurred details; then, researchers invite a large number of volunteers to subjectively evaluate and rank the enhancement results processed by various image enhancement methods (such as histogram equalization, Retinex method, and physical model-based method). In this process, the evaluation criteria are mainly based on the perceptual characteristics of the human visual system, such as color naturalness, contrast, and clarity, and the top-ranked enhancement images are finally selected as the paired training images of the original images.

[0004] However, this data set construction method based on human visual comfort has obvious limitations. First, the evaluation criteria of the human visual system often differ from the needs of computer vision tasks: humans focus more on the overall visual effect and aesthetic quality of images, while underwater target detection and other computer vision tasks focus more on factors that directly affect detection performance, such as target edge clarity and feature distinguishability. Second, this subjective evaluation process is easily influenced by individual differences and environmental factors, leading to inconsistencies and biases in dataset labels. In addition, due to the complexity and diversity of underwater environments, a single evaluation criterion cannot meet the needs of various underwater scenarios.

[0005] Therefore, the images enhanced by deep learning models trained using such datasets tend to improve the subjective perception of human vision, but cannot significantly improve the detection effect for underwater target detection tasks. Specifically, such models may over-enhance the color saturation or contrast of images, improving visual appeal, but may distort target edges or introduce artifacts, thereby reducing target detection accuracy.

[0006] Therefore, there is a need to provide a new solution to the above problems. SUMMARY

[0007] The purpose of the present application is to provide an underwater image degradation diffusion self-supervised enhancement model and a building and training method to solve the technical problems raised in the background art.

[0008] To achieve the above purpose, the present application provides the following technical solution: a building and training method of an underwater image degradation diffusion self-supervised enhancement model, at least comprising the following steps:

[0009] Based on the Koschmieder light scattering degradation model theory and the diffusion model differential idea, the model is built, which adopts an enhancement method based on a physical model, predicts key parameters through a deep learning network, and realizes the clarification of underwater images;

[0010] In the inference stage of the model, an enhancement formula is designed to remove the fogging effect in the input image, restore the details of the scene, and obtain an enhanced image;

[0011] In the model training stage, a forward degradation process and a reverse diffusion process are adopted;

[0012] The forward degradation process simulates the optical degradation mechanism of the real underwater environment to generate high-quality self-supervised training data, gradually superimposes noise and degradation effects based on the Kgschmieder model, generates a degradation image sequence that conforms to the characteristics of the real underwater scene, controls the degradation intensity through the parameterized transmission rate and background light, generates a degradation image sequence that conforms to the characteristics of the real underwater scene, and simulates different underwater environmental conditions;

[0013] The reverse diffusion process combines the inverse derivation of the Kgschmieder formula to realize gradual denoising and image reconstruction, uses the predicted background light and transmission rate parameters to gradually remove noise and reconstruct clear images, and optimizes the key features required for underwater target detection;

[0014] And through the design of three loss functions of transmission rate consistency loss, background light consistency loss and reconstruction loss, the accuracy and stability of the model training are ensured, the transmission rate consistency loss and the background light consistency loss ensure that the predicted parameters are consistent with the forward degradation process, and the reconstruction loss guarantees the quality of the recovered image.

[0015] Further, the inference stage at least comprises the following steps:

[0016] First, the original underwater image is received as input and data preprocessing is performed;

[0017] The preprocessed image is sent to the ResNet50 backbone network for feature extraction;

[0018] After feature extraction, multi-scale feature maps are generated, which are sent to two parallel prediction heads, background light prediction head and transmission rate prediction head;

[0019] The background light prediction head and the transmission rate prediction head share the features extracted by ResNet50, but have independent parameter updating mechanisms to ensure the accuracy of their respective prediction tasks;

[0020] Based on the Koschmieder illumination scattering degradation model, the original formula is transformed to obtain an enhanced formula:

[0021]

[0022] Where J represents the enhanced image; J1 is the input original underwater image; A1 is the background light predicted by the background light prediction head corresponding to the input underwater image J1; t1 is the transmission rate predicted by the transmission rate prediction head corresponding to the input underwater image J1;

[0023] By accurately estimating the background light and transmission rate parameters, the fogging effect in the image can be effectively removed, the details of the scene can be restored, and the enhanced image J can be obtained.

[0024] Further, the data preprocessing at least includes the following steps:

[0025] First, the input image is subjected to data enhancement processing, which at least includes random rotation in an angle range of ±30° and center cropping;

[0026] Then, the image is uniformly scaled to a standard size of 960x960.

[0027] Further, the background light prediction head adopts a fully connected layer structure to predict the background light A of the input image through regression analysis;

[0028] The transmission rate prediction head adopts a convolutional neural network structure to predict the transmission rate t of each pixel.

[0029] Further, the forward degradation process is to combine the Koschmieder formula with the differential idea of the diffusion model to construct a progressive underwater image degradation mechanism;

[0030] The progressive underwater image degradation mechanism simulates the image degradation process in the real underwater environment to provide high-quality self-supervised training data labels for subsequent image enhancement tasks;

[0031] First, a set of random transmission rates t2, t3…t n-1 t nAs degradation parameters, these transmission rate parameters are distributed according to a specified pattern to ensure the continuity and rationality of the degradation process;

[0032] Considering that the atmospheric background light A in the underwater environment is relatively stable, the average value of the RGB three channels of the input underwater image is used as the global atmospheric background light A, and the atmospheric background light A remains constant throughout the entire forward degradation process;

[0033] Based on Koschmieder's formula, the following degeneracy equation was established:

[0034] J2=J1t2+(1-t2)A

[0035] J3=J2t3+(1-t3)A

[0036] =J1t2t3+(1-t2)t3A+(1-t3)A

[0037] =J1t2t3+(1-t2t3)A ......

[0039] J n =J n-1 t n +(1-t n A

[0040] J n =J1(t2t3…t n )+(1-t2t3…t n A

[0041] By using J2 = J1t2 + (1-t2)A, the degraded image J2 after one step can be obtained from the original underwater image J1 using the background light A and the transmission rate t2;

[0042] Subsequently, based on the formula J3=J2t3+(1-t3)A, the degraded image J3 after two degradation steps is obtained from the background light A, the transmission rate t3 and the degraded image J2;

[0043] By repeating this gradual degradation process, the image J after n degradation steps is finally obtained. n .

[0044] Furthermore, the reverse diffusion process employs a deep learning-based progressive recovery strategy;

[0045] The blurred image J obtained by the forward process degradation n The input is fed into the backbone network to extract multi-level features, which are then fed into two key sub-network modules: the background light prediction head and the transmission rate prediction head.

[0046] Through the two prediction heads, the blurred image J n The corresponding predicted background light A n , and the predicted transmission rate t n ;

[0047] It is worth noting that, unlike the forward degradation process, the background light A in the reverse process is no longer simply composed of the mean value of the RGB three channels of the underwater image, but is accurately predicted by the background light prediction head constructed by the deep neural network. This prediction method based on deep learning can better capture the lighting characteristics of the underwater scene, thereby improving the restoration effect, and thus circulates and diffuses to restore the original input underwater image J1;

[0048] In the reverse diffusion process, based on the Koschmieder formula, a complete diffusion recovery equation system is established, and the recovery process follows the following recursive formula relationship:

[0049] ...

[0051]

[0052] Through the blurred image J n obtained by the forward degradation process, combined with the predicted background light A n and the transmission rate t n , the blurred image J n-1 is recovered in reverse;

[0053] Subsequently, by formula the blurred image J n-1 obtained by the forward degradation process, and the background light A n-1 corresponding to the blurred image J n-1 and the transmission rate t n-1 predicted by the blurred image J n-2 , the blurred image J n can be further recovered;

[0054] This process is iterated and iterated, and the image degradation effect is gradually removed, and finally the underwater image J1 is recovered. In this way, the deep learning model learns the ability of underwater image restoration and enhancement;

[0055] And in order to ensure the accuracy and stability of the model training, a self-supervised learning mechanism is introduced in the entire reverse diffusion process;

[0056] The background light A n predicted by the background light prediction head A n-1 … A3A2 sequence and the transmission rate t n predicted by the transmission rate prediction head tn-1 The ...t3t2 sequence will be combined with the global background light during the forward process. and transmission rate sequence Compare;

[0057] Image sequences generated by the forward degradation process are used as self-supervised training data, resulting in a paired dataset that does not require manual annotation.

[0058] Furthermore, the transmission rate consistency loss, background light consistency loss, and reconstruction loss are combined in a weighted manner to jointly supervise and guide the training process of the two parallel prediction heads, ensuring that the model can accurately capture the key features of underwater images.

[0059] Transmission rate consistency loss L t The goal is to constrain the training of the transmission rate predictor so that it predicts the transmission rate t during image reconstruction and the corresponding transmission rate during forward degradation. To maintain consistency, the transmission rate consistency loss L t The mathematical expression is as follows:

[0060]

[0061] in, This represents the i-th degraded image generated during the forward degradation process. The transmission rate, and t i In the reverse diffusion recovery process, the transmission rate predictor determines the recovery based on the recovered image J. i The predicted transmission rate;

[0062] Transmission rate consistency loss function L t By applying L1 norm constraints to the transmission rate generated during forward degradation and backward diffusion recovery of a single underwater image, and summing and averaging all transmission rate differences, the accuracy of transmission rate prediction is ensured.

[0063] Background light uniformity loss L A The aim is to constrain the background light predictor so that the background light A predicted during image reconstruction is similar to the global background light during forward degradation. To maintain consistency, background light consistency loss L A The mathematical expression is as follows:

[0064]

[0065] in, This represents the mean of the RGB three channels of the original underwater image during the forward degradation process, while A... i In the reverse diffusion recovery process, the background light predictor determines the image based on the recovered image J. i The predicted background light;

[0066] background light consistency loss L A By constraining the background light generated in the forward degradation and backward diffusion recovery process of a single underwater image by L1 norm, and averaging all background light differences, the accuracy of background light prediction is ensured;

[0067] reconstruction loss L recon For overall constraint of the quality of the recovered image, in the reverse diffusion process, the reconstruction loss L recon Ensure that the image recovered each time is consistent with the image in the forward degradation process, thereby enhancing the stability of the model and maintaining the balance of the background light predictor and the transmission rate predictor in the bidirectional parallel training, the reconstruction loss L recon The mathematical expression is as follows:

[0068]

[0069] Wherein, The i-th degraded image generated in the forward degradation process is represented by J irecon , and J

[0070] reconstruction loss L recon By constraining the difference between the recovered image and the forward degradation image by L1 norm, and averaging all differences, the high quality and stability of the recovered image are ensured;

[0071] The final enhanced overall loss function is composed of the above three losses weighted combination, its mathematical expression is as follows:

[0072] L enhance = αL t + βL A + γL recon

[0073] Wherein, α, β and γ are weight coefficients of transmission rate consistency loss, background light consistency loss and reconstruction loss respectively.

[0074] An underwater image degradation diffusion self-supervised enhancement model is obtained by the above-mentioned construction and training method of the underwater image degradation diffusion self-supervised enhancement model.

[0075] Compared with the prior art, the beneficial effects of the present application are:

[0076] 1. The present application aims at the problem of underwater image quality degradation leading to target detection performance decline, and proposes an underwater image degradation diffusion self-supervised enhancement model and a building and training method.

[0077] 2. The present application also proposes transmission rate consistency loss, background light consistency loss and reconstruction loss, constructs an unsupervised optimization target, ensures the accuracy and stability of model training, can effectively remove the fogging effect in underwater images, and can also restore the scene detail information, significantly improving the image quality.

[0078] 3. The present application has outstanding effects of mAP50 of 85.2% on DUO underwater target detection data set and mAP50 of 84.4% on RUOD data set compared with other enhancement methods. BRIEF DESCRIPTION OF DRAWINGS

[0079] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0080] Figure 1 The present application is a whole model architecture flowchart.

[0081] Figure 2 The present application is a forward degradation schematic diagram.

[0082] Figure 3 The present application is a reverse diffusion schematic diagram.

[0083] Figure 4 The present application is a DUO data set experimental result visualization schematic diagram. DETAILED DESCRIPTION

[0084] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments.

[0085] Embodiment one:

[0086] Please refer to Figure 1A method for building and training an underwater image degradation diffusion self-supervised enhancement model, comprising at least the following steps:

[0087] The model is built based on the Koschmieder light scattering degradation model theory and the diffusion model differential idea. The model adopts a physical model-based enhancement method, predicts key parameters through a deep learning network, and realizes clear processing of underwater images;

[0088] In the inference stage of the model, an enhancement formula is designed to remove the fogging effect in the input image, restore the details of the scene, and obtain an enhanced image;

[0089] In the model training stage, a forward degradation process and a reverse diffusion process are adopted;

[0090] The forward degradation process simulates the optical degradation mechanism of the real underwater environment to generate high-quality self-supervised training data. Based on the Kgschmieder model, noise and degradation effects are gradually superimposed to generate a degradation image sequence that conforms to the characteristics of real underwater scenes. The degradation intensity is controlled by the parameterized transmission rate and background light to generate a degradation image sequence that conforms to the characteristics of real underwater scenes, simulating different underwater environmental conditions.

[0091] The reverse diffusion process combines the inverse derivation of the Kgschmieder formula to realize gradual denoising and image reconstruction. The predicted background light and transmission rate parameters are used to gradually remove noise and reconstruct clear images, and the key features required for underwater target detection are optimized;

[0092] The design of three loss functions, transmission rate consistency loss, background light consistency loss, and reconstruction loss, ensures the accuracy and stability of the model training. The transmission rate consistency loss and the background light consistency loss ensure that the predicted parameters are consistent with the forward degradation process, and the reconstruction loss guarantees the quality of the recovered images.

[0093] The inference stage comprises at least the following steps:

[0094] First, the original underwater image is received as input and data preprocessing is performed;

[0095] The preprocessed image is sent to the ResNet50 backbone network for feature extraction. The ResNet50 backbone network can effectively extract deep semantic features of the image due to its residual connection structure, while avoiding the problem of gradient disappearance;

[0096] After feature extraction, multi-scale feature maps are generated, which are sent to two parallel prediction heads, the background light prediction head and the transmission rate prediction head;

[0097] The background light prediction head and the transmission rate prediction head share the features extracted by ResNet50, but have independent parameter updating mechanisms to ensure the accuracy of their respective prediction tasks.

[0098] Based on the Koschmieder light scattering degradation model, the original formula is deformed to obtain the enhanced formula:

[0099]

[0100] where J represents the enhanced image; J1 is the input original underwater image; A1 is the background light corresponding to the input underwater image J1 predicted by the background light prediction head; t1 is the transmission rate corresponding to the input underwater image J1 predicted by the transmission rate prediction head;

[0101] By accurately estimating the background light and transmission rate parameters, the fogging effect in the image can be effectively removed, the details of the scene can be restored, and the enhanced image J can be obtained.

[0102] The data preprocessing at least includes the following steps:

[0103] First, the input image is subjected to data enhancement processing, which at least includes random rotation in an angle range of ±30° and center cropping;

[0104] Subsequently, the image is uniformly scaled to a standard size of 960x960.

[0105] The background light prediction head adopts a fully connected layer structure to predict the background light A of the input image through regression analysis;

[0106] The transmission rate prediction head adopts a convolutional neural network structure to predict the transmission rate t of each pixel.

[0107] The forward degradation process is to combine the Koschmieder formula with the differential idea of the diffusion model to construct a progressive underwater image degradation mechanism;

[0108] The progressive underwater image degradation mechanism simulates the image degradation process in the real underwater environment to provide high-quality self-supervised training data labels for subsequent image enhancement tasks;

[0109] First, a set of random transmission rates t2, t3…t n-1 t n As degradation parameters, these transmission rate parameters are distributed according to a specified rule to ensure the continuity and reasonableness of the degradation process;

[0110] Considering the relative stability of the atmospheric background light A in underwater environment, the average value of the RGB three channels of the input underwater image is used as the global atmospheric background light A, and the atmospheric background light A remains constant throughout the entire forward degradation process. This design not only conforms to the physical law, but also simplifies the calculation process;

[0111] Based on the Koschmieder formula, the following degradation equation is established:

[0112] J2 = J1t2 + (1-t2)A

[0113] J3 = J2t3 + (1-t3)A

[0114] = J1t2t3 + (1-t2)t3A + (1-t3)A

[0115] = J1t2t3 + (1-t2t3)A ......

[0117] J n = J n-1 t n +(1-t n )A

[0118] J n = J1(t2t3…t n )+(1-t2t3…t n )A

[0119] Through J2 = J1t2 + (1-t2)A, the degradation image J2 after one step of degradation can be obtained from the original underwater image J1 using the background light A and the transmission rate t2;

[0120] Subsequently, based on the formula J3 = J2t3 + (1-t3)A, the degradation image J3 after two steps of degradation is obtained from the background light A, the transmission rate t3 and the degradation image J2;

[0121] Through the cycle of this progressive degradation process, the image Jn after n steps of degradation is finally obtained n ;

[0122] It is worth noting that the intermediate results J2J3…J n-2 J n-1 have important training value, which will be used as self-supervised labels for subsequent backward diffusion recovery process, providing rich learning samples for the model;

[0123] And through the mathematical derivation of the above formula, it is proved that the Koschmieder formula has recursion under the condition of constant background light A;

[0124] This important property shows that the degradation image Jn after n steps of degradation can be obtained through the transmission rate sequence t2t3…tn-1 t n and the background light A directly utilizes the formula J n = J1(t2t3…t n )+(1-t2t3…t n )A to degrade the original underwater image J1 to J n in one step, thereby reducing the model calculation amount. The model does not have to spend extra storage space to save the intermediate results generated in the forward degradation process, thereby avoiding additional storage overhead and reducing the dependence on computer memory and video memory during model training.

[0125] This innovative forward degradation mechanism not only improves the training efficiency of the model, but also ensures the physical reasonableness of the degradation process. By precisely controlling the transmission rate sequence and the background light parameters, the model can generate degraded images that conform to the characteristics of real underwater environments, laying a solid foundation for subsequent image enhancement tasks. In addition, this design also has good scalability, which can adjust the degradation step number n and the transmission rate sequence according to specific application scenarios to adapt to different underwater imaging conditions and enhancement needs. For example, in deep-sea environments, the degradation step number can be increased to simulate stronger light attenuation effects; while in shallow water areas, the degradation step number can be reduced to maintain image clarity. This flexibility enables the model to adapt to various complex underwater environments, providing greater convenience for practical applications.

[0126] The reverse diffusion process adopts a progressive recovery strategy based on deep learning;

[0127] The blurred image J n degraded by the forward process is input into the backbone network to extract multi-level features, which are then fed into two key sub-network modules, namely the background light prediction head and the transmission rate prediction head.

[0128] Through these two prediction heads, the predicted background light A n and the predicted transmission rate t n corresponding to the blurred image J n can be accurately estimated;

[0129] It is worth noting that, unlike the forward degradation process, the background light A in the reverse process is no longer simply composed of the mean value of the RGB three channels of the underwater image, but is accurately predicted by the background light prediction head constructed by the deep neural network. This deep learning-based prediction method can better capture the lighting characteristics of underwater scenes, thereby improving the recovery effect, and thus repeatedly diffuse and recover the original input underwater image J1.

[0130] In the backward diffusion process, a complete diffusion recovery equation system is established based on the reverse derivation of Koschmieder formula, and the recovery process follows the following recursive formula relationship:

[0131] ......

[0133]

[0134] Through the blurred image J n obtained by the forward degradation process, combined with the predicted background light A n and the transmission rate t n , the blurred image J n-1 is recovered reversely;

[0135] Then, by formula the blurred image J n-1 obtained by the forward degradation process, and the background light A n-1 and the transmission rate t n-1 corresponding to the blurred image J n-1 are predicted, the blurred image J n-2 can be further recovered;

[0136] This process is iterated in a loop, and the image degradation effect is gradually removed, and finally the underwater image J1 is recovered. In this way, the deep learning model learns the ability of underwater image recovery and enhancement;

[0137] And in order to ensure the accuracy and stability of the model training, a self-supervised learning mechanism is introduced in the whole backward diffusion process;

[0138] The background light A n A n-1 …A3A2 sequence predicted by the background light prediction head and the transmission rate t n t n-1 …t3t2 sequence predicted by the transmission rate prediction head will be compared with the global background light and the transmission rate sequence in the forward process;

[0139] The image sequence generated by the forward degradation process is used as self-supervised training data, without the need for manually labeled paired data set;

[0140] The difference between the predicted and true values ​​will be calculated using a subsequently proposed composite loss function, including background light consistency loss, transmission rate consistency loss, and reconstruction loss, thereby achieving comprehensive supervision and optimization of the model. This multi-layered supervision mechanism not only ensures the stability of the model during training but also significantly improves the quality and realism of the final reconstructed image.

[0141] Transmission rate consistency loss, background light consistency loss, and reconstruction loss are weighted and combined to jointly supervise and guide the training process of the two parallel prediction heads, ensuring that the model can accurately capture the key features of underwater images.

[0142] Transmission rate consistency loss L t The goal is to constrain the training of the transmission rate predictor so that it predicts the transmission rate t during image reconstruction and the corresponding transmission rate during forward degradation. To maintain consistency, the transmission rate consistency loss L t The mathematical expression is as follows:

[0143]

[0144] in, This represents the i-th degraded image generated during the forward degradation process. The transmission rate, and t i In the reverse diffusion recovery process, the transmission rate predictor determines the recovery based on the recovered image J. i The predicted transmission rate;

[0145] Transmission rate consistency loss function L t By applying L1 norm constraints to the transmission rate generated during forward degradation and backward diffusion recovery of a single underwater image, and summing and averaging all transmission rate differences, the accuracy of transmission rate prediction is ensured.

[0146] Background light uniformity loss L A The aim is to constrain the background light predictor so that the background light A predicted during image reconstruction is similar to the global background light during forward degradation. To maintain consistency, background light consistency loss L A The mathematical expression is as follows:

[0147]

[0148] in, This represents the mean of the RGB three channels of the original underwater image during the forward degradation process, while A... i In the reverse diffusion recovery process, the background light predictor determines the image based on the recovered image J. i The predicted background light;

[0149] Background light consistency loss L A By constraining the L1 norm of the background light generated in the forward degradation and backward diffusion recovery process of a single underwater image, and averaging all background light differences, the accuracy of background light prediction is ensured;

[0150] Reconstruction loss L recon For overall constraint of the quality of the recovered image, in the backward diffusion process, the reconstruction loss L recon Ensure that the image recovered each time is consistent with the image in the forward degradation process, thereby enhancing the stability of the model and maintaining the balance of the background light predictor and the transmission rate predictor in the bidirectional parallel training, the reconstruction loss L recon The mathematical expression is as follows:

[0151]

[0152] Wherein, The i-th degraded image generated in the forward degradation process is represented by Ii, and The image generated in the backward diffusion recovery process is represented by Ii;

[0153] Reconstruction loss L recon By constraining the L1 norm of the difference between the recovered image and the forward degraded image, and averaging all differences, the high quality and stability of the recovered image are ensured;

[0154] The final enhanced overall loss function is composed of the above three losses weighted combination, the mathematical expression is as follows:

[0155] L enhance =αL t +βL A +γL recon

[0156] Wherein, α, β and γ are the weight coefficients of the transmission rate consistency loss, the background light consistency loss and the reconstruction loss respectively. When α = 0.35, β = 0.05, γ = 0.6, the trained model achieves the best state of underwater image enhancement. Experimental results show that the loss function combination not only significantly improves the quality of underwater images, but also significantly improves the results of underwater target detection.

[0157] Embodiment two:

[0158] An underwater image degradation diffusion self-supervised enhancement model is obtained by the construction and training method of the underwater image degradation diffusion self-supervised enhancement model in the above embodiment one.

[0159] Based on the above two embodiments, the following experiments are proposed:

[0160] Training details and parameter settings

[0161] The proposed model will be evaluated and compared based on two public underwater target detection datasets, DUO and RUOD. The DUO dataset, a medium-sized underwater image dataset, contains 7782 underwater images, divided into 6671 training images and 1111 test images in an 8:2 ratio, covering four typical underwater biological categories: sea cucumber, sea urchin, scallop, and starfish. The RUOD dataset is a larger and more challenging dataset, containing 14000 underwater images, divided into 9799 training images and 4201 test images in a 7:3 ratio. In addition to the four categories in the DUO dataset, it also includes six new categories: fish, coral, diver, squid, turtle, and jellyfish, totaling ten categories, which better simulate real complex underwater scenes.

[0162] In terms of model evaluation, this study uses standard underwater image enhancement indicators and target detection indicators for multi-dimensional evaluation. Specifically, in terms of underwater image quality evaluation, two widely recognized objective evaluation indicators, UCIQE (Underwater Color Image Quality Evaluation) and UIQM (Underwater Image Quality Measure), are used to quantitatively evaluate the image enhancement effect from color restoration, sharpness, and contrast. In terms of target detection performance evaluation, mAP (mean Average Precision) is used as the core evaluation indicator, and the average precision at different IoU thresholds is calculated to comprehensively reflect the performance of the model in the target detection task.

[0163] To ensure the fairness and repeatability of the experiments, the experimental setup was standardized in this study. During the training process of the image enhancement model, the input images were uniformly scaled to a resolution of 960x960, the batch size was set to 1, the initial learning rate was 0.0001, the Adam optimizer was used for adaptive adjustment, and the training period was 10 epochs. In the training process of the target detection model, to maintain consistency, the input images were also scaled to a resolution of 960x960, the batch size was set to 4, the initial learning rate was 0.0001, the Adam optimizer was used for adaptive adjustment, and the training period was extended to 50 epochs to ensure that the target detection model was fully converged. All experiments were conducted in a unified software and hardware environment: the operating system was Ubuntu 20.04LTS, the programming language was Python 3.10.0, the deep learning framework was PyTorch 1.12.1, and the hardware platform was equipped with NVIDIA Tesla V100 GPU (32GB memory) to ensure efficient execution of the experiments and repeatability of the results.

[0164] Experimental results and analysis

[0165] To verify the importance of each component in the proposed model in this chapter, a series of comparative experiments were conducted on the DUO and RUOD datasets using the complete model.

[0166] First, through experiments, the selection of the generation method of the transmission rate t in the forward degradation process and the influence of the two key parameters, the degradation step number n, on the overall effect of the model were discussed. The experimental results are shown in Tables 1 and 2.

[0167] Table 1 Experimental results of different parameter selections on the DUO dataset

[0168]

[0169] Table 2 Experimental results of different parameter selections on the RUOD dataset

[0170]

[0171] Selection of degradation step number n: The experimental results in Tables 1 and 2 show that there is a significant trade-off between the value of the degradation step number n and the model performance and processing efficiency. As the degradation step number n increases, the model processing time increases approximately linearly. From n = 10 to n = 200, the processing time increases from 3s to 102s, while the detection performance improvement shows a marginal decreasing effect. Especially when n > 50, the performance improvement tends to be flat, but the computational cost increases significantly.

[0172] When n = 50, the model achieves the best detection effect on both data sets: on the DUO data set, the mAP50 of YOLO v5 reaches 85.5, and the mAP50 of YOLO v8 reaches 84.3; on the RUOD data set, the mAP50 of YOLO v5 reaches 84.8, and the mAP50 of YOLO v8 reaches 83.5. However, when n = 20, although the detection performance decreases slightly (the mAP50 of YOLO v5 on the DUO data set is 85.2, and the mAP50 of YOLO v8 is 84.1; the mAP50 of YOLO v5 on the RUOD data set is 84.4, and the mAP50 of YOLO v8 is 83.2), the processing time is significantly reduced from 26s to 8s, with an efficiency improvement of more than 200%. Considering the model performance and computational efficiency, n = 20 is finally selected as the optimal degradation step.

[0173] Selection of transmission rate t generation method: The experiment compares the generation methods of linear (0-1 linear value) and nonlinear (sin(0-90°) value) transmission rate sequences. The experimental results in Tables 1 and 2 show that the nonlinear generation method can achieve better detection results in most experimental configurations. For example, on the DUO data set, when n = 50, the nonlinear method improves the mAP50 of YOLO v5 from 85.3 to 85.5, and the mAP50 of YOLO v8 from 83.8 to 84.3; on the RUOD data set, under the same configuration, the mAP50 of YOLO v5 is improved from 84.4 to 84.8, and the mAP50 of YOLO v8 is improved from 83.4 to 83.5. This performance improvement may be due to the nonlinear generation method being able to better simulate the degradation process in real scenes.

[0174] At the same time, in order to comprehensively evaluate the superiority of the method proposed in this chapter, we select the underwater image enhancement algorithms with excellent performance in recent years for comparison experiments. The experimental results are shown in Tables 3 and 4.

[0175] The method proposed in the present application exhibits significant performance advantages on both data sets. On the DUO data set, when using the YOLO v5 detector, the method leads other methods with an mAP50 of 85.2 and an mAP50:95 of 64.3; when using the YOLOv8, it also ranks first with an mAP50 of 84.1 and an mAP50:95 of 64.0. On the RUOD data set, the method also remains leading, with an mAP50 of 84.4 under the YOLO v5 detector and an mAP50 of 83.2 under the YOLO v8 detector.

[0176] It is worth noting that the original dataset without enhancement is superior to some of the enhanced methods in certain indicators. For example, on the DUO dataset, the YOLO v5 detection result of the original data (mAP50 = 84.9) is superior to the enhanced methods such as UGAN, U-shape, RAUNEnet, and FUnIE_GAN. This phenomenon shows that not all enhancement methods can effectively improve the detection performance, and some methods may even introduce noise or distortion, resulting in performance degradation. The method proposed in this application successfully improves the performance by optimizing the key features of the underwater target detection task. This improvement comes from the following aspects: (1) better preservation of key features of the target; (2) more effective suppression of underwater environmental noise; (3) more reasonable balance between enhancement strength and feature preservation.

[0177] Table 3 Detection results of the DUO dataset after enhancement by different methods

[0178]

[0179]

[0180] Table 4 Detection results of the RUOD dataset after enhancement by different methods

[0181]

[0182] Finally, in order to comprehensively evaluate the effect of the method proposed in this chapter on the improvement of underwater image quality, the UICQE and UIQM two widely used objective evaluation indicators are selected, and comparative experiments are conducted with different underwater image enhancement algorithms. The experimental results are shown in Table 5.

[0183] The experimental results show that the method proposed in this application has a significant improvement in UICQE and UIQM indicators compared with the original dataset, although it is not as good as some methods that focus on image quality improvement (such as WaterNet and RAUNEnet). This gap reflects the difference in design goals between the methods. Our method focuses more on the optimization of underwater target detection related features, rather than simply improving visual quality.

[0184] Table 5 Enhancement results of different methods on DUO and RUOD datasets

[0185]

[0186]

[0187] Visual structure

[0188] In order to visually demonstrate the enhancement effect of the method proposed in this application, we conducted a visual analysis of the enhancement results of the DUO dataset, as shown in Figure 4As shown. By comparing the original image with the enhanced image, it can be clearly observed that the enhanced image significantly improves the visual features of the target to be detected. In terms of color representation, the color contrast of the target object is significantly improved, and the originally blurred color boundary becomes clearer. In terms of edge features, the sharpness of the target contour is significantly improved, and the detailed features are more prominent. In terms of texture information, the subtle texture originally hidden by the underwater environment noise is revealed. In addition to the enhancement of the target features, the overall image quality is also significantly improved. The color balance is optimized, and the color cast problem caused by the underwater environment is eliminated. The image clarity is improved, and the blurring degree is obviously reduced. The contrast enhancement makes the scene levels more distinct.

[0189] It will be apparent to those skilled in the art that the application is not limited to the details of the above-exemplified embodiments and that the application can be implemented in other particular forms without departing from the spirit or essential characteristics of the application. The presently disclosed embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. No feature of the application is to be construed as limiting the scope of the claims to their precise form.

Claims

1. A method for constructing and training a self-supervised enhancement model for underwater image degradation diffusion, characterized in that: At least the following steps are included: Based on the Koschmieder light scattering degradation model theory and the differential idea of ​​diffusion model, a model is built. This model adopts a physical model-based enhancement method and uses a deep learning network to predict key parameters to achieve the sharpening of underwater images. During the inference phase of the model, an enhancement formula is designed to remove the fog effect from the input image, restore the details of the scene, and obtain an enhanced image. Forward degradation and backward diffusion processes are employed during the model training phase. The forward degradation process simulates the optical degradation mechanism of the real underwater environment, generating high-quality self-supervised training data. Based on the Kgschmieder model, noise and degradation effects are gradually superimposed to generate a degradation image sequence that conforms to the characteristics of the real underwater scene. By parameterizing the transmission rate and background light to control the degradation intensity, a degradation image sequence that conforms to the characteristics of the real underwater scene is generated to simulate different underwater environmental conditions. The reverse diffusion process, combined with the reverse derivation of the Kgschmieder formula, achieves stepwise denoising and image reconstruction. By utilizing the predicted background light and transmission rate parameters, noise is gradually removed and a clear image is reconstructed, and key features required for underwater target detection are specifically optimized. The accuracy and stability of model training are ensured by designing three loss functions: transmission rate consistency loss, background light consistency loss, and reconstruction loss. Transmission rate consistency loss and background light consistency loss ensure that the predicted parameters are consistent with the forward degradation process, while reconstruction loss guarantees the quality of the restored image.

2. The method for constructing and training an underwater image degradation diffusion self-supervised enhancement model according to claim 1, characterized in that: The reasoning phase includes at least the following steps: First, the raw underwater images are received as input, and then data preprocessing is performed. The preprocessed image is then fed into the ResNet50 backbone network for feature extraction. After feature extraction, multi-scale feature maps are generated. These multi-scale feature maps are then fed into two parallel prediction heads: a background light prediction head and a transmission rate prediction head. The background light prediction head and the transmission rate prediction head share the features extracted by ResNet50, but have independent parameter update mechanisms to ensure the accuracy of their respective prediction tasks. Based on the Koschmieder light scattering degradation model, the original formula is transformed to obtain the enhanced formula: Where J represents the enhanced image; J1 is the input original underwater image; A1 is the background light corresponding to the input underwater image J1 predicted by the background light prediction head; t1 is the transmission rate corresponding to the input underwater image J1 predicted by the transmission rate prediction head. By accurately estimating background light and transmission rate parameters, the fogging effect in the image can be effectively removed, scene details can be restored, and an enhanced image J can be obtained.

3. The method for constructing and training an underwater image degradation diffusion self-supervised enhancement model according to claim 2, characterized in that: The data preprocessing includes at least the following steps: First, the input image is subjected to data augmentation processing, which includes at least random rotation within an angle range of ±30° and center cropping; The images were then uniformly scaled to a standard size of 960×960.

4. The method for constructing and training an underwater image degradation diffusion self-supervised enhancement model according to claim 2, characterized in that: The background light prediction head uses a fully connected layer structure to predict the background light A of the input image through regression analysis. The transmission rate prediction head uses a convolutional neural network structure to predict the transmission rate t of each pixel.

5. The method for constructing and training an underwater image degradation diffusion self-supervised enhancement model according to claim 2, characterized in that: The forward degradation process combines the Koschmieder formula with the differential idea of ​​the diffusion model to construct a progressive underwater image degradation mechanism. The progressive underwater image degradation mechanism simulates the image degradation process in a real underwater environment, providing high-quality self-supervised training data labels for subsequent image enhancement tasks. First, generate a set of random transmission rates t2t3…t n-1 t n As degradation parameters, these transmission rate parameters are distributed according to a specified pattern to ensure the continuity and rationality of the degradation process; Considering that the atmospheric background light A in the underwater environment is relatively stable, the average value of the RGB three channels of the input underwater image is used as the global atmospheric background light A, and the atmospheric background light A remains constant throughout the entire forward degradation process; Based on Koschmieder's formula, the following degeneracy equation was established: J2=J1t2+(1-t2)A J3=J2t3+(1-t3)A =J1t2t3+(1-t2)t3A+(1-t3)A =J1t2t3+(1-t2t3)A ...... J n =J n-1 t n +(1-t n )A J n =J1(t2t3…t n )+(1-t2t3…t n )A By using J2 = J1t2 + (1-t2)A, the degraded image J2 after one step can be obtained from the original underwater image J1 using the background light A and the transmission rate t2; Subsequently, based on the formula J3=J2t3+(1-t3)A, the degraded image J3 after two degradation steps is obtained from the background light A, the transmission rate t3 and the degraded image J2; By repeating this gradual degradation process, the image J after n degradation steps is finally obtained. n .

6. The method for constructing and training an underwater image degradation diffusion self-supervised enhancement model according to claim 5, characterized in that: The reverse diffusion process employs a deep learning-based progressive recovery strategy; The blurred image J obtained by the forward process degradation n The input is fed into the backbone network to extract multi-level features, which are then fed into two key sub-network modules: the background light prediction head and the transmission rate prediction head. Using these two prediction heads, the blurred image J can be accurately estimated. n The corresponding predicted background light A n and predicted transmission rate t n ; It is worth noting that, unlike the forward degradation process, the background light A in the reverse process is no longer simply composed of the average of the three RGB channels of the underwater image. Instead, it is accurately predicted by a background light prediction head built by a deep neural network. This deep learning-based prediction method can better capture the lighting characteristics of the underwater scene, thereby improving the restoration effect. This method is used to repeatedly diffuse and restore the original input underwater image J1. In the reverse diffusion process, a complete system of diffusion recovery equations was established by inverse derivation based on the Koschmieder formula. The recovery process follows the recursive formula relationship: ...... pass Blurred images J obtained using the forward degradation process n Combined with the predicted background light A n and transmission rate t n The blurred image J is recovered in reverse. n-1 ; Subsequently, through the formula Blurred image J obtained using the forward degradation process n-1 , and the blurred image J n-1 Predict its corresponding background light A n-1 and transmission rate t n-1 It can further recover the blurred image J n-2 ; This process is repeated iteratively to gradually remove the image degradation effect and finally recover the underwater image J1. In this way, the deep learning model learns the ability to restore and enhance underwater images. Furthermore, to ensure the accuracy and stability of model training, a self-supervised learning mechanism is introduced throughout the back diffusion process; Background light A predicted by the background light prediction head n A n-1 …Transmission rate t predicted by the A3A2 sequence and transmission rate prediction header n t n-1 The ...t3t2 sequence will be combined with the global background light during the forward process. and transmission rate sequence Compare; Image sequences generated by the forward degradation process are used as self-supervised training data, resulting in a paired dataset that does not require manual annotation.

7. The method for constructing and training an underwater image degradation diffusion self-supervised enhancement model according to claim 6, characterized in that: The transmission rate consistency loss, background light consistency loss, and reconstruction loss are weighted and combined to jointly supervise and guide the training process of the two parallel prediction heads, ensuring that the model can accurately capture the key features of underwater images. Transmission rate consistency loss L t The goal is to constrain the training of the transmission rate predictor so that it predicts the transmission rate t during image reconstruction and the corresponding transmission rate during forward degradation. To maintain consistency, the transmission rate consistency loss L t The mathematical expression is as follows: in, This represents the i-th degraded image generated during the forward degradation process. The transmission rate, and t i In the reverse diffusion recovery process, the transmission rate predictor determines the recovery based on the recovered image J. i The predicted transmission rate; Transmission rate consistency loss function L t By applying L1 norm constraints to the transmission rate generated during forward degradation and backward diffusion recovery of a single underwater image, and summing and averaging all transmission rate differences, the accuracy of transmission rate prediction is ensured. Background light uniformity loss L A The aim is to constrain the background light predictor so that the background light A predicted during image reconstruction is similar to the global background light during forward degradation. To maintain consistency, background light consistency loss L A The mathematical expression is as follows: in, This represents the mean of the RGB three channels of the original underwater image during the forward degradation process, while A... i In the reverse diffusion recovery process, the background light predictor determines the image based on the recovered image J. i The predicted background light; Background light uniformity loss L A By applying L1 norm constraints to the background light generated during forward degradation and backward diffusion recovery of a single underwater image, and summing and averaging all background light differences, the accuracy of background light prediction is ensured. Reconstruction loss L recon Used to constrain the overall quality of image recovery, during backdiffusion, the reconstruction loss L recon To enhance model stability, the recovered image is kept consistent with the image from the forward degradation process. This also maintains a balance between the background light predictor and the transmittance predictor during bidirectional parallel training. The reconstruction loss L... recon The mathematical expression is as follows: in, Let represent the i-th degraded image generated during the forward degradation process, and It is the image generated during the reverse diffusion recovery process; Reconstruction loss L recon By applying L1 norm constraints to the differences between the restored image and the forward-degraded image, and summing and averaging all the differences, the high quality and stability of the restored image are ensured. The final augmented overall loss function is a weighted combination of the three losses mentioned above, and its mathematical expression is as follows: L enhance =αL t +βL A +γL recon Where α, β and γ are the weighting coefficients for transmission rate consistency loss, background light consistency loss and reconstruction loss, respectively.

8. A self-supervised enhancement model for underwater image degradation diffusion, characterized in that: This is obtained through the construction and training method of the underwater image degradation diffusion self-supervised enhancement model as described in any one of claims 1-7.

Images