Underwater crack image enhancement method and system based on data migration and frequency domain decoupling

Abstract

The present invention relates to a method and system for underwater crack image enhancement based on data migration and frequency domain decoupling, belonging to the technical field of underwater crack image enhancement. The method comprises: constructing an ocean image source domain dataset D s With the underwater crack target domain dataset D t The conditional diffusion model based on the Transformer architecture is trained, and the implicit mapping relationship between underwater optical degradation and clear images is learned by training the conditional diffusion model based on the Transformer architecture through the ocean image source domain dataset; the trained conditional diffusion model based on the Transformer architecture is applied to the target domain through the transfer learning framework, and underwater crack image enhancement is achieved through cross-domain feature transfer. The frequency domain signal in the enhanced underwater crack image is decomposed and reconstructed, and the crack information in the underwater crack image is further enhanced to obtain the final enhanced underwater crack image. The present invention can effectively solve the problems of underwater image blur, low contrast and insufficient samples.

Description

Technical Field

[0001] The present invention belongs to the technical field of underwater crack image enhancement, and in particular relates to an underwater crack image enhancement method and system based on data migration and frequency domain decoupling. Background Art

[0002] Underwater structural cracks are a common hazard in water conservancy projects, primarily caused by environmental erosion and material aging. Traditional detection methods rely primarily on manual diving inspections and underwater robotic inspections, but these methods suffer from inefficiencies and poor image quality. Manual inspections are limited by operating depth and visibility, while images captured by robotic inspections often exhibit blurriness and low contrast due to factors such as water turbidity and light scattering.

[0003] Existing underwater fracture image processing faces two major challenges: First, the complex underwater environment causes image color distortion and blurred details, making traditional enhancement methods prone to over-enhancement. Second, the scarcity of high-quality underwater fracture samples limits the effectiveness of deep learning models. Current transfer learning-based solutions often rely on a single data source, lacking adaptability to the unique optical properties of underwater environments and failing to meet practical engineering needs. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the existing technology and provide an underwater crack image enhancement method and system based on data migration and frequency domain decoupling, which can effectively solve the problems of underwater image blur, low contrast and insufficient samples.

[0005] To achieve the above objectives, the technical solution of the present invention is: a method for underwater crack image enhancement based on data migration and frequency domain decoupling, comprising:

[0006] S1. Multi-source dataset construction: Constructing ocean image source domain dataset D s With the underwater crack target domain dataset D t ,The ocean image source domain dataset contains paired degraded and clear ocean images, and the underwater crack target domain dataset contains real underwater crack images. The source domain and target domain meet the cross-domain data difference;

[0007] S2. Conditional Diffusion Model Training: Train a conditional diffusion model based on the Transformer architecture. This model is trained on an ocean image source domain dataset to learn the implicit mapping relationship between underwater optical degradation and clear images. The conditional diffusion model based on the Transformer architecture uses a non-uniform skip sampling strategy to accelerate the inference process.

[0008] S3. Cross-domain transfer enhancement: The trained conditional diffusion model based on the Transformer architecture is applied to the target domain through a transfer learning framework. Underwater fracture image enhancement is achieved through cross-domain feature transfer. During the back-diffusion process, the model uses the original fracture image as the conditional input, gradually removes noise and restores image details, achieving cross-domain underwater fracture image enhancement and obtaining enhanced underwater fracture images.

[0009] S4, turbidity adaptive frequency domain decoupling enhancement: turbidity adaptive frequency domain feature decoupling of the underwater crack image is performed, and the frequency domain signal in the underwater crack image enhanced in step S3 is decomposed and reconstructed to further enhance the crack information in the underwater crack image, realize the dual-stage enhancement of the underwater crack image, and obtain the final enhanced underwater crack image.

[0010] Furthermore, in S1, the ocean image source domain dataset D s Paired ocean images selected from a public dataset are represented as follows:

[0011]

[0012] in For degraded ocean images, is the corresponding clear image, N is D s The total number of data in;

[0013] Underwater crack target domain dataset D t , which is expressed as follows:

[0014]

[0015] in is a real underwater crack image, M is D t The total number of data in;

[0016] The source domain and the target domain satisfy cross-domain data differences, namely:

[0017]

[0018] in is the ocean image source domain dataset D s The probability distribution of images in ; The target domain dataset D for underwater cracks t The probability distribution of the image in , Represents ocean imagery; Represents an underwater crack image.

[0019] Furthermore, in S1, the underwater crack target domain dataset is collected by building a controllable underwater imaging platform, and a concrete crack specimen made of concrete is placed in the underwater imaging platform.

[0020] Furthermore, S2 specifically includes:

[0021] S2.1. Conditional diffusion model based on Transformer architecture learns the implicit mapping relationship between underwater optical degradation and clear images, and builds a forward noise prediction network , forward noise prediction network The objective function for:

[0022]

[0023] in is the noisy image at time step t, and the total step length is T; is the input image; is Gaussian noise; Enter the condition; It is an expectation operation on the distribution of image data and time steps; is the noise scheduling coefficient, and the cosine scheduling strategy is adopted:

[0024]

[0025] S2.2. The back diffusion process restores pure Gaussian noise to the target image by predicting and eliminating the potential noise distribution in the image. The improved DDIM acceleration algorithm is adopted, and the non-uniform skip sampling strategy is used: the T-step back diffusion process is divided into two intervals [0,n] and [n,T]. Non-uniform sampling with a step size of d1 is used in the first n steps, and non-uniform sampling with a step size of d2 is used in steps n to T.

[0026] Furthermore, S3 specifically includes:

[0027] S3.1. Data preprocessing: For the ocean image source domain dataset D s With the underwater crack target domain dataset D t The images in the dataset are uniformly subjected to resolution adjustment and [-1, 1] pixel value normalization, with ocean underwater image enhancement as the source task and underwater crack image enhancement as the target task.

[0028] S3.2. Migrate the trained conditional diffusion model to the target domain and use feature alignment loss Perform model fine-tuning:

[0029]

[0030] in represent the source domain features and target domain features respectively, is the feature mapping function; represents the maximum mean difference;

[0031] S3.3, in the reverse diffusion process, the original target domain underwater crack image As the conditional input, it gradually removes noise and restores image details to generate enhanced underwater crack images. .

[0032] Furthermore, S4 specifically includes:

[0033] S4.1. Underwater crack image enhanced in step S3 Frequency domain decomposition using turbidity adaptive selection mechanism:

[0034]

[0035] in, Represents wavelet basis function The discrete wavelet transform of is the fast Fourier transform; is the estimated value of current water turbidity, is the turbidity threshold, is the crack image after frequency domain decomposition, which includes the high-frequency component H F With the low frequency component L F ;

[0036] S4.2, the high frequency component H after decomposition F Perform adaptive enhancement:

[0037]

[0038] in, is the turbidity-related gain coefficient, is the turbidity optimization coefficient; is a Gaussian smoothing kernel used to suppress noise amplification;

[0039] S4.3. Reconstruct the image by inverse discrete wavelet transform (IDWT) or inverse fast Fourier transform (FFT) to obtain the final enhanced underwater crack image. .

[0040] The present invention also provides an underwater crack image enhancement system based on data migration and frequency domain decoupling, comprising:

[0041] Data acquisition module: Constructing ocean image source domain dataset D s With the underwater crack target domain dataset D t ,The ocean image source domain dataset contains paired degraded and clear ocean images, and the underwater crack target domain dataset contains real underwater crack images. The source domain and target domain meet the cross-domain data difference;

[0042] Model training module: Trains a conditional diffusion model based on the Transformer architecture. This model is trained using ocean image source domain datasets to learn the implicit mapping relationship between underwater optical degradation and clear images.

[0043] Image enhancement module: The trained conditional diffusion model based on the Transformer architecture is applied to the target domain through a transfer learning framework. Underwater crack images are enhanced through cross-domain feature transfer. The frequency domain signals in the enhanced underwater crack images are decomposed and reconstructed to further enhance the crack information in the underwater crack images, resulting in the final enhanced underwater crack images.

[0044] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the program.

[0045] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0046] Compared to existing technologies, this invention offers the following advantages: By constructing a cross-domain dataset consisting of clear and degraded ocean images and real underwater fracture images, this method trains a Transformer-based conditional diffusion model and utilizes transfer learning to remove noise and restore detail in underwater fracture images. Furthermore, it incorporates turbidity-adaptive frequency-domain decomposition technology to dynamically enhance fracture features. The system integrates multi-sensor data acquisition, model training, real-time enhancement, and quality assessment modules, effectively addressing issues such as blurry underwater images, low contrast, and insufficient sample size. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] FIG1 is a flow chart of the method of the present invention;

[0048] FIG2 is a structural diagram of the conditional diffusion model based on the Transformer architecture provided by the present invention;

[0049] FIG3 is a diagram showing the learning process of the conditional diffusion model trained on the ocean image dataset provided by the present invention;

[0050] FIG4 is a diagram of the underwater crack image enhancement process provided by the present invention. DETAILED DESCRIPTION

[0051] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0052] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0053] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form, and it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0054] The present invention provides an underwater crack image enhancement method based on data migration and frequency domain decoupling to achieve cross-domain underwater crack image enhancement. Figure 1 As shown, the method of the present invention comprises the following steps:

[0055] S1. Multi-source dataset construction: Constructing ocean image source domain dataset D s With the underwater crack target domain dataset D t ,The ocean image source domain dataset contains paired degraded and clear ocean images, and the underwater crack target domain dataset contains real underwater crack images. The source domain and target domain meet the cross-domain data difference;

[0056] S2. Conditional Diffusion Model Training: Train a conditional diffusion model based on the Transformer architecture. This model is trained on an ocean image source domain dataset to learn the implicit mapping relationship between underwater optical degradation and clear images. The conditional diffusion model based on the Transformer architecture uses a non-uniform skip sampling strategy to accelerate the inference process.

[0057] S3. Cross-domain transfer enhancement: The trained conditional diffusion model based on the Transformer architecture is applied to the target domain through a transfer learning framework. Underwater fracture image enhancement is achieved through cross-domain feature transfer. During the back-diffusion process, the model uses the original fracture image as the conditional input, gradually removes noise and restores image details, achieving cross-domain underwater fracture image enhancement and obtaining enhanced underwater fracture images.

[0058] S4, turbidity adaptive frequency domain decoupling enhancement: turbidity adaptive frequency domain feature decoupling of the underwater crack image is performed, and the frequency domain signal in the underwater crack image enhanced in step S3 is decomposed and reconstructed to further enhance the crack information in the underwater crack image, realize the dual-stage enhancement of the underwater crack image, and obtain the final enhanced underwater crack image.

[0059] We select a set of low-quality underwater crack images as an example and follow the method flow ( Figure 1 ) for detailed introduction:

[0060] S1, ocean image source domain dataset D s A total of 3,000 paired ocean images were selected from the UIEB and LSUI public datasets, represented as follows:

[0061]

[0062] in 1500 degraded ocean images, is the corresponding 1500 clear images, N is D s The total number of data in;

[0063] Underwater crack target domain dataset D t , which is expressed as follows:

[0064]

[0065] in is a real underwater crack image, M is D t The total number of data in;

[0066] The source domain and the target domain satisfy cross-domain data differences, namely:

[0067]

[0068] in is the ocean image source domain dataset D s The probability distribution of images in ; The target domain dataset D for underwater cracks t The probability distribution of the image in , Represents ocean imagery; Represents an underwater crack image.

[0069] The underwater crack target domain dataset was collected by building a controllable underwater imaging platform (a total of 300 images). Concrete crack specimens (3 groups) made of cast concrete were placed in the underwater imaging platform. The adjustable light source intensity in the platform parameters of the underwater imaging platform was adjusted to 100 lux, 300 lux, 700 lux, and 1000 lux. The shooting distance was set to 0.5m, 1m, 1.5m, and 2m. The water turbidity was set to 10NTU, 50NTU, and 100NTU. The industrial camera was 20 million pixels.

[0070] S2 specifically includes:

[0071] S2.1, such as Figure 2 、 3As shown in the figure, the conditional diffusion model based on the Transformer architecture learns the implicit mapping relationship between underwater optical degradation and clear images, and constructs a forward noise prediction network. , forward noise prediction network The objective function for:

[0072]

[0073] in is the noisy image at time step t, and the total step length is T=1000; is the input image; is Gaussian noise; Enter the condition; It is an expectation operation on the distribution of image data and time steps; is the noise scheduling coefficient, and the cosine scheduling strategy is adopted:

[0074]

[0075] S2.2. The back diffusion process predicts and eliminates the potential noise distribution in the image to restore pure Gaussian noise to the target image. The improved DDIM acceleration algorithm is used, and the non-uniform skip sampling strategy is adopted: the T-step back diffusion process is divided into two intervals: [0, n=300] and [n=300, T=1000]. In the first n=300 steps, non-uniform sampling with a step size of d1=3 is used, and in the steps from n=300 to T=1000, non-uniform sampling with a step size of d2=5 is used.

[0076] S3 specifically includes:

[0077] S3.1. Data preprocessing: For the ocean image source domain dataset D s With the underwater crack target domain dataset D t The images in the dataset are uniformly resized to 256×256 and normalized to [-1,1] pixel values. Ocean underwater image enhancement is used as the source task, and underwater crack image enhancement is used as the target task.

[0078] S3.2. Migrate the trained conditional diffusion model to the target domain and use feature alignment loss Fine-tune the model (fine-tuning iterations: 50 epochs, learning rate 1e-5):

[0079]

[0080] in represent the source domain features and target domain features respectively, is the feature mapping function, which is the third layer feature of the conditional diffusion model; To represent the maximum mean difference, a Gaussian kernel is used with a bandwidth parameter of 1.0;

[0081] S3.3, in the reverse diffusion process, the original target domain underwater crack image As the conditional input, the image is denoised and image details are restored step by step (50 steps (using DDIM accelerated sampling)) to generate an enhanced underwater crack image. .

[0082] Furthermore, S4 specifically includes:

[0083] S4.1. Underwater crack image enhanced in step S3 Frequency domain decomposition using turbidity adaptive selection mechanism:

[0084]

[0085] in, Represents wavelet basis function The discrete wavelet transform of is the fast Fourier transform; is the estimated value of current water turbidity (70 NTU), is the turbidity threshold (50 NTU), is the crack image after frequency domain decomposition, which includes the high-frequency component H F With the low frequency component L F ;because , so discrete wavelet transform is selected to process crack images.

[0086] S4.2, the high frequency component H after decomposition F Perform adaptive enhancement:

[0087]

[0088] in, is the turbidity-related gain coefficient, is the turbidity optimization coefficient ( =0.5); is a Gaussian smoothing kernel (kernel size: 3×3, standard deviation 0.8), used to suppress noise amplification;

[0089] S4.3. Reconstruct the image by inverse discrete wavelet transform (IDWT) or inverse fast Fourier transform (FFT) to obtain the final enhanced underwater crack image. 300 sheets.

[0090] FIG4 is a diagram of the underwater crack image enhancement process provided by the present invention.

[0091] In a preferred embodiment, the method of the present invention is applied to a computer system, comprising:

[0092] Data acquisition module:

[0093] (1) Underwater imaging unit: including adjustable light source intensity and industrial camera control module;

[0094] (2) Environmental control unit: turbidity sensor, distance sensor control module;

[0095] (3) Fixing device for concrete crack specimens;

[0096] Image enhancement module:

[0097] (1) Inference acceleration unit: integrated with improved DDIM acceleration algorithm;

[0098] (2) Real-time processing unit: adopting skip sampling strategy (determining the interval of non-uniform sampling);

[0099] System integration module:

[0100] (1) Data flow management: Automating the process from image acquisition to enhanced evaluation;

[0101] (2) Human-computer interaction interface: provides parameter setting and result display functions;

[0102] (3) Logging system: saves detailed parameters and performance data of each enhancement process.

[0103] The present invention also provides an underwater crack image enhancement system based on data migration and frequency domain decoupling, comprising:

[0104] Data acquisition module: Constructing ocean image source domain dataset D s With the underwater crack target domain dataset D t ,The ocean image source domain dataset contains paired degraded and clear ocean images, and the underwater crack target domain dataset contains real underwater crack images. The source domain and target domain meet the cross-domain data difference;

[0105] Model training module: Trains a conditional diffusion model based on the Transformer architecture. This model is trained using ocean image source domain datasets to learn the implicit mapping relationship between underwater optical degradation and clear images.

[0106] Image enhancement module: The trained conditional diffusion model based on the Transformer architecture is applied to the target domain through a transfer learning framework. Underwater crack images are enhanced through cross-domain feature transfer. The frequency domain signals in the enhanced underwater crack images are decomposed and reconstructed to further enhance the crack information in the underwater crack images, resulting in the final enhanced underwater crack images.

[0107] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the program.

[0108] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0109] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions and effects do not exceed the scope of the technical solution of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A method for underwater crack image enhancement based on data migration and frequency domain decoupling, characterized in that: include: S1. Multi-source dataset construction: Constructing ocean image source domain dataset D s With the underwater crack target domain dataset D t ,The ocean image source domain dataset contains paired degraded and clear ocean images, and the underwater crack target domain dataset contains real underwater crack images. The source domain and target domain meet the cross-domain data difference; S2. Conditional Diffusion Model Training: Train a conditional diffusion model based on the Transformer architecture. This model is trained on an ocean image source domain dataset to learn the implicit mapping relationship between underwater optical degradation and clear images. The conditional diffusion model based on the Transformer architecture uses a non-uniform skip sampling strategy to accelerate the inference process. S3. Cross-domain transfer enhancement: The trained conditional diffusion model based on the Transformer architecture is applied to the target domain through a transfer learning framework. Underwater crack image enhancement is achieved through cross-domain feature transfer. During the back-diffusion process, the model uses the original crack image as the conditional input, gradually removes noise and restores image details, achieving cross-domain underwater crack image enhancement and obtaining enhanced underwater crack images. This includes: S3.

1. Data preprocessing: For the ocean image source domain dataset D s With the underwater crack target domain dataset D t The images in the dataset are uniformly subjected to resolution adjustment and [-1, 1] pixel value normalization, with ocean underwater image enhancement as the source task and underwater crack image enhancement as the target task. S3.

2. Migrate the trained conditional diffusion model to the target domain and use feature alignment loss Perform model fine-tuning: in represent the source domain features and target domain features respectively, is the feature mapping function; to indicate the maximum mean difference; S3.3, in the reverse diffusion process, the original target domain underwater crack image As the conditional input, it gradually removes noise and restores image details to generate enhanced underwater crack images. ; S4, turbidity adaptive frequency domain decoupling enhancement: performing turbidity adaptive frequency domain feature decoupling of the underwater crack image, decomposing and reconstructing the frequency domain signal in the underwater crack image enhanced in step S3, enhancing the crack information in the underwater crack image, achieving dual-stage enhancement of the underwater crack image, and obtaining the final enhanced underwater crack image; specifically comprising: S4.

1. Underwater crack image enhanced in step S3 Frequency domain decomposition using turbidity adaptive selection mechanism: in, Represents wavelet basis function The discrete wavelet transform of is the fast Fourier transform; is the estimated value of current water turbidity, is the turbidity threshold, is the crack image after frequency domain decomposition, which includes the high-frequency component H F With the low frequency component L F ; S4.2, the high frequency component H after decomposition F Perform adaptive enhancement: in, is the turbidity-related gain coefficient, is the turbidity optimization coefficient; is a Gaussian smoothing kernel used to suppress noise amplification; S4.

3. Reconstruct the image by inverse discrete wavelet transform (IDWT) or inverse fast Fourier transform (FFT) to obtain the final enhanced underwater crack image. .

2. The underwater crack image enhancement method based on data migration and frequency domain decoupling according to claim 1 is characterized in that: In S1, the ocean image source domain dataset D s Paired ocean images selected from a public dataset are represented as follows: in For degraded ocean images, is the corresponding clear image, N is D s The total number of data in; Underwater crack target domain dataset D t , which is expressed as follows: in is a real underwater crack image, M is D t The total number of data in; The source domain and the target domain satisfy cross-domain data differences, namely: in is the ocean image source domain dataset D s The probability distribution of images in ; The target domain dataset D for underwater cracks t The probability distribution of images in ; Represents ocean imagery; Represents an underwater crack image.

3. The underwater crack image enhancement method based on data migration and frequency domain decoupling according to claim 1 or 2, characterized in that: In S1, the underwater crack target domain dataset is collected by building a controllable underwater imaging platform, and a concrete crack specimen made of concrete is placed in the underwater imaging platform.

4. The underwater fracture image enhancement method based on data migration and frequency domain decoupling according to claim 1 is characterized in that: S2 specifically includes: S2.

1. Conditional diffusion model based on Transformer architecture learns the implicit mapping relationship between underwater optical degradation and clear images, and builds a forward noise prediction network , forward noise prediction network The objective function for: in is the noisy image at time step t, and the total step length is T; is the input image, is Gaussian noise; Enter the condition; It is an expectation operation on the distribution of image data and time steps; is the noise scheduling coefficient, and the cosine scheduling strategy is adopted: S2.

2. The back diffusion process restores pure Gaussian noise to the target image by predicting and eliminating the potential noise distribution in the image. The improved DDIM acceleration algorithm is adopted, and the non-uniform skip sampling strategy is used: the T-step back diffusion process is divided into two intervals [0,n] and [n,T]. Non-uniform sampling with a step size of d1 is used in the first n steps, and non-uniform sampling with a step size of d2 is used in steps n to T.

5. An underwater crack image enhancement system based on data migration and frequency domain decoupling that implements the underwater crack image enhancement method based on data migration and frequency domain decoupling according to claim 1, characterized in that: include: Data acquisition module: Constructing ocean image source domain dataset D s With the underwater crack target domain dataset D t ,The ocean image source domain dataset contains paired degraded and clear ocean images, and the underwater crack target domain dataset contains real underwater crack images. The source domain and target domain meet the cross-domain data difference; Model training module: Trains a conditional diffusion model based on the Transformer architecture. This model is trained using ocean image source domain datasets to learn the implicit mapping relationship between underwater optical degradation and clear images. Image enhancement module: The trained conditional diffusion model based on the Transformer architecture is applied to the target domain through a transfer learning framework. Underwater crack images are enhanced through cross-domain feature transfer. The frequency domain signals in the enhanced underwater crack images are decomposed and reconstructed to enhance the crack information in the underwater crack images, resulting in the final enhanced underwater crack images.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images