An underwater crack defect intelligent sensing method and system

Abstract

The application discloses an underwater crack defect intelligent sensing method and system and belongs to the technical field of underwater intelligent sensing. The method realizes the enhancement of underwater crack features by using a diffusion model and a checkerboard template, improves the feature contrast of an underwater crack data set, realizes instance segmentation, extraction and measurement of underwater crack features by using a U2Net network segmentation model, constructs an underwater crack defect intelligent detection platform and system, can better extract underwater dam crack features of different types and effectively evaluate the defect degree, and provides good technical support for underwater dam safety detection and reliability evaluation.

Description

Technical Field

[0001] This invention belongs to the field of intelligent underwater sensing technology, specifically relating to an intelligent sensing method and system for underwater crack defects, which can be widely applied in the fields of underwater robots, defect detection, and underwater environmental monitoring. Background Technology

[0002] The transformation of marine science and technology towards innovation-driven development is a major pathway to enhancing the efficiency and potential of the modern marine economy. Underwater infrastructure is the mainstay of modern marine economic development, specifically involving water conservancy and hydropower, bridges, ports, wharves, reservoirs, dams, and nuclear power plants. Safe maintenance of underwater infrastructure is crucial to ensuring its economic efficiency. Compared to land-based infrastructure, underwater infrastructure is affected by high water pressure, high temperature gradients, and water flow impacts, making its structural surfaces prone to cracks, wear, and other defects. Failure to repair these defects in a timely manner can damage the underwater infrastructure, affecting its strength and lifespan.

[0003] Specifically addressing the aging of underwater structures necessitates the detection of cracks and defects on their surfaces, increasing the operational demands on underwater robots. Obtaining clear and reliable underwater images has become a core technological challenge. However, the complex and highly scattering lighting conditions in the underwater environment lead to widespread blurring and noise in the acquired images. This blurriness not only affects the visual perception of unmanned systems but also reduces the accuracy of subsequent processing, making accurate underwater crack detection and instance segmentation difficult. While traditional image denoising methods such as mean filtering and Gaussian filtering can remove noise to some extent, they also weaken the detailed features of the image, resulting in inaccurate crack detection.

[0004] With the development of artificial intelligence technologies, represented by deep learning, they have also played an important role in underwater target detection and recognition. Among existing technologies, deep learning technologies, represented by convolutional neural networks, have become an important means of multi-source remote sensing of water body information and underwater target perception. However, there are still a series of research blind spots in the intelligent perception and feature extraction of fine and complex underwater defects. The main reasons are that the sample datasets used for deep learning are incomplete, and the defects are diverse and irregular.

[0005] Generative deep learning networks can effectively fit the statistical distribution characteristics of source data and generate new samples that approximate the source data, thus effectively solving problems such as missing or insufficient samples in deep learning and making them an effective method for data augmentation. Among them, the diffusion model, through a bidirectional generation process of noise addition and denoising, has shown excellent image enhancement capabilities. However, existing diffusion models still face the problem of missing or blurred crack details in the denoising process of high-noise underwater images. Summary of the Invention

[0006] To address the aforementioned technical issues, this invention provides an intelligent perception method and system for underwater crack defects. It utilizes a diffusion model and a checkerboard template to enhance underwater crack features, improving the feature contrast of the underwater crack dataset. A U2Net network segmentation model is used to segment, extract, and measure underwater crack features, thus constructing an intelligent underwater crack defect detection platform and system. This provides effective technical support for improving the accurate detection and troubleshooting of underwater dam defects.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A method for intelligent sensing of underwater crack defects, the method comprising:

[0009] Step 1: Acquire underwater crack images and preprocess them to construct a checkerboard template for feature injection;

[0010] Step 2: Construct a diffusion model and use the checkerboard template to enhance the features of the preprocessed underwater crack image;

[0011] Step 3: Use the U2Net segmentation network to segment the underwater crack image after feature enhancement;

[0012] Step 4: Perform pixel-level calculations on the output features after crack segmentation to obtain the crack geometry.

[0013] On the other hand, the present invention provides an intelligent sensing system for underwater crack defects, comprising:

[0014] The preprocessing module is used to acquire underwater crack images and perform preprocessing to construct a checkerboard template for feature injection.

[0015] The enhancement module is used to construct a diffusion model and perform feature enhancement on the preprocessed underwater crack image using the checkerboard template.

[0016] The segmentation module is used to segment underwater crack images after feature enhancement using the U2Net segmentation network.

[0017] The calculation module is used to perform pixel-level calculations on the output features after crack segmentation to obtain the crack geometry.

[0018] Thirdly, the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned intelligent sensing method for underwater crack defects.

[0019] Fourthly, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enable the processor to implement the aforementioned intelligent sensing method for underwater crack defects.

[0020] The beneficial effects of this invention are as follows:

[0021] Compared to traditional underwater crack image detection and segmentation methods, this invention utilizes a diffusion model to preprocess a high-contrast training dataset, effectively fusing checkerboard features with the targets to be detected and segmented. This enhances the feature contrast of the underwater crack dataset, resulting in strong practicality and improved detection and segmentation accuracy. Furthermore, compared to traditional methods, the intelligent underwater crack detection system described in this invention does not require cross-domain image transformation, achieving multimodal, high-precision intelligent perception of underwater defects. Therefore, the method of this invention has broad application prospects and can be used in fields such as intelligent detection of underwater dam defects, safety maintenance of underwater structures, and underwater intelligent robots. Attached Figure Description

[0022] Figure 1 This is a flowchart of an intelligent sensing method for underwater crack defects according to the present invention;

[0023] Figure 2(a) shows the actual underwater crack image collected;

[0024] Figure 2(b) shows the underwater crack image after preprocessing using the checkerboard model of this method;

[0025] Figure 3 Image prediction and defect labeling for the U2Net network;

[0026] Figure 4 U2Net segmentation network structure diagram;

[0027] Figure 5 The network structure diagram of the residual UNet module;

[0028] Figure 6 Structure diagram of the optimization method for cross-channel attention mechanism;

[0029] Figure 7 This is a diagram of the channel threshold processing structure. Detailed Implementation

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] like Figure 1 As shown, this invention proposes an intelligent sensing method for underwater crack defects, the specific process of which is as follows:

[0032] Step 1: Acquire underwater crack images and preprocess them to obtain valid sample images of underwater cracks, and construct a checkerboard template for feature injection; use an optical camera to acquire the original underwater crack images, convert them into 512×512 pixel digital images, perform data edge enhancement preprocessing, and then select the low-resolution images to be processed as valid sample images of underwater cracks. This includes:

[0033] The Laplacian operator is used to perform edge enhancement processing on the original underwater crack image, and the variance of the edge response values ​​of the original underwater crack image is calculated. The calculation formula is as follows:

[0034] ,

[0035] in, For the original underwater crack image at the pixel level The response value after Laplace processing, Let M and N be the average response values ​​of all pixels, and M and N be the width and height of the original underwater crack image, respectively; when ( If the original underwater crack image is deemed insufficient in clarity (to meet a preset clarity threshold), a frequency domain analysis-based method is used to estimate the noise level of the original underwater crack image. If the noise level exceeds a preset noise threshold... If the original underwater crack image is found to have excessive noise, it will be determined to be a valid underwater crack sample image only if the original underwater crack image meets the preset threshold requirements in both clarity and noise level indicators, and will then proceed to the subsequent checkerboard template injection step.

[0036] Constructing a checkerboard template for feature injection: in coordinates Define a chessboard image The pixel values ​​follow the following rules:

[0037]

[0038] in, and These are two pixel values ​​with significant contrast. and The grid step size is [number]. [The text then describes the chessboard image]. The underwater crack sample image is adjusted to a preset size to serve as the style map required for style injection. Both the underwater crack sample image and the checkerboard template used for feature injection are adjusted to 512×512 pixels and then input together into the subsequent preset value diffusion model. In this embodiment, , This generates a black and white checkerboard image with the highest contrast.

[0039] Step 2: Construct a diffusion model and use the checkerboard template to enhance the features of the effective sample images of the underwater cracks;

[0040] The diffusion model includes a forward encoder and a reverse encoder. The forward encoder is used to extract the latent feature representation of the effective sample image of the underwater crack and the constructed checkerboard template. The reverse encoder is used to inject the latent feature representation of the checkerboard template into the latent feature representation of the effective sample image of the underwater crack for feature enhancement.

[0041] A forward encoder for the diffusion model is constructed, comprising a noise addition module and an image encoding module. Valid sample images of underwater cracks are input into the forward encoder. Gaussian noise is progressively added to transform the underwater defect images into latent noise representations. Specifically, the noise addition module adds noise to the input image at each time step, progressively transforming it into a latent noise representation. The image encoding module encodes the latent noise representation, generating latent feature representations for use by the reverse decoding module. The specific noise addition process can be represented as follows:

[0042] ,

[0043] in, Indicates the first Images of moments ; It is a parameter related to noise intensity, typically changing with time. The increase of decreases; This represents the noise distributed according to a standard normal distribution, indicating the noise added at each time step. The final effective sample image of the underwater crack is at the [missing value]. The time interval becomes approximately Gaussian noise ;

[0044] The loss function of the diffusion model is defined based on the feature consistency of the optically acquired data and the sharpness of the underwater image to ensure that the model accurately extracts the features of underwater cracks under different optical conditions.

[0045] The checkerboard template obtained in step 1 is also input into the forward encoder of the diffusion model at the same time step. Noise addition and latent feature extraction are performed to obtain the key and value features of the checkerboard image at each time step. During the execution of the diffusion model forward encoder, latent features of the image... It will be back-calculated to Gaussian noise at time T. The process stores intermediate features at each time step t. The noise obtained by reverse diffusion of the effective sample images of the underwater crack is... During the inversion process, at each moment The obtained content query features are denoted as The checkerboard template image is de-diffused to noise. During the inversion process, at each moment The obtained key and value are denoted as In the inference process of the diffusion model, each time step calls the multi-layer structure of the U-Net decoder (each layer contains Residual Block, Self-Attention, Cross-Attention, etc.). In the self-attention layer, the (K, V) of the underwater defect image itself is replaced with the (K, V) of the checkerboard template image. This is equivalent to having the effective sample image of the underwater crack "refer to" the local texture of the checkerboard template to update itself during attention. After obtaining the latent features of the effective sample image of the underwater crack and the checkerboard template, subsequent inferences can be performed... Reverse decoding to Replacement is performed during the generation process to achieve style transfer.

[0046] A diffusion model inverse decoder is constructed, and a checkerboard template is injected to enhance the crack feature information of the original image, generating a processed image dataset as the training set for subsequent segmentation models.

[0047] from Start by back-diffusion of effective sample images of underwater cracks to initialize synthetic latent features. At every moment of back diffusion Query features for obtaining valid sample images of underwater cracks And the corresponding chessboard template , In the attention formula In the middle, Q retains images from the current valid underwater fracture samples, while By using the features of the checkerboard template, the output attention result "injects" the checkerboard template features into the valid sample image of the underwater crack.

[0048] In the self-attention mechanism, if only replacement However, Q, which relies entirely on valid underwater crack sample images, may suffer from excessive style intrusion during iteration, risking damage to the structure of the valid underwater crack sample images. Therefore, at each time t, for the currently generated image... With the effective sample image of underwater cracks Perform linear interpolation, and use the interpolated result. Perform attention calculation. Query content. With the current synthetic query The mixture is then prepared as follows:

[0049] ,

[0050] in This is used to control the preservation of underwater crack structure information; if If the content is larger, the original content structure will be retained more, and the influence of the checkerboard template style will be reduced; if... Smaller sizes tend to focus more on injecting checkerboard template style information.

[0051] The self-attention output can then be replaced.

[0052] ,

[0053] Through this step, at the same time t, query Q combines effective sample image information of underwater cracks, while the key and value... The checkerboard template enables the current generation of latent variables to be updated in a way that simultaneously considers underwater defect structure preservation and checkerboard template feature transfer.

[0054] Because the calculation of self-attention includes Calculated items, directly replace Subsequently, Q and K may become even less mismatched, resulting in a smaller overall value and less distinct differences in the attention distribution, which weakens the contrast of the generated image and leads to a blurry result. The following temperature-scaling attention formula is used to inject high-contrast texture into the keys and values ​​of the checkerboard template:

[0055]

[0056] in, This is used to amplify the peak value of the attention weight distribution. This step, using the channel dimension, makes the injection of local textures clearer and, in practice, avoids large areas of smoothing and blurring in the generated image.

[0057] After setting all the reverse steps, we get... The underwater crack image is then decoded by the decoder and enhanced with a checkerboard template (as shown in Figure 2(b)), which serves as the training set for the subsequent U2Net segmentation network.

[0058] Step 3: Use the U2Net segmentation network to segment the underwater cracks in the feature-enhanced effective sample images;

[0059] Construct a pre-trained dataset based on the U2Net segmentation network, where the U2Net segmentation network structure is as follows: Figure 4As shown, the effective sample image of the underwater crack and the underwater crack image enhanced by the checkerboard template are fused along the channel dimension, and after random cropping and random rotation, they are input into the U2Net segmentation network. The encoder of the U2Net segmentation network consists of four cascaded improved residual UNet modules. The improved residual UNet modules are optimized by a cross-channel attention mechanism. The network structure of the residual UNet modules is as follows. Figure 5 As shown, the structure of the cross-channel attention mechanism optimization method is as follows: Figure 6 As shown; after each improved residual UNet module, a 2×2 max-pooling downsampling is performed, followed by the introduction of... Figure 7 The channel threshold processing module shown performs local soft threshold processing to achieve local soft threshold noise reduction.

[0060] The decoder of the U2Net segmentation network consists of three cascaded improved residual UNet modules. Starting from the lowest-level features, linear interpolation is used for upsampling, followed by skip-connection fusion with the corresponding layer encoder features before inputting into the decoder. The outputs of each decoder are processed using convolutional fusion channels, and the final output is obtained after thresholding. The U2Net segmentation network is trained with the following parameters: the number of iterations (epochs) is set to 200, the batch size is set to 1, and the initial learning rate (lr) is set to 0.01.

[0061] The effective sample images of underwater cracks and their corresponding checkerboard-pattern-enhanced images were divided proportionally, with 70% randomly selected as the training set, 20% as the test set, and 10% as the validation set. The trained U2Net segmentation network was then tested, using the cross-entropy loss function, calculated as follows:

[0062] ,

[0063] Where N is the total number of pixels. This represents the true label of the b-th pixel (whether it belongs to the crack region). This represents the probability that the pixel is a crack as predicted by the network. By minimizing this cross-entropy loss, the deviation between the predicted probability distribution and the actual segmentation label can be effectively measured and penalized, making the network more accurate in identifying crack regions in underwater images, thereby improving the accuracy and robustness of underwater defect detection.

[0064] The main performance indicators for testing underwater crack detection and segmentation networks are: Intersection over Union (IoU), Mean Absolute Error (MAE), and Average Precision (mAP).

[0065] G represents the actual crack region, P represents the crack region predicted by the network, and IoU is the ratio of the number of overlapping pixels between the predicted region and the actual labeled region to the number of pixels in their union. The calculation method is as follows:

[0066] ,

[0067] If there is a total One crack sample, For the true value of the f-th sample, If the model predicts the value, then the mean absolute error (MAE) is calculated as follows:

[0068] ,

[0069] The average accuracy of each crack category was calculated based on the prediction-regression curve. Then, the average value of different thresholds is obtained. :

[0070] ,

[0071] Where C represents the total number of crack categories.

[0072] By employing the aforementioned multiple evaluation methods, the accuracy of underwater crack location and the stability of overall detection can be quantitatively measured, thereby ensuring more robust defect identification and safety assessment in complex underwater environments.

[0073] Step 4: Perform pixel-level calculations on the output features of the image after crack segmentation to obtain the crack geometry; the calculation method is as follows:

[0074] Crack skeleton extraction: Binarized image based on segmentation output Extract the skeleton point set representing the connecting line at the center of the crack. , representing pixels If it belongs to the crack region, it is 0 otherwise; thus, the path composed of skeleton pixels is obtained.

[0075] Crack length calculation: Let the number of skeleton pixels be P, and let... Let p-th skeleton pixel represent the distance between it and its adjacent skeleton pixels, then the crack length is... The calculation method is as follows:

[0076] ,

[0077] Crack area calculation: For all pixels in the segmented region satisfy Total area of ​​the crack region The calculation method is as follows:

[0078] ,

[0079] in This represents the pixel value of the crack. This indicates the physical area size of the corresponding pixel; if each pixel corresponds to a physical area ,but .

[0080] In this embodiment, the underwater crack shown in Figure 2(a) is processed using the method described above, and the prediction result is as follows: Figure 3 As shown. The calculated crack length is 414 pixels, and the total area of ​​the crack region is 36247 pixels. 2 .

[0081] On the other hand, the present invention provides an intelligent sensing system for underwater crack defects, which includes modules capable of implementing the steps of the aforementioned method, specifically including:

[0082] The preprocessing module is used to acquire underwater crack images and perform preprocessing to construct a checkerboard template for feature injection.

[0083] The enhancement module is used to construct a diffusion model and perform feature enhancement on the preprocessed underwater crack image using the checkerboard template.

[0084] The segmentation module is used to segment underwater crack images after feature enhancement using the U2Net segmentation network.

[0085] The calculation module is used to perform pixel-level calculations on the output features after crack segmentation to obtain the crack geometry.

[0086] Thirdly, the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned intelligent sensing method for underwater crack defects.

[0087] Fourthly, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enable the processor to implement the aforementioned intelligent sensing method for underwater crack defects.

[0088] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An underwater crack defect intelligent sensing method, characterized in that, The method comprises: Step 1, collecting an underwater crack image and preprocessing to obtain an underwater crack effective sample image, and constructing a checkerboard template for feature injection; Step 2, constructing a diffusion model, and using the checkerboard template to perform feature enhancement on the underwater crack effective sample image; the diffusion model comprises a forward encoder and a reverse decoder, the forward encoder is used to extract the latent feature representation of the underwater crack effective sample image and the constructed checkerboard template, and the reverse decoder is used to inject the latent feature representation of the checkerboard template into the latent feature representation of the underwater crack effective sample image for feature enhancement; The forward encoder comprises a noise adding module and an image encoding module, the noise adding module gradually converts the input image into a latent noise representation by adding noise to the input image at each time step; the image encoding module encodes the latent noise representation to generate a latent feature representation; At each time of reverse diffusion, the reverse decoder obtains the query feature of the underwater crack effective sample image, and the key and value corresponding to the checkerboard template, maintains the current underwater crack effective sample image, and uses an attention mechanism to inject the feature of the checkerboard template into the underwater crack effective sample image; Step 3, using a U2Net segmentation network to perform crack segmentation on the underwater crack effective sample image after feature enhancement; Step 4, performing pixel-level calculation on the image output feature after crack segmentation to obtain crack geometric dimensions.

2. The intelligent underwater crack defect sensing method according to claim 1, characterized in that, The step 1 comprises: An optical camera is used to collect an original underwater crack image, an edge enhancement process is performed on the collected original underwater crack image by using a Laplacian operator; the variance of the edge response value of the original underwater crack image is calculated, and when the variance is greater than or equal to a preset definition threshold, it is determined that the original underwater crack image meets the definition threshold requirement; a noise level of the original underwater crack image is estimated based on a frequency domain analysis method, and when the noise level is less than or equal to a preset noise threshold, it is determined that the original underwater crack image meets the threshold requirement; When the original underwater crack image meets the definition threshold and noise preset threshold requirements at the same time, it is determined as an underwater crack effective sample image.

3. The underwater crack defect intelligent perception method according to claim 2, wherein: Constructing a checkerboard template for feature injection comprises: In the coordinate A checkerboard image is defined below whose pixel values satisfy the following rule: ， wherein, and are two pixel values of contrasting contrast, and is a chessboard grid step.

4. The intelligent underwater crack defect sensing method according to claim 1, characterized in that, The step 3 comprises fusing the underwater crack effective sample image and the underwater crack image enhanced by the checkerboard template in the channel dimension, inputting to the U2Net segmentation network after random cropping and random rotation operation, the encoder of the U2Net segmentation network is four improved residual UNet modules connected in cascade, and the improved residual UNet module is optimized by cross-channel attention mechanism; the decoder of the U2Net segmentation network is three improved residual UNet modules connected in cascade, starting from the lowest layer feature, first using linear interpolation method for up sampling, and then inputting to the decoder after jump fusion with the corresponding layer encoder feature.

5. The method of claim 1, wherein, The step 4 comprises extracting a skeleton point set representing a crack center connected line based on a binary image of the segmentation output, obtaining a path composed of skeleton pixels, and calculating a crack length and a crack area based on the skeleton pixels.

6. An underwater crack defect intelligent sensing system applied to the method of any one of claims 1-5, characterized in that, The method comprises the following steps: a preprocessing module configured to collect an underwater crack image and perform preprocessing, and construct a checkerboard template for feature injection; an enhancement module configured to construct a diffusion model and perform feature enhancement on the preprocessed underwater crack image by using the checkerboard template; a segmentation module configured to perform crack segmentation on the feature-enhanced underwater crack image by using a U2Net segmentation network; a calculation module configured to perform pixel-level calculation on an output feature after the crack segmentation, and obtain crack geometric dimensions.

7. An electronic device, comprising: The method comprises the following steps: one or more processors; a memory configured to store one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, A computer readable storage medium having stored thereon executable instructions that, when executed by a processor, enable the processor to implement the method according to any one of claims 1-5.

Images