Underwater crack defect intelligent sensing method and system

Abstract

The invention discloses an underwater crack defect intelligent sensing method and system, and belongs to the technical field of underwater intelligent sensing. According to the method, underwater crack features are enhanced by using a diffusion model and a checkerboard template, the feature contrast of an underwater crack data set is improved, instance segmentation, extraction and measurement of the underwater crack features are realized through a U2Net network segmentation model, and an underwater crack defect intelligent detection platform and system are constructed. Different types of underwater dam crack features can be well extracted, the defect degree can be effectively evaluated, and good technical support is provided for underwater dam safety detection and reliability evaluation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent underwater sensing, and particularly relates to an underwater crack defect intelligent sensing method and system, which can be widely applied to the fields of underwater robots, defect detection and underwater environment monitoring. BACKGROUND

[0002] The transformation of marine science and technology into an innovation leading type is a main approach to improving the efficiency and potential of modern marine economy. Underwater infrastructure is the main body of the development of modern marine economy, and specifically relates to water conservancy and hydropower, bridges, ports, wharfs, reservoirs, dams and nuclear power plants. Underwater infrastructure safety maintenance is the key to ensuring its economic soil efficiency. Compared with land infrastructure, underwater infrastructure is affected by high water pressure, high temperature gradient and water flow impact, and defects such as cracks and wear are easily generated on the surface of the structure, which will cause damage to the underwater infrastructure and affect its strength and life if not repaired in time.

[0003] In particular, for the aging of underwater structures, crack defect detection needs to be performed on the surface of underwater structures, and the operation demand of underwater robots is increasing. Obtaining clear and reliable underwater images has become a core problem of related technologies. However, the light in the underwater environment is complex and severely scattered, resulting in a general problem of blur and noise in the collected images. Such blurred images not only affect the visual perception of related unmanned systems, but also reduce the subsequent processing accuracy, making it difficult to achieve accurate underwater crack detection and instance segmentation. Although traditional image denoising methods represented by mean filtering and Gaussian filtering can remove noise in the image to some extent, they also weaken the detailed features in the image, resulting in inaccurate crack detection.

[0004] With the development of artificial intelligence technologies represented by deep learning, it has played an important role in underwater target detection and identification. In the prior art, deep learning technologies represented by convolutional neural networks have become an important means of multi-source remote sensing water body information and underwater target sensing. However, there are still a series of research blind spots in the intelligent sensing and feature extraction of fine and complex underwater defects, the main reason being that the sample data set for deep learning is not yet complete, the defects are diversified, and the non-regularization.

[0005] Generative deep learning networks can effectively fit the statistical distribution characteristics of source data and generate new samples that approximate the source data, thereby effectively solving the problems of sample missing and deficiency in deep learning, and are an effective method of data enhancement. Among them, the diffusion model (Diffusion Model) performs an excellent image enhancement capability through a bidirectional generation process of noise addition and denoising. However, the existing diffusion model still faces the problem of crack detail loss or blurring in the denoising process of high-noise underwater images. SUMMARY

[0006] To solve the above technical problems, the application provides an underwater crack defect intelligent perception method and system, which realizes the enhancement of underwater crack features by using a diffusion model and a checkerboard template, improves the feature contrast of the underwater crack dataset, realizes instance segmentation, extraction and measurement of underwater crack features by a U2Net network segmentation model, and constructs an underwater crack defect intelligent detection platform and system, thereby providing effective technical support for improving the accurate detection and fault elimination of underwater dam defects.

[0007] To achieve the above object, the technical scheme adopted by the application is as follows:

[0008] An underwater crack defect intelligent perception method, the method comprising:

[0009] Step 1, collecting underwater crack images and performing preprocessing, and constructing a checkerboard template for feature injection;

[0010] Step 2, constructing a diffusion model, and using the checkerboard template to perform feature enhancement on the preprocessed underwater crack images;

[0011] Step 3, using a U2Net segmentation network to perform crack segmentation on the underwater crack images after feature enhancement;

[0012] Step 4, performing pixel-level calculation on the output features after crack segmentation to obtain crack geometric dimensions.

[0013] On the other hand, the application provides an underwater crack defect intelligent perception system, comprising:

[0014] A preprocessing module for collecting underwater crack images and performing preprocessing, and constructing a checkerboard template for feature injection;

[0015] An enhancement module for constructing a diffusion model and using the checkerboard template to perform feature enhancement on the preprocessed underwater crack images;

[0016] A segmentation module for using a U2Net segmentation network to perform crack segmentation on the underwater crack images after feature enhancement;

[0017] A calculation module for performing pixel-level calculation on the output features after crack segmentation to obtain crack geometric dimensions.

[0018] In a third aspect, the application provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned underwater crack defect intelligent perception method.

[0019] In a fourth aspect, the present application provides a computer readable storage medium, which stores executable instructions, and the executable instructions are executed by a processor to enable the processor to implement the underwater crack defect intelligent perception method.

[0020] The present application has the following advantages:

[0021] Compared with the traditional underwater crack image detection and segmentation method, the present application realizes the preprocessing of high-contrast training data set through the diffusion model, effectively fuses the checkerboard feature with the target to be detected and segmented, improves the feature contrast of the underwater crack data set, has strong practicability and detection and segmentation precision. In addition, compared with the traditional method, the underwater crack intelligent detection system of the present application does not need to realize the cross-domain conversion of the image, and realizes the multi-modal high-precision underwater defect intelligent perception. Therefore, the method of the present application has wide application prospect, and can be used in the fields of underwater dam defect intelligent detection, underwater building safety maintenance, underwater intelligent robot, etc. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A flow chart of the underwater crack defect intelligent perception method of the present application;

[0023] Fig. 2(a) is an actual underwater crack image;

[0024] Fig. 2(b) is an underwater crack image after preprocessing by the checkerboard model of the present method;

[0025] Figure 3 Fig. 3 is a U2Net network prediction image and defect label image;

[0026] Figure 4 Fig. 4 is a U2Net segmentation network structure diagram;

[0027] Figure 5 Fig. 5 is a residual UNet module network structure diagram;

[0028] Figure 6 Fig. 6 is a cross-channel attention mechanism optimization method structure diagram;

[0029] Figure 7 Fig. 7 is a channel threshold processing structure diagram. DETAILED DESCRIPTION

[0030] The present application will be further described below in combination with the drawings and examples.

[0031] As shown in the drawings, Figure 1 the present application proposes an underwater crack defect intelligent perception method, and the specific process is as follows:

[0032] Step 1, collect underwater crack image and pre-process to obtain underwater crack effective sample image, construct a checkerboard template for feature injection; adopt optical camera to obtain the original collected underwater crack image, convert it into a digital image with 512x512 pixel dimension, pre-process the data edge enhancement, and then screen out the low resolution image to be processed as the underwater crack effective sample image. Including:

[0033] Edge enhancement processing is performed on the original collected underwater crack image by using Laplace operator, and the variance of the edge response value of the original collected underwater crack image is calculated , and the calculation formula is:

[0034] ,

[0035] Among them, is the response value of the original collected underwater crack image at the pixel point after Laplace processing, is the average value of all pixel point response values, and M and N are the width and height of the original collected underwater crack image respectively; when ( is a preset definition threshold), it is determined that the definition of the original collected underwater crack image is insufficient. The noise level of the original collected underwater crack image is estimated by using a method based on frequency domain analysis, and when the noise level is greater than a preset noise threshold , it is determined that the original collected underwater crack image has too much noise. Only when the original collected underwater crack image meets the preset threshold requirements in terms of definition and noise level indicators, it is determined as an underwater crack effective sample image, and enters the subsequent checkerboard template injection step.

[0036] Construct a checkerboard template for feature injection: define a checkerboard image under the coordinates , and its pixel value satisfies the following rules:

[0037]

[0038] Among them, and are two pixel values with obvious contrast, and are the step length of the square. Adjust the checkerboard image to a preset size as a style image required for style injection of the underwater crack effective sample image. Adjust the underwater crack effective sample image and the checkerboard template for feature injection to 512x512 pixels, and input them into the subsequent diffusion model with preset values. In the embodiment, , At this time, the black and white checkerboard image with the highest contrast is generated.

[0039] Step 2, construct a diffusion model, and perform feature enhancement on the effective sample image of the underwater crack by using the checkerboard template;

[0040] The diffusion model comprises a forward encoder and a reverse encoder, the forward encoder is used to extract the latent feature representation of the effective sample image of the underwater crack and the constructed checkerboard template, and the reverse encoder is used to inject the latent feature representation of the checkerboard template into the latent feature representation of the effective sample image of the underwater crack for feature enhancement;

[0041] The diffusion model forward encoder is constructed, and the structure comprises a noise adding module and an image encoding module. The effective sample image of the underwater crack is input into the forward encoder, the underwater defect image is converted into a latent noise representation by gradually adding Gaussian noise, specifically, the noise adding module is responsible for adding noise to the input image at each time step, and gradually converting it into a latent noise representation; the image encoding module encodes the latent noise representation to generate a latent feature representation for use by the reverse decoding module. The specific noise adding process can be represented as:

[0042] ,

[0043] Wherein, represents the image at the moment, ; is a parameter related to the noise intensity, which generally decreases with the increase of the moment ; is a standard normal distribution noise, which represents the noise added at each moment. Finally, the effective sample image of the underwater crack becomes an approximate Gaussian noise at the moment.

[0044] The loss function of the diffusion model is defined based on the feature consistency of the optical acquisition data and the definition of the underwater image clarity, so as to ensure that the model accurately extracts the features of the underwater crack under different optical conditions.

[0045] The checkerboard template obtained in step 1 is also input into the forward encoder of the diffusion model to add noise and extract latent features at the same time step , so as to obtain the Key, Value features of the checkerboard image at each moment . During the execution of the diffusion model forward encoder, the image latent feature will be inversely pushed to the Gaussian noise at the T moment, and the process will store the intermediate features at each moment t. The noise finally obtained by inversely diffusing the effective sample image of the underwater crack is , the query feature of the underwater crack effective sample image at each time t in the inversion process is denoted as The content query feature obtained is denoted as ; the inverse diffusion of the checkerboard template image to the noise is , the query feature of the underwater crack effective sample image at each time t in the inversion process is denoted as The key and value obtained are denoted as In the inference process of the diffusion model, the multi-layer structure of the U-Net decoder (each layer contains a Residual Block, a Self-Attention, a Cross-Attention, etc.) needs to be called at each time step. In the self-attention layer, the (K, V) of the underwater crack image itself is replaced by the (K, V) of the checkerboard template image, which is equivalent to letting the underwater crack effective sample image "refer" to the local texture of the checkerboard template in attention to update itself. After obtaining the latent features of the underwater crack effective sample image and the checkerboard template, the replacement can be performed in the subsequent generation process from to to achieve the purpose of style transfer.

[0046] The diffusion model decoder is constructed, the checkerboard template is injected, the crack feature information of the original image is enhanced, and the processed image dataset is generated as the training set of the subsequent segmentation model:

[0047] The underwater crack effective sample image is subjected to reverse diffusion from , and the synthesized latent feature is initialized. At each time t in the reverse diffusion , the query feature of the underwater crack effective sample image is obtained , and the corresponding , of the checkerboard template. In the attention formula , Q remains from the current underwater crack effective sample image, and uses the features of the checkerboard template, so that the attention result output "injects" the checkerboard template features into the underwater crack effective sample image.

[0048] In the self-attention mechanism, if only is replaced and the Q of the underwater crack effective sample image is completely used, there is a risk of excessive style invasion and damage to the structure of the underwater crack effective sample image in the iteration process. Therefore, at each time t, the of the current generated image is linearly interpolated with the of the underwater crack effective sample image, and the interpolated is used for attention calculation. The content query is mixed with the current synthesized query , and the specific synthesis method is as follows:

[0049] ,

[0050] wherein , for controlling the preservation of underwater crack structure information; if is larger, more original content structure is preserved, and the influence of the checkerboard template style will be weakened; if is smaller, more attention is paid to the injection of checkerboard template style information.

[0051] That is, the self-attention output can be replaced next, that is, the self-attention output can be replaced:

[0052] ,

[0053] Through this step, at the same time t, the query Q mixes the underwater crack effective sample image information, and the key and value come from the checkerboard template, so that the update mode of the current generated latent variable simultaneously focuses on the preservation of the underwater defect structure and the migration of the checkerboard template features.

[0054] Since the calculation of self-attention includes calculation term, after directly replacing , Q and K can be more mismatched, resulting in a smaller overall numerical value of the attention distribution and a less obvious difference, weakening the contrast of the generated image and causing the generated result to be blurred. By using the following temperature scaling attention formula, high-contrast textures are injected into the keys and values of the checkerboard template:

[0055]

[0056] wherein , for amplifying the peak value of the distribution of attention weights, is the channel dimension. This step makes the injection of local textures clearer, and also avoids the occurrence of large-area smoothing and blurring in the generated image in practice.

[0057] After all the reverse steps are set, the is obtained, and then the decoder is decoded to output the underwater crack image enhanced by the checkerboard template (as shown in FIG. 2(b)), which is used as the training set of the subsequent U2Net segmentation network.

[0058] Step 3, using the U2Net segmentation network to perform crack segmentation on the underwater crack effective sample image whose features are enhanced;

[0059] Constructing a pre-training data set based on the U2Net segmentation network, wherein the U2Net segmentation network structure is as follows: Figure 4As shown, the effective sample image of the underwater crack and the underwater crack image enhanced by the checkerboard template are fused along the channel dimension, and after random cropping and random rotation, they are input into the U2Net segmentation network. The encoder of the U2Net segmentation network consists of four cascaded improved residual UNet modules. The improved residual UNet modules are optimized by a cross-channel attention mechanism. The network structure of the residual UNet modules is as follows. Figure 5 As shown, the structure of the cross-channel attention mechanism optimization method is as follows: Figure 6 As shown; after each improved residual UNet module, a 2×2 max-pooling downsampling is performed, followed by the introduction of... Figure 7 The channel threshold processing module shown performs local soft threshold processing to achieve local soft threshold noise reduction.

[0060] The decoder of the U2Net segmentation network consists of three cascaded improved residual UNet modules. Starting from the lowest-level features, linear interpolation is used for upsampling, followed by skip-connection fusion with the corresponding layer encoder features before inputting into the decoder. The outputs of each decoder are processed using convolutional fusion channels, and the final output is obtained after thresholding. The U2Net segmentation network is trained with the following parameters: the number of iterations (epochs) is set to 200, the batch size is set to 1, and the initial learning rate (lr) is set to 0.01.

[0061] The effective sample images of underwater cracks and their corresponding checkerboard-pattern-enhanced images were divided proportionally, with 70% randomly selected as the training set, 20% as the test set, and 10% as the validation set. The trained U2Net segmentation network was then tested, using the cross-entropy loss function, calculated as follows:

[0062] ,

[0063] Where N is the total number of pixels. This represents the true label of the b-th pixel (whether it belongs to the crack region). This represents the probability that the pixel is a crack as predicted by the network. By minimizing this cross-entropy loss, the deviation between the predicted probability distribution and the actual segmentation label can be effectively measured and penalized, making the network more accurate in identifying crack regions in underwater images, thereby improving the accuracy and robustness of underwater defect detection.

[0064] The main performance indicators for testing underwater crack detection and segmentation networks are: Intersection over Union (IoU), Mean Absolute Error (MAE), and Average Precision (mAP).

[0065] G represents the actual crack region, P represents the crack region predicted by the network, and IoU is the ratio of the number of overlapping pixels between the predicted region and the actual labeled region to the number of pixels in their union. The calculation method is as follows:

[0066] ,

[0067] If there is a total One crack sample, For the true value of the f-th sample, If the model predicts the value, then the mean absolute error (MAE) is calculated as follows:

[0068] ,

[0069] The average accuracy of each crack category was calculated based on the prediction-regression curve. Then, the average value of different thresholds is obtained. :

[0070] ,

[0071] Where C represents the total number of crack categories.

[0072] By employing the aforementioned multiple evaluation methods, the accuracy of underwater crack location and the stability of overall detection can be quantitatively measured, thereby ensuring more robust defect identification and safety assessment in complex underwater environments.

[0073] Step 4: Perform pixel-level calculations on the output features of the image after crack segmentation to obtain the crack geometry; the calculation method is as follows:

[0074] Crack skeleton extraction: Based on binarized image of segmentation output Extract the skeleton point set representing the connecting line at the center of the crack. , representing pixels If it belongs to the crack region, it is 0 otherwise; thus, the path composed of skeleton pixels is obtained.

[0075] Crack length calculation: Let the number of skeleton pixels be P, and let... Let the distance between the p-th skeleton pixel and its adjacent skeleton pixels be the crack length. The calculation method is as follows:

[0076] ,

[0077] Crack area calculation: For all pixels in the segmented region satisfy Total area of ​​the crack region The calculation method is as follows:

[0078] ,

[0079] in This represents the pixel value of the crack. This indicates the physical area size of the corresponding pixel; if each pixel corresponds to a physical area ,but .

[0080] In this embodiment, the underwater crack shown in Figure 2(a) is processed using the method described above, and the prediction result is as follows: Figure 3 As shown. The calculated crack length is 414 pixels, and the total area of ​​the crack region is 36247 pixels. 2 .

[0081] On the other hand, the present invention provides an intelligent sensing system for underwater crack defects, which includes modules capable of implementing the steps of the aforementioned method, specifically including:

[0082] The preprocessing module is used to acquire underwater crack images and perform preprocessing to construct a checkerboard template for feature injection.

[0083] The enhancement module is used to construct a diffusion model and perform feature enhancement on the preprocessed underwater crack image using the checkerboard template.

[0084] The segmentation module is used to segment underwater crack images after feature enhancement using the U2Net segmentation network.

[0085] The calculation module is used to perform pixel-level calculations on the output features after crack segmentation to obtain the crack geometry.

[0086] Thirdly, the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the aforementioned intelligent sensing method for underwater crack defects.

[0087] Fourthly, the present invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enable the processor to implement the aforementioned intelligent sensing method for underwater crack defects.

[0088] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent sensing of underwater crack defects, characterized in that, The method includes: Step 1: Acquire underwater crack images and preprocess them to obtain effective sample images of underwater cracks, and construct a checkerboard template for feature injection; Step 2: Construct a diffusion model and use the checkerboard template to enhance the features of the effective sample images of the underwater cracks; Step 3: Use the U2Net segmentation network to segment the underwater cracks in the feature-enhanced effective sample images; Step 4: Perform pixel-level calculations on the output features of the image after crack segmentation to obtain the crack geometry.

2. The intelligent sensing method for underwater crack defects according to claim 1, characterized in that, Step 1 includes: An optical camera is used to acquire raw underwater crack images. The Laplacian operator is used to perform edge enhancement processing on the raw underwater crack images. The variance of the edge response values ​​of the raw underwater crack images is calculated. When the variance is greater than or equal to a preset sharpness threshold, the raw underwater crack images are determined to meet the sharpness threshold requirements. The noise level of the raw underwater crack images is estimated based on a frequency domain analysis method. When the noise level is less than or equal to a preset noise threshold, the raw underwater crack images are determined to meet the threshold requirements. When the original underwater crack image simultaneously meets the requirements for clarity and noise preset threshold, it is determined to be a valid underwater crack sample image.

3. The intelligent sensing method for underwater crack defects according to claim 2, characterized in that: Constructing a checkerboard template for feature injection includes: In coordinates Define a chessboard image The pixel values ​​follow the following rules: ， in, and These are two pixel values ​​with significant contrast. and The step size is the square of the chessboard.

4. The intelligent sensing method for underwater crack defects according to claim 1, characterized in that, In step 2, the diffusion model includes a forward encoder and a reverse encoder. The forward encoder is used to extract the latent feature representation of the effective sample image of the underwater crack and the constructed checkerboard template. The reverse encoder is used to inject the latent feature representation of the checkerboard template into the latent feature representation of the effective sample image of the underwater crack for feature enhancement.

5. The intelligent sensing method for underwater crack defects according to claim 4, characterized in that, The forward encoder includes a noise addition module and an image encoding module. The noise addition module gradually transforms the input image into a latent noise representation by adding noise to the input image at each time step. The image encoding module encodes the latent noise representation to generate a latent feature representation. At each time step of the back diffusion, the reverse decoder acquires the query features of the effective sample image of the underwater crack, as well as the key and value corresponding to the checkerboard template. Q retains the features from the current effective sample image of the underwater crack and uses an attention mechanism to inject the features of the checkerboard template into the effective sample image of the underwater crack.

6. The intelligent sensing method for underwater crack defects according to claim 1, characterized in that, Step 3 includes fusing the effective sample image of the underwater crack with the underwater crack image enhanced by the checkerboard template along the channel dimension, applying random cropping and random rotation operations, and then inputting it into the U2Net segmentation network. The encoder of the U2Net segmentation network consists of four cascaded improved residual UNet modules, which are optimized by a cross-channel attention mechanism. The decoder of the U2Net segmentation network consists of three cascaded improved residual UNet modules. Starting from the bottom layer features, linear interpolation is first used for upsampling, and then the features are fused with the corresponding layer encoder features before being input into the decoder.

7. The intelligent sensing method for underwater crack defects according to claim 1, characterized in that, Step 4 includes extracting a set of skeleton points representing the connected lines at the center of the crack from the binarized image output by segmentation, obtaining the path composed of skeleton pixels, and calculating the crack length and crack area based on the skeleton pixels.

8. An intelligent sensing system for underwater crack defects, characterized in that, include: The preprocessing module is used to acquire underwater crack images and perform preprocessing to construct a checkerboard template for feature injection. The enhancement module is used to construct a diffusion model and perform feature enhancement on the preprocessed underwater crack image using the checkerboard template. The segmentation module is used to segment underwater crack images after feature enhancement using the U2Net segmentation network. The calculation module is used to perform pixel-level calculations on the output features after crack segmentation to obtain the crack geometry.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by the one or more processors, the one or more processors implement the intelligent sensing method for underwater crack defects as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed by a processor, enable the processor to implement the intelligent sensing method for underwater crack defects as described in any one of claims 1-7.

Images