Intelligent identification method and system for concrete surface cracks based on deep learning

Abstract

The application provides a concrete surface crack intelligent identification method and system based on deep learning, and belongs to the technical field of concrete detection. The method comprises the following steps: acquiring image data and ultrasonic data; pre-processing the image data and the ultrasonic data to obtain pre-processed image data and pre-processed ultrasonic data; inputting the pre-processed image data into a pre-constructed crack feature enhancement model to obtain high-purity crack features; inputting the high-purity crack features into a pre-constructed weak light multi-feature fusion model to obtain fusion features; performing crack region segmentation based on the fusion features and the pre-processed ultrasonic data to obtain a crack region identification result; fine crack features are extracted through crack feature enhancement and weak light multi-feature fusion, and then the crack segmentation is completed in combination with the ultrasonic data, and coupled features of mud and cracks are stripped according to a hybrid feature separation sub-network.

Description

Technical Field

[0001] This invention belongs to the field of concrete detection technology, and specifically relates to a method and system for intelligent identification of concrete surface cracks based on deep learning. Background Technology

[0002] Concrete is the core load-bearing material for infrastructure such as buildings, bridges, tunnels, ports, and water conservancy projects. Under the influence of long-term loads, environmental erosion, temperature changes, and uneven foundation settlement, surface cracks are prone to form. Cracks are a typical sign of structural deterioration, not only reducing the structural load-bearing capacity and durability but also becoming channels for the penetration of corrosive media such as moisture and chloride ions, accelerating steel corrosion and concrete spalling, directly threatening the safety and service life of the project. Therefore, accurate identification, quantitative measurement, and dynamic monitoring of surface cracks in concrete are crucial aspects of structural health monitoring and safe operation and maintenance.

[0003] However, existing intelligent identification and measurement systems often require a lot of manual intervention in the detection process, have a low degree of automation, and operate in complex underwater environments, resulting in high operational risks. Furthermore, they are difficult to accurately measure crack defects, and the methods for collecting multi-dimensional information about underwater concrete columns are outdated, with low collection efficiency and difficulty in ensuring data accuracy. Summary of the Invention

[0004] To address the problems in the background art, this invention proposes a method and system for intelligent identification of concrete surface cracks based on deep learning.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, this invention proposes a deep learning-based intelligent identification method for concrete surface cracks, comprising: Acquire image and ultrasound data; The image data and ultrasound data are preprocessed to obtain preprocessed image data and preprocessed ultrasound data; The preprocessed image data is input into a pre-constructed crack feature enhancement model to obtain high-purity crack features; The high-purity crack features are input into a pre-constructed weak-light multi-feature fusion model to obtain fused features; Based on the fused features and the preprocessed ultrasonic data, the crack region is segmented to obtain the crack region identification result.

[0006] Preferably, the image data is preprocessed to obtain preprocessed image data, specifically including: Underwater light refraction distortion correction is performed on the image data to obtain the refraction-corrected image data; Water ripple artifact removal is performed on the refraction-corrected image data to obtain ripple artifact-free image data; The image data without ripple artifacts is denoised to obtain denoised image data. The size of the denoised image data is normalized to obtain the preprocessed image data.

[0007] Preferably, the underwater light refraction distortion correction of the image data to obtain the refraction-corrected image data specifically includes: The angle of light refraction is calculated based on the refractive index of the water, the refractive index of the lens medium, and the angle of light incidence. Based on the light refraction angle, the radius of curvature of the curved lens, and the light incident angle, the pixel refraction offset in the x and y directions is calculated respectively. The image data is corrected by adjusting the pixel position based on the refraction offset in the x and y directions to obtain the refraction-corrected image data.

[0008] Preferably, the ultrasound data is preprocessed, specifically including: The sliding window 3σ criterion was used to automatically identify and remove outliers from the ultrasound data, and the outlier sampling points were replaced by neighborhood linear interpolation to obtain the ultrasound data after outlier removal. Gaussian weighted sliding smoothing filter is used to suppress noise in the ultrasound data after outlier removal, resulting in preprocessed ultrasound data.

[0009] Preferably, the crack feature enhancement model includes a serially connected algorithm backbone network and an algorithm neck network; The algorithm's backbone network comprises a serially connected input layer, a low-level convolutional layer, a parallel feature extraction layer, and an output layer. The parallel feature extraction layer includes a parallelly connected edge feature branch layer and a texture feature branch layer. The low-level convolutional layer is used to extract low-level convolutional features of the image. The edge feature branch layer uses the Sobel gradient operator to calculate gradient components and fuse them to obtain basic crack edge features. The texture feature branch layer uses an optimized Gabor filter operator to extract basic crack texture features. The output layer concatenates the basic crack edge features and the basic crack texture features along the channel dimension to obtain a basic feature map.

[0010] Preferably, the algorithm neck network includes a serially connected basic feature input layer, a multi-scale sampling layer, a feature purification and redundancy removal layer, an optimal parameter learning layer, and a feature enhancement output layer; The multi-scale sampling layer uses three sets of convolutional kernels of different scales to process the basic feature map, and obtains multi-scale fused features through adaptive weighted fusion. The feature purification and redundancy removal layer uses a channel attention mechanism to weight the multi-scale fused features in order to remove redundant interference information and obtain pure features. The optimal parameter learning layer aims to maximize the feature contrast between the crack region and the background region, constructs a contrast loss function, and autonomously learns the optimal gamma correction coefficient and the optimal multi-scale weight. The feature enhancement output layer uses the optimal gamma correction coefficient and optimal multi-scale weights obtained during training to perform multi-scale histogram equalization enhancement on the pure features, and outputs high-purity crack features.

[0011] Preferably, the step of inputting the high-purity crack features into a pre-constructed weak-light multi-feature fusion model to obtain fused features specifically includes: After inputting high-purity crack features into the low-light multi-feature fusion model, pixel-level low-light enhanced images are first obtained through feature mapping reconstruction. Then, features of corresponding dimensions are extracted through parallel texture feature branches, edge feature branches, and detail semantic feature branches. Finally, fused features are obtained through adaptive weighted fusion. Among them, the texture feature branch uses an optimized Gabor filter operator to extract crack texture features in low light environment, the edge feature branch uses the Sobel gradient operator combined with an adaptive threshold to extract crack edge features in low light environment, and the detail semantic feature branch extracts crack detail semantic features through convolution operation and activation function.

[0012] Preferably, the step of segmenting the crack region based on the fused features and the preprocessed ultrasonic data to obtain the crack region identification result specifically includes: Establish the basic correlation between fusion features and physical ultrasound data, and calculate the basic correlation coefficient between visual features and physical ultrasound data; Based on the correlation between the components of each scale of the fusion feature and the ultrasonic medium type data, the fusion weight of the multi-scale feature is adaptively corrected, and the original multi-scale fusion components are weighted twice to obtain the fusion feature after medium adaptation. Based on the characteristics of ultrasonic penetration energy, the fusion features after medium adaptation are enhanced in a directional manner in the shielded area to obtain the crack features after the shielded area is strengthened. The correlation coefficient is used to calibrate the correlation determination threshold. The ultrasonic feature template of the crack area is combined to determine the pixel-by-pixel attribute of the enhanced crack feature in the occluded area, distinguish crack pixels from interference pixels, and output the crack area identification result.

[0013] Preferably, the correlation coefficient is used to calibrate the correlation determination threshold, and the ultrasonic feature template of the crack area is combined to perform pixel-by-pixel attribute determination on the enhanced crack features in the occluded area, distinguishing crack pixels from interference pixels. The expression for the crack region identification result is as follows:

[0014] In the formula, Pixels identified as cracks Pixels identified as attachments or interference items As a template for ultrasonic features in the crack zone, This is the actual judgment threshold after calibration with the global basic correlation coefficient.

[0015] Secondly, this invention proposes a deep learning-based intelligent identification system for concrete surface cracks, comprising: The data acquisition module is used to acquire image data and ultrasound data; The data preprocessing module is used to preprocess the image data and ultrasound data to obtain preprocessed image data and preprocessed ultrasound data. The crack feature enhancement module is used to input the preprocessed image data into a pre-constructed crack feature enhancement model to obtain high-purity crack features; The low-light multi-feature fusion module is used to input the high-purity crack features into the pre-constructed low-light multi-feature fusion model to obtain fused features; The crack region segmentation module is used to segment the crack region based on the fused features and the preprocessed ultrasonic data to obtain the crack region identification result.

[0016] The beneficial effects of this invention are: The method of this invention extracts refined crack features by enhancing crack features and fusing multiple features in low light, and then combines ultrasonic data to complete crack segmentation. Based on the hybrid feature separation sub-network, it removes the coupling features between mud and cracks, adapts to underwater low light and attached object occlusion scenarios, accurately distinguishes cracks from underwater interference, and improves crack recognition accuracy.

[0017] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description and the drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart of the intelligent identification method for concrete surface cracks based on deep learning according to the present invention is shown; Figure 2 A schematic diagram of the underwater robot structure of the present invention is shown; Figure 3A cross-sectional view of the underwater camera of the present invention is shown.

[0020] In the image: 1. Underwater robot; 2. Elastic suction cup component; 3. Underwater camera; 4. Self-cleaning component; 5. Curved lens compartment; 6. Ultrasonic probe. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 1 As shown, a method for intelligent identification of concrete surface cracks based on deep learning specifically includes the following steps: S10. Acquire image data and ultrasound data; Reference Figure 2 and Figure 3 As shown, an underwater robot 1 is used to acquire image and ultrasonic data. The underwater robot 1 is equipped with a retractable elastic suction cup component 2, which is used to adhere to and fix the robot on the surface of the underwater concrete column to be inspected, stabilizing the robot's acquisition posture. The underwater robot 1 also carries an underwater camera 3, with a curved lens housing 5 at its front end. A self-cleaning component 4 is installed on the outside of the curved lens housing 5 to protect the underwater camera 3, reduce water interference, and ensure the imaging effect of the underwater camera 3. The underwater robot 1 is also equipped with a retractable ultrasonic probe 6. The underwater robot 1 acquires image data of the underwater concrete surface through the underwater camera 3 and acquires ultrasonic detection data of the concrete through the retractable ultrasonic probe 6.

[0023] S20. Preprocess the image data and ultrasound data to obtain preprocessed image data and preprocessed ultrasound data. Specifically, the acquired image data consists of underwater images of concrete surfaces taken in real-world conditions. During underwater imaging, image distortion occurs due to refraction of light as it passes through the water and the curved lens housing; simultaneously, water current disturbances combined with lens optical characteristics create ripple artifacts; furthermore, the complex underwater environment introduces various types of image noise. Moreover, image sizes vary depending on the shooting scenario. These issues prevent the raw image data from being directly used for subsequent crack identification and measurement. Therefore, preprocessing of the image data is necessary.

[0024] Image data is preprocessed to obtain preprocessed image data, specifically including the following steps: S201. Perform underwater light refraction distortion correction on the image data to obtain refraction-corrected image data. This includes the following steps: S2011. Based on the predetermined refractive index of the water body, the refractive index of the lens medium, and the incident angle of the light, calculate the angle of light refraction. The expression for this angle is:

[0025] In the formula, The refractive index of the water body The refractive index of the lens medium, The angle of incidence of light. The angle of light refraction.

[0026] S2012. Based on the angle of light refraction, and the predetermined radius of curvature of the curved lens and the angle of incidence of light, calculate the refraction offset in the x-direction and the refraction offset in the y-direction. The expressions are as follows:

[0027]

[0028] In the formula, For image pixels in The amount of refraction offset in direction; For image pixels in The amount of refraction offset in direction; The x-coordinate of the image pixels; The x-coordinate of the optical center of the curved lens compartment; The vertical coordinate of the image pixels; The ordinate of the optical center of the curved lens compartment; For image Orientation pixel scale factor; For image Orientation pixel scale factor.

[0029] S2013. Based on the refraction offset in the x-direction and the refraction offset in the y-direction, the image data is corrected for pixel positions to obtain the refraction-corrected image data, the expression of which is:

[0030] In the formula, Pixel coordinates in the refraction-corrected image data Pixel value at; Coordinates in image data The pixel value at that location.

[0031] S202. Remove water ripple artifacts from the refraction-corrected image data to obtain image data without ripple artifacts. Based on the predetermined optical characteristics of the curved lens chamber and the coupling characteristics of underwater water ripples, a multi-frame image fusion and ripple feature filtering algorithm is used to eliminate water ripple artifacts and obtain ripple artifact-free image data. S203. Denoise the image data without ripple artifacts to obtain the denoised image data. Denoising can eliminate image noise generated by the underwater environment.

[0032] S204. Perform size normalization on the denoised image data to obtain preprocessed image data; By standardizing image size, the final preprocessed image data is obtained.

[0033] Furthermore, in step S20 above, the acquired ultrasonic data is underwater concrete detection data. During underwater acquisition, due to interference from complex environments, invalid outliers may be mixed into the data, and random fluctuations may occur, resulting in insufficient data accuracy and poor stability, making it unsuitable for direct use in subsequent crack depth measurement. Therefore, preprocessing of the ultrasonic data is necessary.

[0034] Preprocessing the ultrasound data to obtain preprocessed ultrasound data includes the following steps: S211. Perform outlier removal processing on the ultrasonic data to remove invalid data caused by underwater interference, and obtain the processed ultrasonic data. During underwater ultrasonic detection, bubbles, suspended particles, and transient interference from equipment circuits in the water can cause invalid anomalies in the echo signal amplitude.

[0035] Step S211 uses the sliding window 3σ criterion to automatically identify outliers and replaces outliers using linear interpolation to eliminate extreme interference data while preserving effective echo characteristics.

[0036] Specifically, firstly, the original ultrasound sampling sequence is traversed using a fixed-length sliding window, and the mean and standard deviation of the data within the window are calculated point by point. The expression is as follows:

[0037]

[0038] In the formula, The first in the original ultrasound data The amplitude of each sampling point; This refers to the sequence number of the ultrasonic sampling point; The length of the sliding window is a positive odd number; Use the index for traversing sampling points within the window; For the first The mean value of ultrasound data within the window corresponding to each sampling point; For the first The standard deviation of the ultrasound data within the window corresponds to each sampling point.

[0039] Next, outliers were identified based on the 3σ criterion and replaced using neighborhood linear interpolation to obtain the ultrasound data after outlier removal, expressed as:

[0040] In the formula, After outlier removal, the first Ultrasonic data values ​​from each sampling point; This represents the step size from the anomaly point to the first normal sampling point. This represents the step size from the outlier point to the first normal sampling point. When the amplitude of a sampling point deviates from the window mean by more than 3 times the standard deviation, it is determined to be an invalid outlier caused by underwater interference, and the interpolation result of the preceding and following normal points is used instead.

[0041] S212. Perform smoothing filtering on the processed ultrasound data to make the ultrasound data stable and accurate, obtaining preprocessed ultrasound data. This includes the following steps: Even after outlier removal, the ultrasonic data still exhibits minor random fluctuations due to water scattering and circuit thermal noise, which can reduce the accuracy of subsequent crack depth measurements.

[0042] Step S212 employs Gaussian weighted sliding smoothing filtering to suppress random noise and improve data stability while preserving key features such as echo leading edge and peak value.

[0043] Specifically, first, a Gaussian weighted filter kernel is constructed, with the weights following a Gaussian distribution based on the offset from the center of the window. Its expression is:

[0044] In the formula, For the first in the filter window The weight value of each position; This represents the offset of the sampling point within the window relative to the center of the window, with a value range of [value missing]. Integers within; The standard deviation of the Gaussian kernel is used to control the smoothing strength; the larger the value, the stronger the filtering and smoothing effect. The length of the filter window is a positive odd number.

[0045] Next, a Gaussian weighted moving average was used to smooth the ultrasound data point by point after outlier removal, resulting in the final preprocessed ultrasound data, expressed as:

[0046] In the formula, For the first preprocessed Ultrasonic data values ​​from each sampling point; After outlier removal, the first The ultrasonic data values ​​from each sampling point were used. Random noise was suppressed by weighted averaging, while key morphological features of the echo signal were preserved.

[0047] Although the preprocessed image data has eliminated distortion, artifacts, and noise interference and standardized its size, it still suffers from feature defects caused by the underwater environment. In deep water with extremely low light conditions, crack features on concrete surfaces become blurred, and minute cracks are difficult to identify effectively. Simultaneously, redundant interference information such as algae, suspended particles, scale, and other deposits remains in the image, resulting in low contrast between crack and background areas and insufficient crack feature purity. These issues prevent the preprocessed image data from being directly used for subsequent multi-feature fusion and crack detection, making it difficult to accurately extract effective crack information. Therefore, it is necessary to input the preprocessed image data into a pre-constructed crack feature enhancement model to obtain high-purity crack features. S30. Input the preprocessed image data into the pre-constructed crack feature enhancement model to obtain high-purity crack features; In step S30 above, the crack feature enhancement model includes a serially connected algorithm backbone network and an algorithm neck network. The algorithm backbone network includes a serially connected input layer, a low-level convolutional layer, a parallel feature extraction layer, and an output layer; the parallel feature extraction layer includes a parallelly connected edge feature branch layer and a texture feature branch layer. Wherein: The input layer is used to acquire the preprocessed image data after step S20.

[0048] The low-level convolutional layers use a combination of three sets of convolutions, ReLU activation function, and max pooling to downsample and extract low-level visual features from the preprocessed image data, thus obtaining the low-level convolutional features of the image. .

[0049] The edge feature branch layer uses the Sobel gradient operator to calculate the low-level convolutional features of the image output from the low-level convolutional layer. The gradient components are fused to obtain the basic features of the crack edge, and their expression is:

[0050] In the formula, for Oriented gradient components, for Oriented gradient components, These are the basic features of the crack edge.

[0051] The texture feature branch layer uses an optimized Gabor filter operator to apply the low-level convolutional features of the image output from the low-level convolutional layer. The basic features of the crack texture are extracted through processing, and their expression is as follows:

[0052] In the formula, The basic features of crack texture,

[0053] It is an improved Gabor filter core. In terms of filtering direction, As a scale factor, For the center frequency, For filter kernel size, , This is the traversal index for the filter kernel.

[0054] The output layer is used to concatenate the basic features of the crack edge and the basic features of the crack texture along the channel dimension to obtain the basic feature map. Its expression is:

[0055] In the formula, For channel-dimensional feature splicing, Based on the feature map.

[0056] The algorithm's neck network is serially connected to the algorithm's backbone network. It receives the basic feature map output by the backbone network, learns optimal feature enhancement parameters autonomously during the training phase, and utilizes these learned parameters to purify and enhance crack features during the inference phase, ultimately outputting high-purity crack features. The algorithm's neck network includes a serially connected basic feature input layer, a multi-scale sampling layer, a feature purification and redundancy removal layer, an optimal parameter learning layer, and a feature enhancement output layer. Wherein: The basic feature input layer is used to obtain the basic feature maps output by the algorithm's backbone network. ; The multi-scale sampling layer uses three sets of convolutional kernels of different scales: 1×1, 3×3, and 5×5, to sample the basic feature maps. Multi-scale convolution processing is performed to generate three sets of basic features at different scales. , , The three sets of basic features at different scales were weighted and fused to obtain multi-scale fused features. Its expression is:

[0057] In the formula, For multi-scale fusion features, The adaptive weights for the k-th scale features. This represents the basic feature at the k-th scale.

[0058] The feature purification and redundancy removal layer employs a channel attention mechanism to process multi-scale fused features. After weighted processing to remove redundant and interfering information such as algae, suspended particles, and scale, the pure features after removing redundancy are obtained, and their expression is:

[0059] In the formula, To remove redundant features and achieve a clean profile. This is a channel attention weight map.

[0060] The optimal parameter learning layer is executed only during the training phase. Its optimization objective is to maximize the feature contrast between the crack region and the background region. A contrast loss function is constructed, and this loss function is minimized using the gradient descent algorithm. The optimal gamma correction coefficients and optimal multi-scale weights are then learned autonomously, and their expressions are as follows:

[0061]

[0062] In the formula, Let be the contrast loss function. The characteristic variance calculation function is... Characteristics of the cracked area. The optimal gamma correction coefficient is... The optimal multi-scale weights are for the k-th scale.

[0063] The feature enhancement output layer is executed only during the inference and deployment phase. Based on the optimal gamma correction coefficients and optimal multi-scale weights learned during the training phase, it enhances the purity of features. Multi-scale histogram equalization enhancement processing is performed to obtain high-purity crack features. Its expression is:

[0064] In the formula, Characterized by high-purity cracks. For the k-th scale histogram equalization operation, These are the low-level convolutional features of the image after optimal gamma correction.

[0065] S40. Input the high-purity crack features into the pre-constructed weak light multi-feature fusion model to obtain the fused features; Specifically, the low-light multi-feature fusion model is designed for extremely low light conditions in deep water areas. It is used to extract and weightedly fuse high-purity crack features in multiple dimensions, thereby enhancing the expression of crack features in low-light environments. The low-light multi-feature fusion model includes a crack feature input layer, a parallel feature extraction layer, an adaptive weighted fusion layer, and an output layer that are connected in a series. The parallel feature extraction layer includes parallel texture feature branches, edge feature branches, and detail semantic feature branches.

[0066] The crack feature input layer is used to obtain the high-purity crack features output in step S30. ; The low-light texture feature extraction layer employs an optimized Gabor filter operator to enhance images in low light conditions. Based on this, crack texture features are extracted under low-light conditions, and their expression is:

[0067] In the formula, The texture features are those of a crack in low light. It is an improved Gabor filter core. In terms of filtering direction, As a scale factor, For the center frequency, For filter kernel size, , This is the traversal index for the filter kernel.

[0068] It should be further explained that low-light enhancement images High-purity crack features output from step S30 The feature mapping reconstruction yields a pixel-level image that combines crack semantic guidance with grayscale details by mapping multi-channel high-purity semantic crack features back to the pixel space. This image serves as a unified pixel-level computational basis for subsequent feature extraction of the three branches: texture, edge, and detail semantics.

[0069] The edge feature branch addresses the problem of weak edge gradients under extremely low illumination by employing the Sobel gradient operator combined with an adaptive threshold to extract crack edge features. Its expression is as follows:

[0070]

[0071]

[0072] In the formula, The gradient component is in the x-direction. The gradient component is in the y-direction. Features of the crack edge This is an adaptive threshold.

[0073] The detail semantic feature branch extracts crack detail semantic features through convolution operations and the ReLU activation function. Its expression is as follows:

[0074] In the formula, For the semantic features of crack details, The number of feature channels, For convolution kernel weights, For bias, This is the ReLU activation function.

[0075] The adaptive weighted fusion layer weights and fuses the three types of features output from the texture feature branch, edge feature branch, and detail semantic feature branch according to their contribution, maximizing the proportion of effective features. Its expression is:

[0076] In the formula, Features of fusion; , , For adaptive weights.

[0077] The output layer is used to output the final fused features. .

[0078] S50. Based on the fusion features and preprocessed ultrasonic data, the crack region is segmented to obtain the crack region identification result.

[0079] Specifically, preprocessed ultrasound data Including the thickness of the attachment Penetrating remaining energy Media type Ultrasonic feature template of crack zone Initial launch energy and physical ultrasound data .

[0080] Based on the preprocessed ultrasound data, the fusion features are quantized and correlated, parameters are corrected, and occlusion areas are enhanced to provide highly discriminative features for segmentation. The specific steps include the following: S501. Establish the basic correlation between fusion features and physical ultrasound data, complete the global matching calibration of visual and ultrasound signals, and use the obtained basic correlation coefficient as the normalization benchmark for subsequent pixel-level correlation determination to eliminate the overall modal matching deviation of the two types of signals under different underwater environments. Its expression is:

[0081] In the formula, The basic correlation coefficient between visual features and physical ultrasound data; This refers to the preprocessed physical ultrasound data; This is the function for calculating the Pearson correlation coefficient.

[0082] S502. Based on the correlation between the components of each scale of the fusion feature and the ultrasonic medium type data, the multi-scale feature fusion weights are adaptively corrected. The original multi-scale fusion components are then weighted twice to obtain the medium-adapted fusion feature. The feature enhancement intensity is adapted to different medium regions such as concrete, attachments, and water. Its expression is as follows:

[0083]

[0084] In the formula, , These are the corrected and original k-th scale feature fusion weights, respectively. For correction factor, For the k-th scale fusion feature, For ultrasonic media type data, This refers to the fusion features after media adaptation correction.

[0085] S503, relying on the penetrating energy characteristics of ultrasound, enhances the expression of crack features in the shielded area, highlighting crack traces covered by thin mud and scale. Its expression is:

[0086] In the formula, To enhance the characteristic values ​​of cracks in the shielded area, The penetration enhancement coefficient, This represents the residual energy of the ultrasound wave after it penetrates the attached material. This represents the initial energy value of the ultrasound.

[0087] S504. Combining the ultrasonic feature template of the crack area with the correlation threshold, the pixel attributes are determined to distinguish between cracks and interference items such as attached materials and suspended particles. The expression is as follows:

[0088]

[0089] In the formula, Pixels identified as cracks Pixels identified as attachments or interference items As a template for ultrasonic features in the crack zone, This is the initial correlation threshold. This is the actual judgment threshold after calibration with the global basic correlation coefficient.

[0090] Furthermore, the loss function used for training the aforementioned crack region segmentation model is a hybrid loss function composed of cross-entropy loss and Dice loss, used to balance pixel-level segmentation accuracy and contour integrity.

[0091] Specifically, the expression for the hybrid loss function is:

[0092] The partial loss formulas include cross-entropy loss and Dice loss. The cross-entropy loss formula is as follows:

[0093] The Dice loss formula is:

[0094] The formula for outputting pixel probabilities in segmentation calculation is:

[0095] in, These are the weighting coefficients. For real labels, The probability of crack pixels predicted by the model; Use the Sigmoid activation function; The number of layers in a multi-scale semantic segmentation network. For the convolution / fusion operation of the l-th layer, The fused features are output by the multi-scale feature fusion subnetwork; For the first The weights and biases of the hierarchy, The number of pixel rows (total number of pixels in the height direction) of the input image; The number of pixel columns (total number of pixels in the width direction) of the input image.

[0096] Based on the same inventive concept as the above method, this invention also proposes a deep learning-based intelligent identification system for concrete surface cracks, comprising: The data acquisition module is used to acquire image data and ultrasound data; The data preprocessing module is used to preprocess image data and ultrasound data to obtain preprocessed image data and preprocessed ultrasound data. The crack feature enhancement module is used to input the preprocessed image data into the pre-built crack feature enhancement model to obtain high-purity crack features; The low-light multi-feature fusion module is used to input high-purity crack features into a pre-built low-light multi-feature fusion model to obtain fused features; The crack region segmentation module is used to segment crack regions based on fused features and preprocessed ultrasonic data to obtain crack region identification results.

[0097] Based on the same inventive concept as the above method, the present invention also proposes a device, including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and the processor executes the above-described deep learning-based intelligent identification method for concrete surface cracks when executing the computer instructions.

[0098] Based on the same inventive concept as the above method, the present invention also proposes a computer-readable storage medium storing computer instructions, which, when executed by the computer instructions, can realize the above-mentioned intelligent identification method for concrete surface cracks based on deep learning.

[0099] Any references to memory, storage, database, or other media used in the embodiments provided in this invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0100] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0101] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent identification of concrete surface cracks based on deep learning, characterized in that, include: Acquire image and ultrasound data; The image data and ultrasound data are preprocessed to obtain preprocessed image data and preprocessed ultrasound data; The preprocessed image data is input into a pre-constructed crack feature enhancement model to obtain high-purity crack features; The high-purity crack features are input into a pre-constructed weak-light multi-feature fusion model to obtain fused features; Based on the fused features and the preprocessed ultrasonic data, the crack region is segmented to obtain the crack region identification result.

2. The intelligent identification method for concrete surface cracks based on deep learning according to claim 1, characterized in that, The image data is preprocessed to obtain preprocessed image data, specifically including: Underwater light refraction distortion correction is performed on the image data to obtain the refraction-corrected image data; Water ripple artifact removal is performed on the refraction-corrected image data to obtain ripple artifact-free image data; The image data without ripple artifacts is denoised to obtain denoised image data. The size of the denoised image data is normalized to obtain the preprocessed image data.

3. The intelligent identification method for concrete surface cracks based on deep learning according to claim 2, characterized in that, The underwater light refraction distortion correction of the image data to obtain the refraction-corrected image data specifically includes: The angle of light refraction is calculated based on the refractive index of the water, the refractive index of the lens medium, and the angle of light incidence. Based on the light refraction angle, the radius of curvature of the curved lens, and the light incident angle, the pixel refraction offset in the x and y directions is calculated respectively. The image data is corrected by adjusting the pixel position based on the refraction offset in the x and y directions to obtain the refraction-corrected image data.

4. The intelligent identification method for concrete surface cracks based on deep learning according to claim 1, characterized in that, The ultrasound data is preprocessed, specifically including: The sliding window 3σ criterion was used to automatically identify and remove outliers from the ultrasound data, and the outlier sampling points were replaced by neighborhood linear interpolation to obtain the ultrasound data after outlier removal. Gaussian weighted sliding smoothing filter is used to suppress noise in the ultrasound data after outlier removal, resulting in preprocessed ultrasound data.

5. The intelligent identification method for concrete surface cracks based on deep learning according to claim 1, characterized in that, The crack feature enhancement model includes a serially connected algorithm backbone network and an algorithm neck network; The algorithm's backbone network comprises a serially connected input layer, a low-level convolutional layer, a parallel feature extraction layer, and an output layer. The parallel feature extraction layer includes a parallelly connected edge feature branch layer and a texture feature branch layer. The low-level convolutional layer is used to extract low-level convolutional features of the image. The edge feature branch layer uses the Sobel gradient operator to calculate gradient components and fuse them to obtain basic crack edge features. The texture feature branch layer uses an optimized Gabor filter operator to extract basic crack texture features. The output layer concatenates the basic crack edge features and the basic crack texture features along the channel dimension to obtain a basic feature map.

6. The intelligent identification method for concrete surface cracks based on deep learning according to claim 5, characterized in that, The algorithm's neck network comprises a serially connected basic feature input layer, a multi-scale sampling layer, a feature purification and redundancy removal layer, an optimal parameter learning layer, and a feature enhancement output layer. The multi-scale sampling layer uses three sets of convolutional kernels of different scales to process the basic feature map, and obtains multi-scale fused features through adaptive weighted fusion. The feature purification and redundancy removal layer uses a channel attention mechanism to weight the multi-scale fused features in order to remove redundant interference information and obtain pure features. The optimal parameter learning layer aims to maximize the feature contrast between the crack region and the background region, constructs a contrast loss function, and autonomously learns the optimal gamma correction coefficient and the optimal multi-scale weight. The feature enhancement output layer uses the optimal gamma correction coefficient and optimal multi-scale weights obtained during training to perform multi-scale histogram equalization enhancement on the pure features, and outputs high-purity crack features.

7. The intelligent identification method for concrete surface cracks based on deep learning according to claim 1, characterized in that, The process of inputting the high-purity crack features into a pre-constructed weak-light multi-feature fusion model to obtain fused features specifically includes: After inputting high-purity crack features into the low-light multi-feature fusion model, pixel-level low-light enhanced images are first obtained through feature mapping reconstruction. Then, features of corresponding dimensions are extracted through parallel texture feature branches, edge feature branches, and detail semantic feature branches. Finally, fused features are obtained through adaptive weighted fusion. Among them, the texture feature branch uses an optimized Gabor filter operator to extract crack texture features in low light environment, the edge feature branch uses the Sobel gradient operator combined with an adaptive threshold to extract crack edge features in low light environment, and the detail semantic feature branch extracts crack detail semantic features through convolution operation and activation function.

8. The intelligent identification method for concrete surface cracks based on deep learning according to claim 1, characterized in that, The step of segmenting the crack region based on the fused features and the preprocessed ultrasonic data to obtain the crack region identification result specifically includes: Establish the basic correlation between fusion features and physical ultrasound data, and calculate the basic correlation coefficient between visual features and physical ultrasound data; Based on the correlation between the components of each scale of the fusion feature and the ultrasonic medium type data, the fusion weight of the multi-scale feature is adaptively corrected, and the original multi-scale fusion components are weighted twice to obtain the fusion feature after medium adaptation. Based on the characteristics of ultrasonic penetration energy, the fusion features after medium adaptation are enhanced in a directional manner in the shielded area to obtain the crack features after the shielded area is strengthened. The correlation coefficient is used to calibrate the correlation determination threshold. The ultrasonic feature template of the crack area is combined to determine the pixel-by-pixel attribute of the enhanced crack feature in the occluded area, distinguish crack pixels from interference pixels, and output the crack area identification result.

9. The intelligent identification method for concrete surface cracks based on deep learning according to claim 1, characterized in that, The correlation coefficient is used to calibrate the correlation determination threshold. Then, the ultrasonic feature template of the crack area is used to perform pixel-by-pixel attribute determination of the enhanced crack features in the occluded area, distinguishing crack pixels from interference pixels. The expression for the crack region identification result is as follows: In the formula, Pixels identified as cracks Pixels identified as attachments or interference items As a template for ultrasonic features in the crack zone, This is the actual judgment threshold after calibration with the global basic correlation coefficient.

10. A deep learning-based intelligent identification system for concrete surface cracks, characterized in that, include: The data acquisition module is used to acquire image data and ultrasound data; The data preprocessing module is used to preprocess the image data and ultrasound data to obtain preprocessed image data and preprocessed ultrasound data. The crack feature enhancement module is used to input the preprocessed image data into a pre-constructed crack feature enhancement model to obtain high-purity crack features; The low-light multi-feature fusion module is used to input the high-purity crack features into the pre-constructed low-light multi-feature fusion model to obtain fused features; The crack region segmentation module is used to segment the crack region based on the fused features and the preprocessed ultrasonic data to obtain the crack region identification result.

Images