A concrete crack and damage detection research method and system based on double-flow fusion features

Abstract

The present application relates to a kind of concrete crack damage detection research method and system based on double-flow fusion feature, belong to detection technical field, solve the problem of low detection precision, low detection efficiency and unable to adapt to different scenes in prior art.The method of the present application, pre-processes a group of input concrete surface images, extracts the global features and local detail features of the processed images through a double-flow model, obtains the reinforced crack features by interactive fusion of global features and local features, and finally obtains the prediction results through an activation function, realizing pixel-level classification.High-precision crack detection of concrete surface images is realized, and high detection accuracy can be maintained in different scenarios.

Description

Technical Field

[0001] This invention belongs to the field of detection technology and relates to a research method and system for detecting concrete cracks based on dual-flow fusion characteristics. Background Technology

[0002] Currently, my country is in a stage of rapid economic development, and railway transportation is a vital artery of the national economy. However, with the increase in the time of operation, affected by climate, environment, and service life, the concrete surface of bridges gradually shows signs of deterioration such as cracks and localized damage. Steel structures of bridges and column-free canopies of stations exhibit signs of deterioration such as peeling of anti-corrosion layers, localized rust, loosening of connections, and missing high-strength bolts. These defects accelerate concrete carbonation, reduce the concrete's resistance to various corrosive media, and affect the structural strength and stability of concrete structures, posing potential risks to safe operation. Therefore, effectively and quickly detecting defects in the concrete of bridges and other infrastructure has become one of the important requirements for ensuring the safety of railway and other infrastructure facilities.

[0003] To address the problem of identifying and detecting surface deterioration such as cracks in concrete, machine vision-based image processing technology has been widely used to replace traditional manual visual inspection. However, current machine vision-based image processing technologies for identifying and detecting surface deterioration such as cracks in concrete suffer from several drawbacks: firstly, the detection accuracy of existing methods is insufficient, resulting in mostly coarse detection results with significant differences between the crack edges and the actual crack edges; secondly, the detection speed is relatively low; and thirdly, the detection performance is poor in complex backgrounds, easily introducing indistinguishable noise that may lead to the segmentation of areas that are not actually cracks. Summary of the Invention

[0004] In view of the above problems, the present invention provides a concrete crack detection method and system based on a dual-stream fusion feature extraction architecture, which solves the problems of low detection accuracy, low detection efficiency and inability to adapt to different scenarios in the prior art.

[0005] This invention provides a method for detecting concrete cracks based on a dual-flow feature fusion architecture, the specific steps of which are as follows:

[0006] S1. Preprocess the image to be detected to obtain the processed image;

[0007] S2. Use a two-stream feature extraction architecture to extract local and global features from the processed image;

[0008] The dual-stream feature extraction architecture consists of a local convolutional neural network model and a self-attention network model; local and global features of the processed image are extracted based on the local convolutional neural network model and the self-attention network model, respectively.

[0009] S3. Combine the global and local features obtained in step 2 to obtain the fused features;

[0010] S4. Obtain predicted features by performing edge optimization processing based on fused features;

[0011] Specifically, the fused features are decoupled into main features and edge features through flow deformation; the main features and edge features are then added together to obtain the predicted features.

[0012] S5. Based on the predicted features, predict the results to obtain the image damage detection results.

[0013] Optionally, the image to be detected is cropped to a custom size to obtain a processed image; the processed image is then normalized.

[0014] Optionally, the local convolutional neural network model is the ResNet50 model; the self-attention network model is the Vision Transformer model.

[0015] Optionally, in the ResNet50 local convolutional neural network model, convolution operations are performed on the processed image using a sliding window of the same size as the convolution kernel to extract local features of the processed image; local features of the processed image X are obtained through the calculation of multiple convolutional layers.

[0016] Optionally, in the Vision Transformer model, the image is divided into several small image blocks of fixed size; each small image block is input into an image encoding program to obtain the encoding of each image block; the encoded image blocks are input into multiple progressively downsampled global feature extraction layers to extract the global features of the processed image.

[0017] Optionally, when fusing global and local features, the correlation between global and local features is obtained through attention calculation, and global features are used to provide global structural information for local features to enhance local information; based on the correlation between global and local features and the enhanced local information, fused features are obtained.

[0018] Optionally, during the flow deformation operation, pseudo-cluster centers are obtained, and each element in the fused features is aligned to the pseudo-cluster centers by learning the feature offset field to obtain the main features; the main features in the fused features are subtracted to obtain the edge features; and the main features and edge features are added together to obtain the predicted features.

[0019] Optionally, the predicted features are passed through an activation function to output the prediction result.

[0020] Another aspect of the present invention discloses a concrete crack detection system based on dual-stream fusion features, including an image acquisition module, a preprocessing module, a feature extraction module, a feature fusion module, an edge optimization module, and a model prediction module;

[0021] The image acquisition module is used to acquire a set of shallow concrete surface images;

[0022] The preprocessing module is used to preprocess a set of input concrete surface images; then, normalization processing is performed.

[0023] The feature extraction module is used to extract features from an image;

[0024] The feature fusion module is used to fuse the features of the extracted image based on their relevance to obtain an optimized image.

[0025] The edge optimization module is used to segment and optimize edges in an image;

[0026] The model prediction module is used to predict the results of image detection.

[0027] Compared with the prior art, the present invention has at least the following beneficial effects:

[0028] (1) The method of the present invention is not sensitive to the background texture in the image and can produce features with good generalization ability.

[0029] (2) The method of the present invention can accurately detect cracks in concrete surface images, and the detection results have relatively accurate edges. Attached Figure Description

[0030] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0031] Figure 1 This is a flowchart of the crack damage detection method of the present invention. Detailed Implementation

[0032] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0033] A specific embodiment of the present invention, such as Figure 1 A research method for concrete crack detection based on a dual-flow feature fusion architecture is presented, with the following specific steps:

[0034] S1. Preprocess the image to be detected to obtain the processed image;

[0035] The specific steps of preprocessing are as follows:

[0036] The image to be detected, X0, is randomly cropped to a custom size to obtain the processed image X. The cropped processed image X is then normalized so that the pixel values ​​of the processed image X are adjusted to the range of [0,1].

[0037] S2. Extract local and global features from image X;

[0038] Optionally, a two-stream feature extraction architecture is used to extract local and global features of the processed image X; the two-stream feature extraction architecture consists of a local convolutional neural network model and a self-attention network model, which respectively extract local and global features of the image X based on the local convolutional neural network model and the model.

[0039] Preferably, the local convolutional neural network model is the ResNet50 model; the self-attention network model is the Vision Transformer model.

[0040] Furthermore, the dual-stream feature extraction architecture consists of two parallel feature extraction models; the dual-stream feature extraction architecture extracts both local and global features simultaneously.

[0041] The Vision Transformer model is used to extract global features from the processed image to filter background noise; the ResNet50 local convolutional neural network model is used to extract local features from the image. Preferably, global features are information about the global structure, such as high-level semantic information, such as identifying the approximate location of the target object, while local features are information about local details, such as the edges of the image.

[0042] In the ResNet50 local convolutional neural network model, convolution operations are performed on the image X using a sliding window of the same size as the convolution kernel W to extract local features of the image. Residual connections are used in the ResNet50 model to prevent gradient vanishing. The local features F of the image X are obtained through computation across multiple convolutional layers. l The expression is:

[0043] F l =ResNet(X)=F(X,{})+;

[0044] Where F(X,{Wi}) represents the convolution operation; X is the information of the input image X to be processed.

[0045] Optionally, the information for processing image X is the set of pixels of image X.

[0046] In the Vision Transformer model, the image X to be processed is divided into N small image blocks P of size p×p. i,j ; divide each small image block P i,jThe encoding P of each image block is obtained in the input image encoding program. x The expression is:

[0047] P i,j =[p 2 [,N,C];

[0048] P x =Batch_Norm(Conv(P i,j ));

[0049] Among them, P i,j Let P be the image patch located in the i-th row and j-th column; p is the side length of the image patch; N is the image patch P. i,j The total number; C is the number of image channels;

[0050] The encoded image block P x Input four progressively downsampled global feature extraction layers TL m Extract global features F from the image. g The expression is:

[0051]

[0052] S3. Combine the global and local features obtained in step 2 to obtain the fused features;

[0053] Fusion of global features F g With local features F l Attention-based computation is used to perceive the correlation between global and local features, thereby strengthening effective features and weakening redundant features. Simultaneously, global features provide global structural information to local features, enhancing local information, while local features provide low-level detail information such as object edge information. Based on the enhanced fused features, the edges of the image to be recognized are more clearly obtained and are not interfered with by objects with other similar features. The fused feature expression is:

[0054]

[0055] CoFuse is the feature fusion processing module. "t" represents the activation function, and "·" represents the tensor dot product operation.

[0056] S4. Obtain predicted features by performing edge optimization processing based on fused features;

[0057] F fusion feature F fuse Decoupling as the main feature F body and edge features F edge ;The main feature F body and edge features F edge Add to obtain the predicted feature F final .

[0058] Optionally, the fusion feature F can be fused through flow deformation. fuse Decoupling to obtain the main feature F body and edge features F edge .

[0059] Furthermore, during the flow deformation operation, the pseudo-cluster centers F are obtained. flow By learning the feature offset field, the fused feature F fuse Each element in the cluster points to the pseudo-cluster center F. flow Alignment yields main feature F body To maintain internal consistency among similar objects; in the fusion feature F fuse Subtract the main feature F from the basis body Obtain edge features F edge The predicted feature F is obtained by adding the main features and edge features. final The expression is:

[0060] F body =B wrap (F fuse Grid h×w +F flow );

[0061] F edge =F fuse -G body ;

[0062] F final =F edge +F body .

[0063] Among them, Grid h×w B represents the reference object for irregular downsampling. wrap This is a flow deformation operation.

[0064] Fusion feature F final Edge-mapping effectively improves the edge prediction accuracy of the detection method.

[0065] S5. Based on the predicted features, predict the results to obtain the final image crack detection results;

[0066] Predictive feature F final After activation, the prediction result Y is output; further, the prediction result Y is the probability value of cracking; the final result is determined by the cracking threshold. If the probability value is greater than the cracking threshold, it is determined to be cracked; if the probability value is less than or equal to the cracking threshold, it is a background pixel.

[0067] Preferably, the damage threshold is 0.5.

[0068] Another specific embodiment of the present invention discloses a concrete crack detection system based on dual-stream fusion features, using the aforementioned concrete crack detection method, including an image acquisition module, a preprocessing module, a feature extraction module, a feature fusion module, an edge optimization module, and a model prediction module.

[0069] The image acquisition module is used to acquire a set of shallow concrete surface images, enabling the detection system to perform batch automatic detection.

[0070] The preprocessing module is used to preprocess the input set of concrete surface images before feature extraction; the concrete surface images are cropped into the same shape and size and then normalized.

[0071] The feature extraction module is used to extract features from an image, including local feature extraction and global feature extraction.

[0072] The feature fusion module is used to fuse global and local features based on their correlation to obtain an optimized image, so as to achieve the interaction between global and local features, which preserves high-level semantic information without losing detailed information.

[0073] The edge optimization module is used to segment and optimize the edges in the image, improving the edge clarity. It decomposes the extracted features into main features and edge features, decouples them, and uses multi-task learning to improve the accuracy of edge detection.

[0074] The model prediction module is used to predict the final image detection results.

[0075] Optionally, the preprocessing module uses the PIL module to read a set of concrete surface images acquired by the image acquisition module.

[0076] To illustrate the effectiveness of the method proposed in this invention, the following detailed description of the above technical solution is provided through a specific embodiment. The specific implementation steps are as follows:

[0077] Step 1: Collect the images to be detected, randomly crop the images to 384 pixels in both width and height, and then normalize the images to adjust all pixel values ​​to the range of 0 to 1.

[0078] Step 2: Input the image processed in Step 1 into both the ResNet network and the Vision Transformer network. ResNet is used to extract local features of the image, while Vision Transformer is used to extract global features.

[0079] The third step is to integrate the acquired global and local features through a fusion module, calculate the feature correlation using attention calculation, and multiply the correlation coefficient by the feature to perform interactive fusion.

[0080] Step 4: Perform edge optimization based on the fused features. First, extract the main features, then subtract the main features from the fused features to obtain the edge features. Finally, add the edge features to the main features to obtain the accurate features.

[0081] Step 5: Apply an activation function to the feature map to obtain a probability map with a confidence interval of 0 to 1. Use a set threshold of 0.5 to determine the category. If the probability value is greater than 0.5, it is determined to be a crack; otherwise, it is a background pixel.

[0082] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A concrete crack detection method based on a dual-flow feature fusion architecture, characterized in that, The specific steps are as follows: S1. Preprocess the image to be detected to obtain the processed image; S2. Use a two-stream feature extraction architecture to extract local and global features from the processed image; The dual-stream feature extraction architecture consists of a local convolutional neural network model and a self-attention network model; local and global features of the processed image are extracted based on the local convolutional neural network model and the self-attention network model, respectively. The local convolutional neural network model is the ResNet50 model; the self-attention network model is the Vision Transformer model. In the ResNet50 model, convolution operations are performed on the processed image using a sliding window of the same size as the convolution kernel to extract local features of the processed image; the processed image is obtained through the computation of multiple convolutional layers. X Local features; In the Vision Transformer model, the image is divided into several small image blocks of fixed size; each small image block is input into an image encoding program to obtain the encoding of each image block; the encoded image blocks are input into multiple progressively downsampled global feature extraction layers to extract the global features of the processed image; S3. Combine the global and local features obtained in step 2 to obtain the fused features; When fusing global and local features, the correlation between global and local features is obtained through attention calculation. Global features are used to provide global structural information for local features to enhance local information. Based on the correlation between global and local features and the enhanced local information, fused features are obtained. S4. Obtain predicted features by performing edge optimization processing based on fusion features; Specifically, the fused features are decoupled into main features and edge features through flow deformation; the main features and edge features are then added together to obtain the predicted features. During the flow deformation operation, pseudo-cluster centers are obtained. By learning the feature offset field, each element in the fused feature is aligned to the pseudo-cluster centers to obtain the main features. The main features in the fused features are subtracted to obtain the edge features. S5. Based on the predicted features, predict the results to obtain the image damage detection results.

2. The crack damage detection method according to claim 1, characterized in that, The image to be detected is cropped to a custom size to obtain the processed image; the image is then normalized.

3. The crack damage detection method according to claim 1, characterized in that, The predicted features are processed by an activation function to output the prediction results.

4. A concrete crack detection system based on dual-flow fusion features, comprising using the concrete crack detection method based on dual-flow feature fusion architecture as described in any one of claims 1-3 for concrete crack detection, characterized in that, It includes an image acquisition module, a preprocessing module, a feature extraction module, a feature fusion module, an edge optimization module, and a model prediction module; The image acquisition module is used to acquire a set of shallow concrete surface images; The preprocessing module is used to preprocess a set of input concrete surface images; then, normalization processing is performed. The feature extraction module is used to extract features from an image; The feature fusion module is used to fuse the features of the extracted image based on their relevance to obtain an optimized image. The edge optimization module is used to segment and optimize edges in an image; The model prediction module is used to predict the results of image detection.

Images