Gallery crack detection network oriented to visual semantic segmentation field
Through a corridor crack detection network using an encoder-decoder architecture and self-attention mechanism, the problem of low detection efficiency of existing models in concrete structure health monitoring is solved, efficient and accurate crack segmentation is achieved, and expanded to other concrete scenarios.
Patent Information
- Application Number
- CN202510537913.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-27
- Publication Date
- 2025-08-08
AI Technical Summary
Existing deep learning models such as FCN, DeepLab and U-net are difficult to achieve the best balance between accuracy and efficiency in concrete structure health monitoring, and are unable to efficiently detect concrete cracks.
The corridor crack detection network adopts an encoder-decoder architecture, combining self-attention mechanism and modular design, includes an encoder with a 19-layer feature extraction module and a downsampling module, and a decoder with a 7-layer upsampling and convolution module. Through the self-attention mechanism and a hybrid fully connected feedforward neural network, the accuracy and efficiency of crack detection are improved.
It achieves significant improvement in computing efficiency and detection accuracy without reducing performance, and can more effectively segment the corridor concrete cracks and apply them to other concrete scenarios.
Smart Images
Figure CN120451092A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision, and in particular to a corridor crack detection network oriented to the field of visual semantic segmentation, which is used for semantic segmentation tasks in images or videos. Background Art
[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.
[0003] Over time and under the combined influence of environmental factors, concrete structures gradually develop surface defects such as cracks, wear, cavitation, and erosion. Cracks pose the greatest threat of these defects. They not only persist on the surface but are also likely to penetrate deep within the structure, posing a significant threat to the overall safety of the structure. Currently, crack detection in concrete structures relies primarily on manual screening and recording. Traditional manual inspections for concrete cracks are not only laborious and resource-intensive, but also extremely inefficient, failing to achieve rapid detection. With the rapid development of computer science, the use of computer vision to accurately detect concrete cracks has become a major research goal.
[0004] Yu et al. used the Sobel operator to locate crack edges in images. Through region filling, labeling, and adjacent region joining operations, they identified the actual cracks in the images. Li et al. used a wavelet-based algorithm to filter out image noise and combined the CV model with the Canny edge detector to segment cracks. Compared with the original Canny edge detector, the proposed method outperformed the original Canny edge detector. Similarly, Li et al. used the Canny edge detector to extract cracks from images and optimized the crack shape using a region-based active contour model. They then employed a support vector machine and a greedy search strategy to remove noise caused by crack surface contamination and uneven illumination, ultimately achieving pixel-level crack detection results with clear shapes. Although traditional digital image processing techniques for crack detection have been extensively studied, their shortcomings are also significant. Traditional image processing techniques rely on manual feature extraction and segmentation, limiting their application scenarios.
[0005] Since AlexNet won the ImageNet competition in 2012, Convolutional Neural Networks (CNN) have become increasingly popular in the field of computer vision. Convolutional neural networks not only perform well in crack image classification, but also have made significant progress in crack image segmentation. For example, Ren et al.
[50] used dilated convolution and spatial pyramid pooling modules to fill the deep full convolutional neural network for efficient multi-scale feature extraction, aggregation and resolution reconstruction, which has high accuracy and generalization ability for crack detection; Qing et al. proposed a semantic segmentation tunnel crack objective and rapid recognition algorithm based on computer vision for complex situations such as water stains, scratches, structural seams, etc. in tunnel images; Wooram et al. used standard convolution, DenSep module, improved spatial pyramid pooling (ASPP) module and decoder module to form SDDNet network. Experimental results show that SDDNet is very effective for segmented crack segmentation; Liu et al.
[60] chose to learn and aggregate multi-scale features from low convolution layers to high convolution layers during training. The deep neural network DeepCrack was obtained by combining multi-level features with guided filtering and conditional random fields (CRFs) to optimize the model, and an F-score of 86.5% was obtained. Ju et al. proposed CrackU-net, which increased the depth of U-Net and outperformed the original U-Net in pavement crack detection. Lei et al. proposed a triple attention (TA) module to detect spatial, channel and pixel attention information to suppress background and useless information, which is beneficial to the characterization of microcracks. Cao et al. proposed the HC-Unet++ network segmentation model for road crack detection to solve the problem of crack discontinuity in segmentation results. SUN et al. added a multi-scale attention module to the decoder of DeepLabv3+ to effectively reduce the number of cracks missed.
[0006] In summary, semantic segmentation methods are highly effective for fine-grained crack identification. However, existing FCN, DeepLab, and U-net methods are widely used for scene parsing or biomedical image segmentation. The morphological characteristics, spatial distribution, and data distribution of concrete cracks differ significantly from those in publicly available datasets. Therefore, it is necessary to further refine the network architecture based on the characteristics of crack datasets to meet the crack detection needs of the structural health monitoring field. Summary of the Invention
[0007] The purpose of the present invention is to: target existing well-known deep learning models such as FCN, DeepLab and U-net, which are widely used in multi-category datasets such as PASCAL VOC, ImageNet, COCO, etc. In contrast, in concrete structure health monitoring, monitors only need to pay attention to a small number of categories, or even at least one category, such as cracks. Therefore, a CNN architecture can be developed that achieves the best balance between accuracy and efficiency for specific tasks and specific fields, which can significantly improve computational efficiency without reducing performance. For these reasons, the present invention provides a corridor crack detection network for the field of visual semantic segmentation. The network architecture follows the encoder-decoder architecture and adopts a modular design concept, which is robust, efficient and versatile. The network can more effectively segment corridor concrete cracks and can be applied to other concrete scenarios.
[0008] The technical solutions of the present invention are as follows:
[0009] A corridor crack detection network for the field of visual semantic segmentation is constructed using an encoder-decoder architecture. The encoder contains a 19-layer structure consisting of a feature extraction module based on the self-attention mechanism and a downsampling module, while the decoder contains a 7-layer structure consisting of an upsampling module and a convolution module.
[0010] Furthermore, the encoder reduces the spatial dimension to 1 / 8 of the original input through three downsampling operations with a stride of 2. The downsampling module is composed of a 3×3 convolutional layer and a maximum pooling module with a stride of 2 in cascade.
[0011] Furthermore, the decoder restores the original input resolution through three inverse convolution operations with a stride of 2, and the convolution kernel stride of the inverse convolution is set to 2 and the padding parameter is set to 1.
[0012] Furthermore, the feature extraction module based on the self-attention mechanism includes a residual structure, point-by-point convolution, a self-attention mechanism with an expansion rate, and a hybrid fully connected feedforward neural network.
[0013] Furthermore, the calculation process of the self-attention mechanism includes:
[0014] a: Calculate the query vector, key vector and value vector respectively;
[0015] b: Calculate attention weight;
[0016] c: Weighted summation to obtain the final output representation.
[0017] Furthermore, for each position in the input data, a query vector, a key vector, and a value vector are obtained through linear transformation.
[0018] Furthermore, the attention weight is obtained by calculating the dot product between the query vector and the key vector and then normalizing it.
[0019] Furthermore, the output is represented as follows:
[0020]
[0021] in,
[0022] Q represents the query vector;
[0023] K represents the key vector;
[0024] V represents the value vector;
[0025] d head Indicates the length of the vector key vector.
[0026] Furthermore, the residual structure is specifically implemented as follows:
[0027] After the input features are expanded in dimension by the first layer of point-by-point convolution, they pass through the self-attention mechanism and the hybrid fully connected feedforward neural network in sequence;
[0028] The processed features are added element-by-element to the original input features;
[0029] The tail point-by-point convolution restores the number of channels to the input dimension.
[0030] Furthermore, the hybrid fully connected feedforward neural network comprises:
[0031] Global feedforward neural network and local feedforward neural network.
[0032] Compared with the existing technology, the beneficial effects of the present invention are:
[0033] This paper presents a corridor crack detection network for visual semantic segmentation. The network architecture follows an encoder-decoder architecture and adopts a modular design concept, resulting in strong robustness, high efficiency, and high versatility. This network can more effectively segment corridor concrete cracks and can be applied to other concrete scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 A schematic diagram of a corridor crack detection network architecture for visual semantic segmentation.
[0035] Figure 2 Schematic diagram of the downsampling module;
[0036] Figure 3 Schematic diagram of the upsampling module;
[0037] Figure 4Schematic diagram of the feature extraction module based on the self-attention mechanism. DETAILED DESCRIPTION
[0038] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.
[0039] The features and performance of the present invention are further described in detail below with reference to the embodiments.
[0040] Example 1
[0041] See also Figure 1 , a corridor crack detection network for the field of visual semantic segmentation, built using an encoder-decoder architecture;
[0042] The encoder consists of 19 layers of feature extraction modules and downsampling modules based on the self-attention mechanism. The encoder can classify and analyze local pixels of the image to obtain high-level semantic information.
[0043] The decoder contains a 7-layer structure consisting of upsampling modules and convolution modules; that is, the decoder has a total of 7 layers, and its main task is to parse the semantic information of the encoder and map the same object back to the corresponding pixel points in the original space. It consists of upsampling modules and convolution modules.
[0044] In this embodiment, it should be noted that each layer in the encoder has the ability to extract pixel features, but as the network depth increases, the model calculation amount will gradually increase, and the hardware configuration required for actual training and prediction will also increase. Moreover, when such a deep network is performing feature extraction, the crack features that originally occupied a relatively low proportion of the entire image will be gradually lost. In order not to lose the model segmentation accuracy and reduce the model complexity, in this example, the network sets three downsampling operations with a step size of 2 and a corresponding feature extraction module based on the self-attention mechanism, which can reduce the spatial dimension by 1 / 8 to ensure computational efficiency and save memory; that is, the encoder reduces the spatial dimension to 1 / 8 of the original input through three downsampling operations with a step size of 2, and the downsampling module is composed of a cascade of a 3×3 convolutional layer and a maximum pooling module with a step size of 2.
[0045] In this embodiment, it should be noted that in order to keep the input and output image sizes consistent, three upsampling operations with a step size of 2 are also set to complete the entire segmentation; that is, the decoder restores the original input resolution through three inverse convolution operations with a step size of 2, and the convolution kernel stride of the inverse convolution is set to 2 and the padding parameter is set to 1.
[0046] In this embodiment, it should be noted that in deep learning, the role of the downsampling module is to reduce the complexity and parameter amount of the model by reducing the size and number of feature maps. Downsampling can help the model better understand and extract the key information of the crack image and crack features, while reducing the feature map to give it better generalization ability. There are currently two downsampling methods. One is to use a pooling layer, such as maximum pooling and average pooling. Pooling downsampling is simple to calculate and can better retain texture features. The other is to use a convolutional layer with a stride of 2 (3×3Conv). The downsampling process is a process of information loss, and the pooling layer is not learnable. Using a learnable convolutional layer with a stride of 2 can achieve better results, but of course it also increases a certain amount of calculation.
[0047] Therefore, see Figure 2 and Figure 3 This embodiment combines the advantages of pooling and convolution, cascading a single 3×3 convolution with a maximum pooling module with a stride of 2 to perform downsampling, avoiding excessive loss of crack features during image size reduction. The upsampling module uses inverted convolution to gradually adjust the model's output during the decoding phase to match the input resolution. Inverted convolution is a parameterized operation that can restore detail and spatial information lost during downsampling to a certain extent by adjusting the convolution kernel's stride, padding, and the size of the output space.
[0048] In this embodiment, specifically, the feature extraction module based on the self-attention mechanism includes a residual structure, sequentially connected point-by-point convolution, a self-attention mechanism with an expansion rate, and a hybrid fully connected feedforward neural network.
[0049] In this embodiment, specifically, it should be noted that the feature extraction module based on the self-attention mechanism is a network module with a residual structure. The residual structure is used in several recent classification and segmentation task networks. Such a structure is helpful for network learning and significantly reduces the degradation problem existing in deep network architectures. For a stacking layer structure (composed of several layers stacked together), when the input is x, the learned feature is recorded as H(x). Now it is hoped that it can learn the residual F(x), so that the original learning feature is F(x)+x. The reason for this is that residual learning is easier than directly learning the original features. When the residual is 0, the stacking layer only performs an identity mapping. At least the network performance will not decrease. In fact, the residual will not be 0, which will also enable the stacking layer to learn new features based on the input features, thereby having better performance.
[0050] In addition to using a residual structure as the basic module framework, a feature extraction module based on the self-attention mechanism is constructed by sequentially connecting point-by-point convolution (1×1 conv), a self-attention mechanism, and a hybrid fully connected feedforward neural network. However, when using convolutional neural network semantic segmentation models to process irregularly shaped objects such as cracks, it is desirable for such models to have a larger receptive field (RF), rather than focusing on a small area. This is because cracks have demanding generation conditions and a low probability of occurrence, and they occupy a small proportion of the entire image, making them less prominent compared to background features. This necessitates collecting more surrounding information to ensure that cracks are correctly separated from the background. Without surrounding information from the original image, features in small areas cannot be clearly separated into cracks. This can be achieved by increasing the size of the convolution kernel or the depth of the convolution layer. However, since the number of channels in channel-by-channel convolution scales with the number of input channels during feature extraction, this cannot be increased. To address this receptive field issue, a dilation rate parameter is introduced into the self-attention mechanism to expand the receptive field, resulting in a dilated self-attention mechanism with a larger receptive field.
[0051] An example of a feature extraction module based on the self-attention mechanism is Figure 4As shown in the figure, the role of the first point-by-point convolution in the module is to map the low-dimensional space to the high-dimensional space (expanding the number of channels before the data enters the convolution). Here, the dimensional expansion multiple is achieved by setting hyperparameters. The expansion multiple can be adjusted according to the actual situation. The default value is 6, which means that the channels are expanded by 6 times. By expanding the number of channels, the representation ability of the model can be increased, enabling it to learn richer and more complex features, and the controllability of the computational complexity can be ensured when the number of channels of the model is expanded. The role of the tail point-by-point convolution is to map the high-dimensional features to the low-dimensional space (reducing the number of channels), ensuring that the input and output sizes of the residual structure match the number of channels, and significantly reducing the computational cost by reducing the number of channels.
[0052] In this example, it's important to note that the self-attention mechanism addresses the difficulty traditional neural networks have in capturing long-range dependencies when processing sequential data. It calculates the correlation between positions in the input data, generates an attention weight for each position, and then performs a weighted summation of the input data based on these weights to produce a new representation.
[0053] Specifically, the calculation process of the self-attention mechanism can be divided into the following steps:
[0054] a: Calculate the query vector, key vector, and value vector separately: For each position in the input data, a query vector, a key vector, and a value vector are obtained through linear transformation.
[0055] b: Calculate attention weights: By calculating the dot product between the query vector and the key vector and then normalizing it, we get the attention weights. This weight represents the correlation between different positions in the input data.
[0056] c: Weighted summation: Perform weighted summation on the value vector according to the attention weight to obtain the final output representation:
[0057]
[0058] in,
[0059] Q represents the query vector;
[0060] K represents the key vector;
[0061] V represents the value vector;
[0062] d head Indicates the length of the vector key vector.
[0063] In this embodiment, specifically, the residual structure is implemented as follows:
[0064] After the input features are expanded in dimension by the first layer of point-by-point convolution, they pass through the self-attention mechanism and the hybrid fully connected feedforward neural network in sequence;
[0065] The processed features are added element-by-element to the original input features;
[0066] The tail point-by-point convolution restores the number of channels to the input dimension.
[0067] In this embodiment, it should be noted that in the traditional visual model, each attention sublayer is connected to a fully connected feedforward neural network (FFN). In the mixed fully connected feedforward neural network, in addition to the traditional fully connected feedforward neural network, another type of FFN, namely Mix-FFN, is introduced. Mix-FFN provides a more flexible feature extraction method by introducing a different feedforward neural network structure. Traditional FFN uses the same nonlinear transformation at each position, while Mix-FFN allows different nonlinear transformations to be used at different positions, thereby increasing the expressive power of the model.
[0068] Specifically, Mix-FFN uses two different feedforward neural network structures: a global feedforward neural network (Global FFN) and a local feedforward neural network (Local FFN). Global FFN is a feedforward neural network with a larger receptive field that can better capture global context information. Local FFN is a feedforward neural network with a smaller receptive field that can better capture local detail information. By using global FFN and local FFN at the same time, Mix-FFN can be more flexible and accurate in processing features at different locations. Global FFN can help the model capture longer-range dependencies and semantic information, while local FFN can better handle local details and subtle changes.
[0069] The above-described embodiments merely represent specific implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of protection of the present application. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the technical concept of the present application, and all such variations and improvements fall within the scope of protection of the present application.
[0070] This background section is provided to generally present the context of the invention, and the work of the presently named inventors, the work to the extent described in this background section, and aspects of the description in this section that did not constitute prior art at the time of filing are neither explicitly nor implicitly admitted to be prior art to the present invention.
Claims
1. A corridor crack detection network for the field of visual semantic segmentation, characterized by: It is constructed using an encoder-decoder architecture; the encoder contains a 19-layer structure consisting of a feature extraction module based on the self-attention mechanism and a downsampling module, and the decoder contains a 7-layer structure consisting of an upsampling module and a convolution module.
2. The corridor crack detection network for visual semantic segmentation according to claim 1 is characterized in that: The encoder reduces the spatial dimension to 1 / 8 of the original input through three downsampling operations with a stride of 2. The downsampling module is composed of a 3×3 convolutional layer and a maximum pooling module with a stride of 2 in cascade.
3. The corridor crack detection network for visual semantic segmentation according to claim 2 is characterized in that: The decoder restores the original input resolution through three inverse convolution operations with a stride of 2. The convolution kernel stride of the inverse convolution is set to 2 and the padding parameter is set to 1.
4. The corridor crack detection network for visual semantic segmentation according to claim 3 is characterized in that: The feature extraction module based on the self-attention mechanism includes a residual structure, point-by-point convolution, a self-attention mechanism with an expansion rate, and a hybrid fully connected feedforward neural network.
5. The corridor crack detection network for visual semantic segmentation according to claim 4 is characterized in that: The calculation process of the self-attention mechanism includes: a: Calculate the query vector, key vector and value vector respectively; b: Calculate attention weight; c: Weighted summation to obtain the final output representation.
6. The corridor crack detection network for visual semantic segmentation according to claim 5 is characterized in that: For each position in the input data, a query vector, a key vector, and a value vector are obtained through linear transformation.
7. The corridor crack detection network for visual semantic segmentation according to claim 5 is characterized in that: The attention weight is obtained by calculating the dot product between the query vector and the key vector and then normalizing it.
8. The corridor crack detection network for visual semantic segmentation according to claim 5 is characterized in that: The output is represented as follows: in, Q represents the query vector; K represents the key vector; V represents the value vector; d head Indicates the length of the vector key vector.
9. The corridor crack detection network for visual semantic segmentation according to claim 4 is characterized in that: The residual structure is specifically implemented as follows: After the input features are expanded in dimension by the first layer of point-by-point convolution, they pass through the self-attention mechanism and the hybrid fully connected feedforward neural network in sequence; The processed features are added element-by-element to the original input features; The tail point-by-point convolution restores the number of channels to the input dimension.
10. The corridor crack detection network for visual semantic segmentation according to claim 4, characterized in that: The hybrid fully connected feedforward neural network comprises: Global feedforward neural network and local feedforward neural network.