Image recognition method, device and system based on convolutional neural network

By introducing gradient layers and self-attention mechanisms into the VGG-16 fully convolutional neural network, the network architecture is optimized, and the problems of low efficiency and low accuracy in crack detection of bridge concrete roads are solved, achieving efficient, fast and accurate crack recognition.

CN115272826BActive Publication Date: 2025-08-22DONGGUAN UNIV OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210925219.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-03
Publication Date
2025-08-22
Estimated Expiration
2042-08-03

AI Technical Summary

Technical Problem

The prior art has problems in the detection of cracks on bridge concrete roads, which are low detection efficiency, low accuracy, rely on manual detection and high image quality requirements, especially the identification of fine cracks is poor, and traditional methods are susceptible to noise interference.

Method used

A fully convolutional neural network based on the VGG-16 model is adopted, combining the gradient layer and self-attention mechanism, and the global information learning ability of the network is enhanced through the self-attention mechanism module, and a gradient layer is introduced during the training process to accelerate convergence, and the network architecture is optimized to adapt to the self-attention mechanism and gradient layer module.

Benefits of technology

The identification speed and accuracy of bridge concrete road cracks are improved, efficient, fast and accurate crack detection is achieved, the calculation amount is reduced and the identification ability of fine cracks is enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115272826B_ABST
    Figure CN115272826B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, device and system for identifying cracks in bridge concrete road images based on a convolutional neural network. By introducing a gradient layer and a self-attention mechanism module on the basis of the VGG-16 fully convolutional neural network model, the model has the effect of accelerating convergence when training on a data set. While ensuring the model's lightweight, it can also improve the network model's ability to learn the global information of the input feature map, thereby improving its crack recognition speed and accuracy in concrete road images. The concrete road crack recognition model ultimately trained by the present invention can perform image crack detection efficiently, quickly and with high precision.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to technical fields such as computer vision and image processing and analysis, and in particular to a method, device and system for recognizing crack images of bridge concrete roads based on a convolutional neural network. Background Art

[0002] With the development of the transportation industry, the number of transportation-related bridges and highways has increased significantly, creating a significant demand for bridge and highway maintenance. Bridges, as a crucial component of today's transportation hubs, are essential transportation infrastructure for convenient travel and logistics delivery. However, over long-term use, the concrete structures of bridges are inevitably subject to various types of damage. Due to the bridge's downward-facing load-bearing structure, most bridge defects first manifest as cracks or honeycombing on the bridge's surface and back. Therefore, regular inspections of crack growth on both the surface and back of bridges are an essential and primary task in bridge and highway maintenance. In actual inspections, due to the random distribution and generation of cracks, many small cracks are easily disturbed by noise such as rutting and water stains, leading to missed or false detections and posing significant safety risks. To prevent these issues, research on small cracks is crucial.

[0003] Traditional crack detection primarily involves inspecting cracks on both the surface and back of bridges. Surface cracks can be detected by inspectors through manual visual inspection. For the back of a bridge, inspectors are transported to the base of the bridge by bridge inspection vehicles. They then visually locate the approximate area where cracks are present and use specialized crack detection tools (such as crack width detectors and crack depth gauges) to obtain quantitative crack data. With the increasing number of bridges, manual inspections are not only time-consuming and labor-intensive, but also require inspectors to carefully identify numerous fine cracks, resulting in low detection efficiency. The effectiveness of inspections is heavily dependent on the staff's work status and professional expertise.

[0004] Traditional machine vision algorithms are used to identify bridge cracks, but they have high requirements for image quality and can often only extract a certain type of crack features, such as texture and gradient, and the algorithm has poor generalization ability. The unsupervised neural network model method can realize the classification of crack pixels according to different additional conditions, but this method has high requirements for image quality and is easily affected by noise with similar color, texture, and position due to the limitations of its additional conditions, resulting in low accuracy in crack identification. In the supervised method, the neural network model needs to extract image features of different scales layer by layer. Shallower networks are not comprehensive enough in extracting image features and cannot completely segment cracks in the image. They are also prone to segmenting interference noise with similar color, texture, and position as cracks. Deeper networks extract image features but also bring about problems such as difficulty in network convergence and doubling of computational complexity, resulting in a long time and low efficiency in crack segmentation by the network model. Summary of the Invention

[0005] The purpose of this application is to provide an image recognition method, device and system based on convolutional neural network.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A method for recognizing crack images of bridge concrete roads based on a convolutional neural network comprises the following steps:

[0008] Step S100: Using the VGG-16 model to build a cascaded fully convolutional neural network;

[0009] Step S200: Add a gradient layer to the model input end, and use the four-channel matrix composed of the gradient layer and the original image as a new input end;

[0010] Step S300: Add a self-attention mechanism module to the output of the last convolutional layer of the last three encoder modules to introduce an attention mechanism into the network model;

[0011] Step S400: Place a deconvolution layer module at the output of each processing module and self-attention mechanism module, and segment the final image through the convolution layer;

[0012] Step S500: training the fully convolutional neural network model using a calibration data set, wherein the calibration data set is a set of feature maps of calibrated concrete crack features;

[0013] In step S600 , the bridge road image to be detected is processed frame by frame, and the processed image is input into the neural network model for crack segmentation to segment the cracks in the bridge road image to be detected.

[0014] In step S500, the fully convolutional neural network model is trained using a calibration dataset, including the following steps:

[0015] 1) Get the output of the last convolutional layer in the processing module through The query, key, and value matrices output by the convolution kernel; where W, H, C, B, Θ, Φ, and g are the width, height, number of channels, number of batches, query matrix, key matrix, and value matrix of the input feature map, respectively;

[0016] 2) Perform a dot multiplication on the Query matrix and the Key matrix to obtain a size of Similarity weight matrix;

[0017] 3) Perform Softmax operation on the similarity weight matrix to achieve normalization of the similarity weight matrix;

[0018] 4) Perform dot multiplication on the normalized similarity weight matrix and the Value matrix to obtain a value of Attention value; after After the number of channels of the convolution kernel is expanded to C, the Attention value is superimposed on the output matrix of the last convolution layer in the processing module.

[0019] The fully convolutional neural network includes an input module, an encoder module and a decoder module.

[0020] The input module consists of a gradient layer and an input image; the gradient layer first adopts the horizontal and vertical coordinate directions of the input image The Sobel first-order difference operator is used to process the gradient matrix in the horizontal and vertical coordinate directions, and then the gradient layer is obtained based on the sum of the absolute values ​​of the gradient in the horizontal and vertical coordinate directions of each point coordinate as the gradient amplitude.

[0021] The encoder module has a total of five blocks. The first encoder performs two convolution operations on the input feature map. The input consists of the feature map and gradient layer in the calibration data set. The second encoder consists of a pooling layer and two convolution layers. Through the maximum pooling operation, the length and width dimensions of the input feature map are halved, thereby extracting higher-order semantic information and reducing the amount of calculation. The next three encoders are composed of a pooling layer and three convolution layers.

[0022] The decoder module corresponds one-to-one to the encoder module and the self-attention mechanism module, and its input feature maps come from the output of the previous processing module respectively; the decoder module performs a transposed convolution operation on the output of the previous processing module to double the length and width dimensions of the feature map, and then superimposes the transposed feature map with the transposed output of the corresponding self-attention mechanism module, and then performs another convolution operation.

[0023] A road crack image recognition device based on a convolutional neural network, comprising:

[0024] A model building module uses a cascaded fully convolutional neural network based on the VGG-16 model. The fully convolutional neural network architecture consists of multiple processing modules, each of which includes a cascade of five encoder modules and five decoder modules.

[0025] The gradient layer and self-attention mechanism establishment module is used to add a gradient layer to the model input, use the four-channel matrix composed of the gradient layer and the original image as the new input, and add the self-attention mechanism to the output of the last convolutional layer of each processing module to obtain a fully convolutional neural network model with a self-attention mechanism.

[0026] A neural network model training module is used to train the fully convolutional neural network model using a calibration data set, wherein the calibration data set is a set of feature maps of calibrated concrete road crack characteristics;

[0027] The concrete crack detection module is used to read the road image to be detected frame by frame, and input it into the concrete road crack recognition model for crack recognition, and segment the concrete road cracks in the image to be detected.

[0028] A road crack image recognition system based on a convolutional neural network, the image recognition system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of any one of the methods of claims 1 to 6.

[0029] The beneficial effects of the present invention are

[0030] The present invention discloses a method, device and system for recognizing crack images of bridge concrete roads based on a convolutional neural network. By introducing a gradient layer and a self-attention mechanism module on the basis of a VGG-16 fully convolutional neural network model, the model plays a certain acceleration role when training convergence using a calibration data set. While ensuring lightweight, the model can also improve the network model's ability to learn the global information of the input feature map, thereby improving its recognition speed and accuracy for concrete road cracks. The concrete road crack recognition model finally trained by the present invention can perform image crack detection efficiently, quickly and with high precision. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of a method for segmenting bridge concrete road crack images based on a VGG-16 fully convolutional neural network according to an embodiment of the present invention;

[0032] Figure 2 Schematic diagram of a VGG-16 fully convolutional neural network model in an embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of the structure of the self-attention mechanism in an embodiment of the present invention;

[0034] Figure 4 Schematic diagram of the network structure of the VGG-16 fully convolutional neural network model in an embodiment of the present invention;

[0035] Figure 5 1 is an example diagram of recognition results of an original image, a label image, a VGG-16-based fully convolutional neural network, and an image detected by a fully convolutional neural network model with a self-attention mechanism module added in the present invention in an embodiment of the present invention;

[0036] Figure 6 This is a comparison chart of the convergence speed of the loss function during the training process of a network with an added gradient layer module or a residual module and the original network in an embodiment of the present invention.

[0037] The core idea of ​​the residual module is to enable the input signal of the lower layer to be directly transmitted to the higher layer through skip-layer connection, making the forward and backward transmission of information smoother.

[0038] Figure 7 This is a structural block diagram of a concrete road crack image recognition device based on a VGG-16 fully convolutional neural network in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The implementation of the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0040] The purpose of this embodiment is: (1) to address the problems of incomplete and discontinuous crack detection or false detection of pseudo-cracks in the existing crack segmentation algorithm based on the fully convolutional neural network, the present invention introduces a self-attention mechanism module on the basis of the DeepCrack semantic segmentation model based on the fully convolutional neural network model, enhances the network's ability to learn global information and distinguish crack and pseudo-crack features, improves the accuracy of the crack detection network, and extracts crack edges more completely and accurately.

[0041] (2) For the crack segmentation algorithm based on the fully convolutional neural network, the present invention introduces a gradient layer module to enable the network to obtain important gradient information of the image at the beginning of training, thereby improving the convergence speed of training; and optimizes and adjusts the network architecture and decoding architecture of the feature extraction network to adapt to the added self-attention mechanism module and gradient layer module.

[0042] refer to Figure 1 ,like Figure 1The present invention provides a method for identifying cracks in bridge and road images based on a fully convolutional neural network, which includes the following steps:

[0043] Step S100: Using the VGG-16 model to build a cascaded fully convolutional neural network;

[0044] Among them, combined Figure 2 , Figure 2 The structure diagram of the fully convolutional neural network in this paper is as follows. The architecture of the VGG-16 model consists of multiple processing modules, including an initial convolution module, four encoder modules, and four decoder modules cascaded in sequence.

[0045] Step S200: Add a gradient layer to the model input end, and use the four-channel matrix composed of the gradient layer and the original image as a new input end;

[0046] Step S300: Add a self-attention mechanism module to the output of the last convolutional layer of the last three encoder modules to introduce an attention mechanism into the network model;

[0047] In this embodiment, the purpose of adding the self-attention mechanism module to the output end of the last convolutional layer of the last three encoder modules is that the features extracted by the subsequent encoder modules belong to the deep semantic information of the image. That is, after the subsequent encoders add the self-attention mechanism module, the network will learn the semantic features of the edges, colors, contours, etc. of the cracks more comprehensively, thereby improving the accuracy of crack segmentation.

[0048] Step S400: Place a deconvolution layer module at the output of each processing module and self-attention mechanism module, and segment the final image through the convolution layer;

[0049] In this embodiment, the purpose of placing the deconvolution layer module after each processing module and the self-attention mechanism module is that the deep features learned by the self-attention mechanism module may contain certain noise semantic information, which will interfere with the image segmentation results. Adding the deconvolution layer module to each processing module allows the neural network to retain the crack features learned by the original network, and then optionally superimpose the deep features learned by the self-attention mechanism module to improve the robustness of the network.

[0050] Step S500: training the fully convolutional neural network model using a calibration data set, wherein the calibration data set is a set of feature maps of calibrated concrete crack features;

[0051] In the embodiment of the present invention, a public dataset and a manually collected and calibrated dataset are used as calibration datasets. Optionally, the public dataset is a DeepCrack concrete crack dataset that can be obtained from a public channel.

[0052] Processing the bridge and road image to be detected in step S600 frame by frame, inputting the processed image into a neural network model for crack segmentation, and segmenting cracks in the bridge and road image to be detected;

[0053] In this technical field, neural networks can be used to effectively perform visual tasks (such as image recognition, image classification, target detection, etc.). Its essence is that the neural network model is a highly complex mathematical expression with a large number of parameters. The more parameters there are, the stronger the model's fitting ability is. The model is trained with a large amount of data, and the parameter weights are optimized through the gradient descent algorithm to gradually find the optimal solution. However, a large number of parameters lead to a sharp increase in the amount of model calculation, making the neural network model extremely dependent on the computing power of the device. The embodiment provided by the present invention introduces a self-attention mechanism based on the VGG-16 model, thereby enhancing the global information learning ability of the network processing module. Since the operation of the self-attention mechanism module involves length Therefore, if the self-attention mechanism module is added to the previous encoder module, it will bring a huge computational burden to the network. By adding the self-attention mechanism module to the output end of the last convolutional layer of the last three encoder modules, the model is made more lightweight while ensuring the recognition accuracy. This not only ensures the network's recognition speed of bridge and road cracks, but also further improves the recognition accuracy of bridge and road cracks through the global information learning ability of the self-attention mechanism module. The bridge and road crack recognition model finally trained by the present invention can perform crack segmentation image recognition efficiently, quickly and accurately.

[0054] refer to Figure 3 , Figure 3 Schematic diagram of the structure of the self-attention mechanism. As an improvement to the above embodiment, in step S500, the fully convolutional neural network model is trained using a calibration dataset, including the following steps:

[0055] 1) Get the output of the last convolutional layer in the processing module through The query, key, and value matrices output by the convolution kernel; where W, H, C, B, Θ, Φ, and g are the width, height, number of channels, number of batches, query matrix, key matrix, and value matrix of the input feature map, respectively;

[0056] 2) Perform a dot multiplication on the Query matrix and the Key matrix to obtain a size of Similarity weight matrix; Among them, the number of channels of the Query matrix and Key matrix obtained in step 1 can be adjusted according to the computing power of the server. The number of channels selected in this paper is C / 2;

[0057] 3) Perform Softmax operation on the similarity weight matrix to achieve normalization of the similarity weight matrix.

[0058] 4) Perform dot multiplication on the normalized similarity weight matrix and the Value matrix to obtain a value of Attention value; after After the convolution kernel is expanded to C, the Attention value is superimposed on the output matrix of the last convolution layer in the processing module;

[0059] pass When the convolution kernel obtains the Query, Key, and Value matrices, the output channels of the above three matrices are compressed. The compression multiple is set manually. Selecting a larger compression multiple can reduce the parameters required for network calculations, but the characteristic information of some channels will be lost. In this invention, the number of channels of the Query, Key, and Value matrices is set to C / 2.

[0060] In this embodiment, the correlation between each pixel of the input feature map and the global feature pixel is calculated through the self-attention mechanism, and is automatically generated through network model training without relying on manual pre-setting. The convolution parameters in the encoder are not changed before and after the feature map is input, and the semantic features of the original encoder are retained. The weight of each pixel of the input encoder feature map is adaptively adjusted through the self-attention mechanism.

[0061] refer to Figure 2 ,The fully convolutional neural network based on VGG-16 includes an input module, an encoder module, and a decoder module, as shown in the figure;

[0062] The input module consists of a gradient layer and an input image. The gradient layer first uses the horizontal and vertical coordinates of the input image to The Sobel first-order difference operator is used to process the gradient matrix in the horizontal and vertical coordinate directions, and then the gradient layer is obtained based on the sum of the absolute values ​​of the gradient in the horizontal and vertical coordinate directions of each point coordinate as the gradient amplitude.

[0063] The encoder module in the present invention has a total of five blocks. The first encoder performs two convolution operations on the input feature map. The input consists of the feature map and gradient layer in the calibration data set; the second encoder consists of a pooling layer and two convolution layers. Through the maximum pooling operation, the length and width dimensions of the input feature map are halved, thereby extracting higher-order semantic information and reducing the amount of calculation; the next three encoders consist of a pooling layer and three convolution layers. Due to the previous pooling operation, the length and width dimensions of the input feature map are reduced, thereby reducing the computational burden required for the subsequent use of the three convolution layers.

[0064] The decoder module corresponds one-to-one with the encoder module and the self-attention mechanism module. Its input feature maps come from the output of the previous processing module. The decoder module performs a transposed convolution operation on the output of the previous processing module, doubling the length and width dimensions of the feature map. It then performs a channel-wise superposition of the transposed feature map with the transposed output of the corresponding self-attention mechanism module, and then performs another convolution operation.

[0065] refer to Figure 4 The figure lists the convolution operation of each processing module and the number of feature map channels output by the self-attention mechanism module. Among them, the convolution kernel size of the convolution operation is The stride is 1, there is no zero padding on the edges, and the dilation coefficient is 1, meaning no dilated convolution is used. For the two convolution operations in the first encoder block, the number of channels in the output feature map is 64. Each convolution operation is followed by a ReLU nonlinear activation function to make the network more expressive.

[0066] refer to Figure 5 ,from Figure 5 It can be seen that due to the addition of the self-attention mechanism to the VGG-16-based fully convolutional neural network, the concrete road crack recognition model has a higher recall rate and accuracy, can more completely identify cracks with subtle depth changes and small lengths, and has a strong generalization ability.

[0067] refer to Figure 6 ,from Figure 6 As can be seen in the figure, the fully convolutional neural network DeepCrack, which adds gradient layers and residual modules, has a faster convergence speed. In the first few iterations, the network loss drops to between 0.1 and 0.2. Therefore, the gradient layer module can accelerate network training convergence and improve the training of deep neural networks by reducing gradient vanishing and gradient exploding.

[0068] refer to Figure 7 The embodiment of the present invention further provides a road crack image recognition device based on a convolutional neural network, comprising:

[0069] The model building module 100 uses a cascaded fully convolutional neural network based on the VGG-16 model. The fully convolutional neural network architecture is composed of multiple processing modules, each of which includes a cascade of five encoder modules and five decoder modules.

[0070] A gradient layer and self-attention mechanism establishment module 200 is used to add a gradient layer to the model input, use the four-channel matrix composed of the gradient layer and the original image as the new input, and add a self-attention mechanism to the output of the last convolutional layer of each processing module to obtain a fully convolutional neural network model with a self-attention mechanism;

[0071] A neural network model training module 300 is used to train the fully convolutional neural network model using a calibration dataset, wherein the calibration dataset is a set of feature maps of calibrated concrete road crack characteristics;

[0072] The concrete crack detection module 400 is used to read the road image to be detected frame by frame, input it into the concrete road crack recognition model for crack recognition, and segment the concrete road cracks in the image to be detected.

[0073] An embodiment of the present invention also provides a road crack image recognition system based on a convolutional neural network, the system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of any of the above methods.

[0074] In the description of the embodiments of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to a fixed connection, an indirect connection via an intermediate medium, internal communication between two components, or an interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the embodiments of this application based on specific circumstances.

[0075] In the embodiments of the present application, any device or element referred to or implied must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be understood as limiting the embodiments of the present application. In the description of the embodiments of the present application, the meaning of "plurality" is two or more, unless otherwise specifically specified.

[0076] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the embodiments of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "may include" and "have" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0077] Finally, it should be noted that the above embodiments are intended only to illustrate the technical solutions of the embodiments of this application, and are not intended to limit them. Although the embodiments of this application have been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, such modifications or replacements do not deviate from the essence of the corresponding technical solutions within the scope of the various embodiments of this application.

Claims

1. A method for recognizing crack images of bridge concrete roads based on convolutional neural networks, characterized by: The following steps are involved: Step S100: Using the VGG-16 model to build a cascaded fully convolutional neural network, the fully convolutional neural network includes an input module, five encoder modules, and a decoder module; Step S200: The input module is composed of a gradient layer and an input image. The gradient layer firstly uses 3 3 is processed by the Sobel first-order difference operator to obtain the gradient matrix in the horizontal and vertical coordinate directions, and then the gradient layer is obtained based on the sum of the absolute values ​​of the gradient in the horizontal and vertical coordinate directions of each point coordinate as the gradient amplitude; Step S300: Add a self-attention mechanism module to the output of the last convolutional layer of the last three encoder modules to introduce an attention mechanism into the network model; Step S400: Place a deconvolution layer module at the output of each processing module and self-attention mechanism module, and segment the final image through the convolution layer; Step S500: training the fully convolutional neural network model using a calibration data set, wherein the calibration data set is a set of feature maps for calibrating concrete crack features; processing the bridge and road image to be detected in step S600 frame by frame, inputting the processed image into the neural network model for crack segmentation, and segmenting cracks in the bridge and road image to be detected; In step S500, the fully convolutional neural network model is trained using a calibration dataset, including the following steps: 1) Get the output of the last convolutional layer in the processing module through 1 1. Query, Key, and Value matrices output by the convolution kernel; where W, H, C, B, Θ, Φ, and g are the width, height, number of channels, number of batches, Query matrix, Key matrix, and Value matrix of the input feature map, respectively; 2) Perform a dot multiplication on the Query matrix and the Key matrix to obtain a BHW size BHW similarity weight matrix; 3) Perform Softmax operation on the similarity weight matrix to achieve normalization of the similarity weight matrix; 4) Perform a dot multiplication operation on the normalized similarity weight matrix and the Value matrix to obtain a BHW matrix. Attention value of C / 2; after 1 After the convolution kernel of 1 is expanded to C, the Attention value is superimposed on the output matrix of the last convolution layer in the processing module.

2. The method for recognizing cracks in bridge concrete roads based on a convolutional neural network according to claim 1, characterized in that: The first encoder performs two convolution operations on the input feature map, which consists of the feature map and gradient layer in the calibration dataset. The second encoder consists of a pooling layer and two convolution layers. Through the maximum pooling operation, the length and width dimensions of the input feature map are halved, thereby extracting higher-order semantic information and reducing the amount of computation. The next three encoders consist of a pooling layer and three convolution layers.

3. The method for recognizing cracks in bridge concrete roads based on a convolutional neural network according to claim 1, characterized in that: The decoder module corresponds one-to-one with the encoder module and the self-attention mechanism module, and its input feature map comes from the output of the previous processing module; The decoder module performs a transposed convolution operation on the output of the previous processing module, doubling the length and width dimensions of the feature map. It then superimposes the transposed feature map with the transposed output of the corresponding self-attention mechanism module and performs another convolution operation.

4. A bridge concrete road crack image recognition device based on convolutional neural network, characterized by: Adopting the method as described in any one of claims 1 to 3 above; It includes: A model building module uses a cascaded fully convolutional neural network based on the VGG-16 model. The fully convolutional neural network architecture consists of multiple processing modules, each of which includes a cascade of five encoder modules and five decoder modules. The gradient layer and self-attention mechanism establishment module is used to add a gradient layer to the model input, use the four-channel matrix composed of the gradient layer and the original image as the new input, and add the self-attention mechanism to the output of the last convolutional layer of each processing module to obtain a fully convolutional neural network model with a self-attention mechanism. A neural network model training module is used to train the fully convolutional neural network model using a calibration data set, wherein the calibration data set is a set of feature maps of calibrated concrete road crack characteristics; The concrete crack detection module is used to read the road image to be detected frame by frame, and input it into the concrete road crack recognition model for crack recognition, and segment the concrete road cracks in the image to be detected.

5. A bridge concrete road crack image recognition system based on convolutional neural network, characterized by: The image recognition system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of any one of the methods of claims 1 to 3 when executed by the processor.

Citation Information

Patent Citations

  • Nuclear power pipeline defect detection system based on deep learning attention mechanism

    CN111899224A

  • Pavement crack detection method and related equipment

    CN114418937A