Bridge crack detection method and system based on convolutional neural network

Abstract

The invention relates to the technical field of bridge detection, and discloses a bridge crack detection method and system based on a convolutional neural network, and the method comprises the steps: obtaining a pavement grayscale image of an ith bridge road section, and carrying out the copy copying of the pavement grayscale image, and obtaining a first grayscale copy image and a second grayscale copy image; dividing a first foreground region from the first gray scale copy image by using a convolutional neural network, and dividing a second foreground region from the second gray scale copy image; image layer overlapping is carried out on the first foreground area and the second foreground area, an overlapping area formed after image layer overlapping is marked as a crack area, area data of the crack area are obtained, and i is made to be equal to i + 1; repeating the steps until i is equal to M, ending circulation, obtaining area data of the N crack areas, and evaluating the structural health degree of the to-be-detected bridge according to the area data of all the crack areas to obtain an evaluation result; according to the invention, the segmentation precision of bridge crack detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge detection, and more specifically, to a bridge crack detection method and system based on convolutional neural network. Background Art

[0002] Bridges are an indispensable component of transportation infrastructure, and their structural safety directly affects public travel safety and traffic efficiency. With increasing service life, bridges are prone to fatigue and aging, which can lead to cracks. Cracks are one of the early signs of bridge structural damage. If not detected and repaired in a timely manner, they may cause more serious structural problems and even lead to serious safety accidents such as bridge collapse. Therefore, early detection and monitoring of bridge cracks are of great significance to ensuring bridge safety.

[0003] Currently, the detection of bridge cracks mainly relies on manual inspections and video monitoring. However, manual inspections are inefficient, time-consuming, and easily restricted by the environment and manpower, often leading to missed and false detections. While video monitoring improves detection efficiency, the accuracy of crack segmentation has always been a challenge due to factors such as image quality, lighting conditions, and shooting angles. Traditional methods are often unable to accurately segment crack areas, especially in complex environments, and cannot meet high-precision detection requirements. In addition, existing systems have limited capabilities in determining the nature of cracks and intelligently analyzing their development trends, making it difficult to provide a comprehensive and accurate assessment. Therefore, the development of a high-precision, intelligent bridge crack detection method and system is of great practical significance and application value. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a bridge crack detection method and system based on convolutional neural network.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A bridge crack detection method based on a convolutional neural network, the method comprising:

[0007] S101: Obtain a road surface grayscale image of an i-th bridge section in a road section set, and duplicate the road surface grayscale image to obtain a first grayscale duplicate image and a second grayscale duplicate image of the i-th bridge section; the road section set includes M bridge sections of the bridge to be detected, where i and M are both integers greater than zero;

[0008] S102: using a preconfigured first convolutional neural network to separate a first foreground area from the first grayscale copy image, and using a preconfigured second convolutional neural network to separate a second foreground area from the second grayscale copy image;

[0009] S103: Overlaying the first foreground area and the second foreground area, marking the overlapping area formed by the overlaying as a crack area, obtaining region data of the crack area, setting i=i+1, and returning to step S101; the region data includes the region area and the number of regions;

[0010] S104: Repeat the above steps S101 to S103 until the loop ends when i=M, obtain regional data of N crack areas, and evaluate the structural health of the bridge to be inspected based on the regional data of all crack areas to obtain an evaluation result, where N is an integer greater than zero.

[0011] Furthermore, before obtaining the road surface grayscale image of the i-th bridge section in the road section set, the following steps are included:

[0012] Obtain a road surface color image of the i-th bridge section in the road section set, perform grayscale processing on the road surface color image, and obtain an original grayscale image;

[0013] Obtaining image parameter data of the original grayscale image and inputting the image parameter data into a pre-built quality coefficient calculation model to obtain the quality coefficient of the original grayscale image; the image parameter data includes image size, grayscale value of each pixel, average grayscale value, grayscale standard deviation, and gradient in the horizontal x direction and vertical y direction;

[0014] The mathematical expression of the pre-built quality coefficient calculation model is as follows:

[0015]

[0016] Where: Q(I) is the quality coefficient; H and W are the height and width of the image, which are used to represent the image size; σ is the grayscale standard deviation; μ is the average grayscale value; I(x,y) is the grayscale value at the position (x,y) in the image, Represents the gradient of the image in the horizontal x direction; Represents the gradient of the image in the vertical y direction;

[0017] Comparing the quality coefficient with a preset quality coefficient range;

[0018] If the quality coefficient is within the preset quality coefficient range, the original grayscale image is used as the road surface grayscale image of the i-th bridge section;

[0019] If the quality coefficient is outside the preset quality coefficient range, the original grayscale image is enhanced using a preset nonlinear transformation model to obtain an enhanced original grayscale image;

[0020] The enhanced original grayscale image is marked as the road surface grayscale image of the i-th bridge section.

[0021] Furthermore, dividing the first foreground area from the first grayscale copy image includes:

[0022] Extract the first grayscale copy image and convert it into a two-dimensional matrix Where: A is a two-dimensional matrix with m rows and n columns, a mn Represents the grayscale value of the pixel in the mth row and nth column of the matrix;

[0023] The two-dimensional matrix is input into a preconfigured first convolutional neural network to obtain a prediction mask. The prediction mask shows a first foreground area where the bridge crack is located, indicating whether each pixel point belongs to the crack area.

[0024] Furthermore, the training logic of the preconfigured first convolutional neural network is as follows:

[0025] Acquire historical image segmentation training data, and divide the historical image segmentation training data into an image segmentation training set and an image segmentation test set; the historical image segmentation training data includes a first grayscale copy image and its corresponding prediction mask;

[0026] Construct a convolutional neural network, use the first grayscale copy image in the image segmentation training set as the input of the convolutional neural network, and the predicted mask in the image segmentation training set as the output of the convolutional neural network, to obtain an initial convolutional neural network;

[0027] The initial convolutional neural network is model verified using the image segmentation test set, and the initial convolutional neural network that minimizes the preset loss function is obtained as the preconfigured first convolutional neural network.

[0028] Furthermore, the expression of the preset loss function is as follows:

[0029]

[0030] Where: L is the loss function, R is the total number of pixels in the image, Z r is the true label of the r-th pixel, and its value is 0 or 1, 0 means that the pixel does not belong to the crack area, and 1 means it belongs to the crack area; is the predicted value of the r-th pixel, indicating the probability that the r-th pixel belongs to the crack area, with a value between 0 and 1; A is the predicted crack area, the binary mask generated by the model, representing the set of all pixels that the network believes belong to the crack area; B is the actual crack area, the set of all pixels marked as crack areas in the true label.

[0031] Furthermore, the first convolutional neural network includes 6 convolutional layers, 3 pooling layers, 2 hybrid attention modules, 1 transformer module, 1 global average pooling layer and 1 fully connected layer;

[0032] Among them, the convolution kernel size of the first convolution layer is 7×7, the stride is 2, and the number of convolution kernel channels is 64;

[0033] The convolution kernel size of the second convolutional layer is 5×5, the stride is 2, and the number of convolution kernel channels is 128;

[0034] The convolution kernel size of the third convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 256;

[0035] The convolution kernel size of the fourth convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 256;

[0036] The convolution kernel size of the fifth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 512;

[0037] The convolution kernel size of the sixth convolution layer is 1×1, the stride is 1, and the number of convolution kernel channels is 1024.

[0038] The first pooling layer is a maximum pooling layer with a pooling kernel size of 3×3 and a stride of 2.

[0039] The second pooling layer is a maximum pooling layer with a pooling kernel size of 2×2 and a stride of 2;

[0040] The third pooling layer is a maximum pooling layer with a pooling kernel size of 2×2 and a step size of 2.

[0041] Among them, the hybrid attention module includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism calculates the channel weight through global average pooling and global maximum pooling, and the spatial attention mechanism calculates the spatial weight through a 7×7 convolution kernel;

[0042] The transformer module enhances global feature perception through a multi-head self-attention mechanism;

[0043] The global average pooling layer is used to perform global pooling on the feature map and compress the convolution output features to a fixed size;

[0044] The fully connected layer is used to output the prediction mask for crack detection, and the activation function is Sigmoid.

[0045] Furthermore, the second convolutional neural network includes: 5 convolutional layers, 2 pooling layers, 2 sparse convolution modules, 1 adaptive feature enhancement module, 1 multi-branch structure module, 1 global average pooling layer and 1 fully connected layer;

[0046] Among them, the convolution kernel size of the first convolution layer is 3×3, the stride is 1, and the number of convolution kernel channels is 64;

[0047] The convolution kernel size of the second convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 128;

[0048] The convolution kernel size of the third convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 256;

[0049] The convolution kernel size of the fourth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 512;

[0050] The convolution kernel size of the fifth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 1024;

[0051] Wherein, the two pooling layers are the first pooling layer and the second pooling layer respectively;

[0052] The first pooling layer is a maximum pooling layer with a pooling kernel size of 2×2 and a stride of 2;

[0053] The second pooling layer is an average pooling layer with a pooling kernel size of 2×2 and a stride of 2;

[0054] The sparse convolution module uses a sparse convolution kernel to perform convolution operations. The convolution kernel size is 3×3, which reduces the amount of calculation and retains key features.

[0055] The adaptive feature enhancement module dynamically fuses multi-scale features through convolution kernels of different scales;

[0056] The multi-branch structure module extracts crack features of different scales by using multiple convolution kernels in parallel;

[0057] The global average pooling layer performs global pooling on the convolution output to reduce the dimension of the feature map;

[0058] The fully connected layer is used to output the prediction mask of the crack area, and the activation function is Sigmoid.

[0059] Furthermore, the assessment of the structural health of the bridge to be inspected includes:

[0060] According to the regional data of all crack areas, the cumulative area of all crack areas is summed up to obtain the cumulative area of the cracks in the bridge to be detected.

[0061] Retrieving the total surface area of the bridge to be inspected, and calculating the ratio of the cumulative area of the cracks to the total surface area of the bridge to be inspected to obtain an area ratio;

[0062] Setting an area ratio threshold and comparing the area ratio with the area ratio threshold; the area ratio threshold includes an area ratio threshold Td1 and an area ratio threshold Td2, Td1>Td2>0;

[0063] If the area ratio is greater than the area ratio threshold Td1, the bridge to be inspected is determined to be at level three health level, and the level three health level is used as the evaluation result;

[0064] If the area ratio is less than or equal to the area ratio threshold Td1 and greater than or equal to the area ratio threshold Td2, the bridge to be inspected is determined to be of level 2 health, and the level 2 health is used as the evaluation result;

[0065] If the area ratio is less than the area ratio threshold Td2, the bridge to be inspected is determined to be of level one health, and the level one health level is used as the evaluation result.

[0066] A bridge crack detection system based on convolutional neural network, comprising:

[0067] an image copy module, configured to obtain a road surface grayscale image of the i-th bridge section in a road section set, and copy the road surface grayscale image to obtain a first grayscale copy image and a second grayscale copy image of the i-th bridge section; the road section set includes M bridge sections of the bridge to be detected, where i and M are both integers greater than zero;

[0068] a region segmentation module, configured to segment a first foreground region from the first grayscale copy image using a preconfigured first convolutional neural network, and to segment a second foreground region from the second grayscale copy image using a preconfigured second convolutional neural network;

[0069] a crack detection module configured to perform layer superposition on the first foreground area and the second foreground area, mark the overlapping area formed after the layer superposition as a crack area, obtain region data of the crack area, and set i=i+1 to trigger the image replication module; the region data includes the region area and the number of regions;

[0070] The bridge assessment module is used to repeat the above-mentioned image replication module to the crack detection module until the loop ends when i=M, obtain the regional data of N crack areas, and evaluate the structural health of the bridge to be inspected based on the regional data of all crack areas to obtain an assessment result, where N is an integer greater than zero.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] The present application discloses a bridge crack detection method and system based on a convolutional neural network, comprising: obtaining a grayscale image of a road surface of an i-th bridge section, and duplicating the grayscale image to obtain a first grayscale copy image and a second grayscale copy image; using a convolutional neural network to divide a first foreground area from the first grayscale copy image, and to divide a second foreground area from the second grayscale copy image; superimposing the first foreground area and the second foreground area, marking the overlapping area formed after the superimposition of the layers as a crack area, obtaining regional data of the crack area, and setting i=i+1; repeating the above steps until i=M. A loop is performed to obtain regional data of N crack areas, and the structural health of the bridge to be inspected is evaluated based on the regional data of all crack areas to obtain an evaluation result. Based on the above content, the present invention can significantly improve the segmentation accuracy of bridge crack detection and solve the problem of inaccurate crack segmentation in complex environments. The method has an efficient automated detection process, enhances the robustness of the system, and can intelligently evaluate the health status of the bridge structure and predict the development trend of cracks. Through accurate and timely monitoring and early warning, this solution provides reliable technical support for the safe management and maintenance of bridges, reduces potential safety risks, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 A flow chart of the bridge crack detection method based on convolutional neural network provided by the present invention;

[0074] Figure 2 This is a module structure diagram of the bridge crack detection system based on convolutional neural network provided by the present invention. DETAILED DESCRIPTION

[0075] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0076] Example 1

[0077] See also Figure 1 As shown, this embodiment discloses a bridge crack detection method based on a convolutional neural network, the method comprising:

[0078] S101: Obtain a road surface grayscale image of an i-th bridge section in a road section set, and duplicate the road surface grayscale image to obtain a first grayscale duplicate image and a second grayscale duplicate image of the i-th bridge section; the road section set includes M bridge sections of the bridge to be detected, where i and M are both integers greater than zero;

[0079] It should be noted that the road section set includes M bridge sections of the bridge to be inspected. The M bridge sections are formed by technicians by dividing the bridge to be inspected into equal sections. The length and width of each bridge section obtained after equal division are completely consistent.

[0080] Specifically, before obtaining the road surface grayscale image of the i-th bridge section in the road section set, the following steps are performed:

[0081] Obtain a road surface color image of the i-th bridge section in the road section set, perform grayscale processing on the road surface color image, and obtain an original grayscale image;

[0082] Obtaining image parameter data of the original grayscale image, and inputting the image parameter data into a pre-built quality coefficient calculation model to obtain the quality coefficient of the original grayscale image;

[0083] Specifically, the image parameter data includes image size, grayscale value of each pixel, average grayscale value, grayscale standard deviation, and gradient in the horizontal x direction and vertical y direction;

[0084] The mathematical expression of the pre-built quality coefficient calculation model is as follows:

[0085]

[0086] Where: Q(I) is the quality coefficient; H and W are the height and width of the image, which are used to represent the image size; σ is the grayscale standard deviation; μ is the average grayscale value; I(x,y) is the grayscale value at the position (x,y) in the image, Represents the gradient of the image in the horizontal x direction; Represents the gradient of the image in the vertical y direction;

[0087] Comparing the quality coefficient with a preset quality coefficient range;

[0088] If the quality coefficient is within the preset quality coefficient range, the original grayscale image is used as the road surface grayscale image of the i-th bridge section;

[0089] If the quality coefficient is outside the preset quality coefficient range, the original grayscale image is enhanced using a preset nonlinear transformation model to obtain an enhanced original grayscale image;

[0090] It can be understood that when the quality coefficient is within the preset quality coefficient range, it means that the quality coefficient is less than or equal to the maximum value of the quality coefficient range and greater than or equal to the minimum value of the quality coefficient range; and when the quality coefficient is outside the preset quality coefficient range, it means that the quality coefficient is greater than the maximum value of the quality coefficient range, or less than the minimum value of the quality coefficient range. When the quality coefficient is within the preset quality coefficient range, it means that the corresponding original grayscale image has a high contrast and no contrast enhancement processing is required. On the contrary, when the quality coefficient is outside the preset quality coefficient range, the original grayscale image is overexposed or underexposed, and grayscale value transformation is required to achieve the purpose of contrast enhancement.

[0091] Specifically, the expression of the preset nonlinear transformation model is as follows:

[0092]

[0093] Where: I′(x,y) is the grayscale value at position (x,y) in the enhanced image; tanh(·) is the hyperbolic tangent function; α is a parameter used to control the contrast enhancement degree of the output image. A larger α value will enhance the contrast and highlight the details; β is a parameter used to adjust the nonlinear degree of image grayscale change. A larger β value will amplify smaller grayscale differences more; γ is a parameter used to adjust the overall brightness of the output image, which can be used to compensate for the brightness change after image enhancement; α>β>γ>0;

[0094] It is understandable that, in some alternative solutions, the preset nonlinear transformation model may be an existing nonlinear transformation model, such as a gamma transformation and a logarithmic transformation, etc., and the present invention does not make any specific limitation thereto, and any existing nonlinear transformation model may be an application object of the present invention;

[0095] The enhanced original grayscale image is marked as the road surface grayscale image of the i-th bridge section;

[0096] It should be understood that the first grayscale copy image and the second grayscale copy image of the i-th bridge section are completely consistent in all aspects.

[0097] S102: using a preconfigured first convolutional neural network to separate a first foreground area from the first grayscale copy image, and using a preconfigured second convolutional neural network to separate a second foreground area from the second grayscale copy image;

[0098] In an implementation, dividing the first foreground area from the first grayscale copy image includes:

[0099] Extract the first grayscale copy image and convert it into a two-dimensional matrix Where: A is a two-dimensional matrix with m rows and n columns, a mn Represents the grayscale value of the pixel in the mth row and nth column of the matrix;

[0100] Inputting the two-dimensional matrix into a preconfigured first convolutional neural network to obtain a prediction mask, wherein the prediction mask displays a first foreground region where the bridge crack is located and indicates whether each pixel belongs to the crack region;

[0101] The training logic of the preconfigured first convolutional neural network is as follows:

[0102] Acquire historical image segmentation training data, and divide the historical image segmentation training data into an image segmentation training set and an image segmentation test set; the historical image segmentation training data includes a first grayscale copy image and its corresponding prediction mask;

[0103] Construct a convolutional neural network, use the first grayscale copy image in the image segmentation training set as the input of the convolutional neural network, and the predicted mask in the image segmentation training set as the output of the convolutional neural network, to obtain an initial convolutional neural network;

[0104] The initial convolutional neural network is model-verified using the image segmentation test set, and the initial convolutional neural network that minimizes the preset loss function is obtained as the preconfigured first convolutional neural network;

[0105] Specifically, the expression of the preset loss function is as follows:

[0106]

[0107] Where: L is the loss function, R is the total number of pixels in the image, Z r is the true label of the r-th pixel, and its value is 0 or 1, 0 means that the pixel does not belong to the crack area, and 1 means it belongs to the crack area; is the predicted value of the r-th pixel, indicating the probability that the r-th pixel belongs to the crack area, with a value between 0 and 1; A is the predicted crack area, a binary mask generated by the model, representing the set of all pixels that the network believes belong to the crack area; B is the actual crack area, the set of all pixels marked as crack areas in the true label;

[0108] Similarly, the training process of the preconfigured second convolutional neural network is the same, except that the input of the second convolutional neural network is the second grayscale copy image. For details, refer to the training logic of the first convolutional neural network, which will not be repeated here;

[0109] The first convolutional neural network includes 6 convolutional layers, 3 pooling layers, 2 hybrid attention modules, 1 transformer module, 1 global average pooling layer and 1 fully connected layer;

[0110] The convolution kernel size of the first convolution layer is 7×7, the stride is 2, and the number of convolution kernel channels is 64;

[0111] The convolution kernel size of the second convolutional layer is 5×5, the stride is 2, and the number of convolution kernel channels is 128;

[0112] The convolution kernel size of the third convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 256;

[0113] The convolution kernel size of the fourth convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 256;

[0114] The convolution kernel size of the fifth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 512;

[0115] The convolution kernel size of the sixth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 1024. The first pooling layer is a maximum pooling layer, the pooling kernel size is 3×3, and the stride is 2.

[0116] The second pooling layer is a maximum pooling layer with a pooling kernel size of 2×2 and a stride of 2;

[0117] The third pooling layer is a maximum pooling layer with a pooling kernel size of 2×2 and a step size of 2.

[0118] Among them, the hybrid attention module includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism calculates the channel weight through global average pooling and global maximum pooling, and the spatial attention mechanism calculates the spatial weight through a 7×7 convolution kernel;

[0119] The transformer module enhances global feature perception through a multi-head self-attention mechanism;

[0120] The global average pooling layer is used to perform global pooling on the feature map and compress the convolution output features to a fixed size;

[0121] The fully connected layer is used to output the prediction mask for crack detection, and the activation function is Sigmoid;

[0122] The second convolutional neural network includes: 5 convolutional layers, 2 pooling layers, 2 sparse convolution modules, 1 adaptive feature enhancement module, 1 multi-branch structure module, 1 global average pooling layer and 1 fully connected layer;

[0123] The convolution kernel size of the first convolution layer is 3×3, the stride is 1, and the number of convolution kernel channels is 64;

[0124] The convolution kernel size of the second convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 128;

[0125] The convolution kernel size of the third convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 256;

[0126] The convolution kernel size of the fourth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 512;

[0127] The convolution kernel size of the fifth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 1024;

[0128] Wherein, the two pooling layers are the first pooling layer and the second pooling layer respectively;

[0129] The first pooling layer is a maximum pooling layer with a pooling kernel size of 2×2 and a stride of 2;

[0130] The second pooling layer is an average pooling layer with a pooling kernel size of 2×2 and a stride of 2;

[0131] The sparse convolution module uses a sparse convolution kernel to perform convolution operations. The convolution kernel size is 3×3, which reduces the amount of calculation and retains key features.

[0132] The adaptive feature enhancement module dynamically fuses multi-scale features through convolution kernels of different scales (1×1, 3×3, 5×5);

[0133] The multi-branch structure module extracts crack features of different scales by using multiple convolution kernels (1×1, 3×3, 5×5) in parallel;

[0134] The global average pooling layer performs global pooling on the convolution output to reduce the dimension of the feature map;

[0135] The fully connected layer is used to output the prediction mask of the crack area, and the activation function is Sigmoid.

[0136] S103: Overlaying the first foreground area and the second foreground area, marking the overlapping area formed by the overlaying as a crack area, obtaining region data of the crack area, setting i=i+1, and returning to step S101; the region data includes the region area and the number of regions;

[0137] The area of the region is obtained by calculating the pixel area of each region based on a predefined proportional conversion relationship. To further explain, the proportional conversion relationship between the pixel area and the actual region area is pre-stored in the system database. The proportional conversion relationship is set by a technician based on experiments or experience. The pixel area of the crack region is proportionally converted using the proportional conversion relationship to obtain the region area of each crack region.

[0138] It should be understood that before overlaying the first foreground area and the second foreground area, the first foreground area or the second foreground area needs to be set to a semi-transparent state, and the semi-transparency ratio of the first foreground area or the second foreground area is 50%.

[0139] S104: Repeat steps S101 to S103 until i=M, and then terminate the loop. Regional data of N crack regions are obtained, and the structural health of the bridge to be inspected is evaluated based on the regional data of all crack regions to obtain an evaluation result, where N is an integer greater than zero.

[0140] In implementation, the assessment of the structural health of the bridge to be inspected includes:

[0141] According to the regional data of all crack areas, the cumulative area of all crack areas is summed up to obtain the cumulative area of the cracks in the bridge to be detected.

[0142] Retrieving the total surface area of the bridge to be inspected, and calculating the ratio of the cumulative area of the cracks to the total surface area of the bridge to be inspected to obtain an area ratio;

[0143] Setting an area ratio threshold and comparing the area ratio with the area ratio threshold; the area ratio threshold includes an area ratio threshold Td1 and an area ratio threshold Td2, Td1>Td2>0;

[0144] If the area ratio is greater than the area ratio threshold Td1, the bridge to be inspected is determined to be at level three health level, and the level three health level is used as the evaluation result;

[0145] If the area ratio is less than or equal to the area ratio threshold Td1 and greater than or equal to the area ratio threshold Td2, the bridge to be inspected is determined to be of level 2 health, and the level 2 health is used as the evaluation result;

[0146] If the area ratio is less than the area ratio threshold Td2, the bridge to be inspected is determined to be of level one health, and the level one health level is used as the evaluation result;

[0147] It should be noted that: the first-level health level is higher than the second-level health level, and the second-level health level is higher than the third-level health level; further explanation is that if the assessment result shows that the bridge to be inspected is at the third-level health level, it means that the bridge to be inspected has a large number of bridge cracks, and the bridge to be inspected is qualitatively diagnosed to have serious structural health problems, and the relevant municipal departments are reminded to replace the corresponding bridge in time; if the assessment result shows that the bridge to be inspected is at the second-level health level, it means that the bridge to be inspected has a certain amount of bridge cracks, and the bridge to be inspected is qualitatively diagnosed to have general structural health problems, and the relevant road maintenance departments are reminded to maintain the corresponding bridge in time; if the assessment result shows that the bridge to be inspected is at the first-level health level, it means that the bridge to be inspected has a small amount of or no bridge cracks, and the bridge to be inspected is qualitatively diagnosed to have no structural health problems, and there is no need to take any measures for the bridge to be inspected;

[0148] Through convolutional neural network technology, the segmentation accuracy of bridge crack detection is significantly improved, solving the problem of inaccurate crack segmentation in complex environments. This method has an efficient automated detection process, enhances the robustness of the system, and can intelligently assess the health of bridge structures and predict crack development trends. Through accurate and timely detection and early warning, this solution provides reliable technical support for bridge safety management and maintenance, reduces potential safety risks, and has broad application prospects.

[0149] Example 2

[0150] See also Figure 2 As shown, based on the same inventive concept, this embodiment discloses a bridge crack detection system based on a convolutional neural network. For details not provided in this embodiment, please refer to the relevant parts of the above embodiment 1. The system includes:

[0151] An image copy module 210 is configured to obtain a road surface grayscale image of the i-th bridge section in a road section set, and to copy the road surface grayscale image to obtain a first grayscale copy image and a second grayscale copy image of the i-th bridge section; the road section set includes M bridge sections of the bridge to be detected, where i and M are both integers greater than zero;

[0152] a region segmentation module 220 for segmenting a first foreground region from the first grayscale copy image using a preconfigured first convolutional neural network, and segmenting a second foreground region from the second grayscale copy image using a preconfigured second convolutional neural network;

[0153] The crack detection module 230 is configured to perform layer superposition on the first foreground area and the second foreground area, mark the overlapping area formed after the layer superposition as a crack area, obtain region data of the crack area, and set i=i+1 to trigger the image replication module 210; the region data includes the region area and the number of regions;

[0154] The bridge assessment module 240 is configured to repeat the steps from the image replication module 210 to the crack detection module 230 until the loop ends when i=M, obtain regional data of N crack regions, and assess the structural health of the bridge to be inspected based on the regional data of all crack regions to obtain an assessment result, where N is an integer greater than zero.

[0155] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters, weights and thresholds in the formulas are set by technicians in this field according to actual conditions.

[0156] The above embodiments can be implemented in whole or in part through software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired or wireless network. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, or a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0157] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0158] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0159] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only one type. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0160] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0161] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0162] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0163] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A bridge crack detection method based on convolutional neural network, characterized in that: The method comprises: S101: Obtain a road surface grayscale image of an i-th bridge section in a road section set, and duplicate the road surface grayscale image to obtain a first grayscale duplicate image and a second grayscale duplicate image of the i-th bridge section; the road section set includes M bridge sections of the bridge to be detected, where i and M are both integers greater than zero; S102: using a preconfigured first convolutional neural network to separate a first foreground area from the first grayscale copy image, and using a preconfigured second convolutional neural network to separate a second foreground area from the second grayscale copy image; S103: Overlaying the first foreground area and the second foreground area, marking the overlapping area formed by the overlaying as a crack area, obtaining region data of the crack area, setting i=i+1, and returning to step S101; the region data includes the region area and the number of regions; S104: Repeat the above steps S101 to S103 until the loop ends when i=M, obtain regional data of N crack areas, and evaluate the structural health of the bridge to be inspected based on the regional data of all crack areas to obtain an evaluation result, where N is an integer greater than zero.

2. The bridge crack detection method based on convolutional neural network according to claim 1 is characterized in that: Before obtaining the road surface grayscale image of the i-th bridge section in the road section set, it includes: Obtain a road surface color image of the i-th bridge section in the road section set, perform grayscale processing on the road surface color image, and obtain an original grayscale image; Obtaining image parameter data of the original grayscale image and inputting the image parameter data into a pre-built quality coefficient calculation model to obtain the quality coefficient of the original grayscale image; the image parameter data includes image size, grayscale value of each pixel, average grayscale value, grayscale standard deviation, and gradient in the horizontal x direction and vertical y direction; The mathematical expression of the pre-built quality coefficient calculation model is as follows: Where: Q(I) is the quality coefficient; H and W are the height and width of the image, which are used to represent the image size; σ is the grayscale standard deviation; μ is the average grayscale value; I(x,y) is the grayscale value at the position (x,y) in the image, Represents the gradient of the image in the horizontal x direction; Represents the gradient of the image in the vertical y direction; Comparing the quality coefficient with a preset quality coefficient range; If the quality coefficient is within the preset quality coefficient range, the original grayscale image is used as the road surface grayscale image of the i-th bridge section; If the quality coefficient is outside the preset quality coefficient range, the original grayscale image is enhanced using a preset nonlinear transformation model to obtain an enhanced original grayscale image; The enhanced original grayscale image is marked as the road surface grayscale image of the i-th bridge section.

3. The bridge crack detection method based on convolutional neural network according to claim 2 is characterized in that: The step of dividing the first foreground area from the first grayscale copy image includes: Extract the first grayscale copy image and convert it into a two-dimensional matrix Where: A is a two-dimensional matrix with m rows and n columns, a mn Represents the grayscale value of the pixel in the mth row and nth column of the matrix; The two-dimensional matrix is input into a preconfigured first convolutional neural network to obtain a prediction mask. The prediction mask shows a first foreground area where the bridge crack is located, indicating whether each pixel point belongs to the crack area.

4. The bridge crack detection method based on convolutional neural network according to claim 3 is characterized in that: The training logic of the preconfigured first convolutional neural network is as follows: Acquire historical image segmentation training data, and divide the historical image segmentation training data into an image segmentation training set and an image segmentation test set; the historical image segmentation training data includes a first grayscale copy image and its corresponding prediction mask; Construct a convolutional neural network, use the first grayscale copy image in the image segmentation training set as the input of the convolutional neural network, and the predicted mask in the image segmentation training set as the output of the convolutional neural network, to obtain an initial convolutional neural network; The initial convolutional neural network is model verified using the image segmentation test set, and the initial convolutional neural network that minimizes the preset loss function is obtained as the preconfigured first convolutional neural network.

5. The bridge crack detection method based on convolutional neural network according to claim 4 is characterized in that: The expression of the preset loss function is as follows: Where: L is the loss function, R is the total number of pixels in the image, Z r is the true label of the r-th pixel; is the predicted value of the r-th pixel; A is the predicted crack area, which is the set of all pixels belonging to the crack area; B is the actual crack area, which is the set of all pixels marked as the crack area in the real label.

6. The bridge crack detection method based on convolutional neural network according to claim 5 is characterized in that: The first convolutional neural network includes 6 convolutional layers, 3 pooling layers, 2 hybrid attention modules, 1 transformer module, 1 global average pooling layer and 1 fully connected layer; Among them, the convolution kernel size of the first convolution layer is 7×7, the stride is 2, and the number of convolution kernel channels is 64; The convolution kernel size of the second convolutional layer is 5×5, the stride is 2, and the number of convolution kernel channels is 128; The convolution kernel size of the third convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 256; The convolution kernel size of the fourth convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 256; The convolution kernel size of the fifth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 512; The convolution kernel size of the sixth convolution layer is 1×1, the stride is 1, and the number of convolution kernel channels is 1024. The first pooling layer is a maximum pooling layer with a pooling kernel size of 3×3 and a stride of 2. The second pooling layer is a maximum pooling layer with a pooling kernel size of 2×2 and a stride of 2; The third pooling layer is a maximum pooling layer with a pooling kernel size of 2×2 and a step size of 2. Among them, the hybrid attention module includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism calculates the channel weight through global average pooling and global maximum pooling, and the spatial attention mechanism calculates the spatial weight through a 7×7 convolution kernel; The transformer module enhances global feature perception through a multi-head self-attention mechanism; The global average pooling layer is used to perform global pooling on the feature map and compress the convolution output features to a fixed size; The fully connected layer is used to output the prediction mask for crack detection, and the activation function is Sigmoid.

7. The bridge crack detection method based on convolutional neural network according to claim 6 is characterized in that: The second convolutional neural network includes: 5 convolutional layers, 2 pooling layers, 2 sparse convolution modules, 1 adaptive feature enhancement module, 1 multi-branch structure module, 1 global average pooling layer and 1 fully connected layer; Among them, the convolution kernel size of the first convolution layer is 3×3, the stride is 1, and the number of convolution kernel channels is 64; The convolution kernel size of the second convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 128; The convolution kernel size of the third convolutional layer is 3×3, the stride is 1, and the number of convolution kernel channels is 256; The convolution kernel size of the fourth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 512; The convolution kernel size of the fifth convolutional layer is 1×1, the stride is 1, and the number of convolution kernel channels is 1024; Wherein, the two pooling layers are the first pooling layer and the second pooling layer respectively; The first pooling layer is a maximum pooling layer with a pooling kernel size of 2×2 and a stride of 2; The second pooling layer is an average pooling layer with a pooling kernel size of 2×2 and a stride of 2; The sparse convolution module uses a sparse convolution kernel to perform convolution operations. The convolution kernel size is 3×3, which reduces the amount of calculation and retains key features. The adaptive feature enhancement module dynamically fuses multi-scale features through convolution kernels of different scales; The multi-branch structure module extracts crack features of different scales by using multiple convolution kernels in parallel; The global average pooling layer performs global pooling on the convolution output to reduce the dimension of the feature map; The fully connected layer is used to output the prediction mask of the crack area, and the activation function is Sigmoid.

8. The bridge crack detection method based on convolutional neural network according to claim 7 is characterized in that: The evaluation of the structural health of the bridge to be inspected includes: According to the regional data of all crack areas, the cumulative area of all crack areas is summed up to obtain the cumulative area of the cracks in the bridge to be detected. Retrieving the total surface area of the bridge to be inspected, and calculating the ratio of the cumulative area of the cracks to the total surface area of the bridge to be inspected to obtain an area ratio; Setting an area ratio threshold and comparing the area ratio with the area ratio threshold; the area ratio threshold includes an area ratio threshold Td1 and an area ratio threshold Td2, Td1>Td2>0; If the area ratio is greater than the area ratio threshold Td1, the bridge to be inspected is determined to be at level three health level, and the level three health level is used as the evaluation result; If the area ratio is less than or equal to the area ratio threshold Td1 and greater than or equal to the area ratio threshold Td2, the bridge to be inspected is determined to be of level 2 health, and the level 2 health is used as the evaluation result; If the area ratio is less than the area ratio threshold Td2, the bridge to be inspected is determined to be of level one health, and the level one health level is used as the evaluation result.

9. A bridge crack detection system based on convolutional neural network, characterized in that: include: an image copy module, configured to obtain a road surface grayscale image of the i-th bridge section in a road section set, and copy the road surface grayscale image to obtain a first grayscale copy image and a second grayscale copy image of the i-th bridge section; the road section set includes M bridge sections of the bridge to be detected, where i and M are both integers greater than zero; a region segmentation module, configured to segment a first foreground region from the first grayscale copy image using a preconfigured first convolutional neural network, and to segment a second foreground region from the second grayscale copy image using a preconfigured second convolutional neural network; a crack detection module configured to perform layer superposition on the first foreground area and the second foreground area, mark the overlapping area formed after the layer superposition as a crack area, obtain region data of the crack area, and set i=i+1 to trigger the image replication module; the region data includes the region area and the number of regions; The bridge assessment module is used to repeat the above-mentioned image replication module to the crack detection module until the loop ends when i=M, obtain the regional data of N crack areas, and evaluate the structural health of the bridge to be inspected based on the regional data of all crack areas to obtain an assessment result, where N is an integer greater than zero.

Images