Bridge crack intelligent recognition method and system based on efficient sampling and multi-scale fusion

Abstract

The application discloses a bridge crack intelligent identification method and system based on efficient sampling and multi-scale fusion, and relates to the technical field of image recognition. When a bridge crack segmentation model is constructed, a U-Net image segmentation convolutional neural network is taken as a baseline, in the down-sampling process in the coding stage, the sparse structure of an efficient down-sampling module is used to extract rich crack feature information, and the height and width of a feature map are reduced; in the up-sampling process in the decoding stage, the sparse structure of an efficient up-sampling module is used to search dense crack feature details, and the height and width of the feature map are amplified; a correction coordinate attention module is integrated at the jump connection between the coding structure and the decoding structure, the extraction of long-distance relations of the feature map by the model is enhanced, the learning of global channels and position information is considered, and therefore high-precision detection of bridge structure apparent cracks in a complex background is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a bridge crack intelligent recognition method and system based on efficient sampling and multi-scale fusion. BACKGROUND

[0002] As one of the important components of road traffic infrastructure, the integrity and stability of the bridge structure have an important influence on the travel safety of vehicles and pedestrians. However, with the increase of service time, under the influence of many adverse factors such as extreme natural environment, aging of construction materials and heavy traffic load, the bridge structure will gradually appear damage cracking (i.e. cracks). Among them, the apparent crack is the most common form of structural damage in bridge damage cracking, and is also an important object of bridge apparent detection. If the apparent cracks of the bridge structure cannot be found in time and repaired, with the passage of time, the aging of the concrete material and the corrosion of the internal steel bars of the structure will be accelerated, and further development into more serious structural damage will cause adverse effects on the strength and stability of the bridge structure. This not only greatly reduces the structural bearing capacity and service life of the bridge, but also causes unnecessary maintenance cost expenditure, and the repair work of the bridge cracks will also affect the normal traffic on the bridge surface, and even cause traffic accidents. Therefore, before the apparent cracks of the bridge structure develop viciously, accurately obtaining crack information and repairing it in time can ensure the long-term service performance of the bridge and maximize the reduction of bridge structure maintenance costs.

[0003] Based on the shortcomings of traditional manual detection methods, such as strong subjectivity, low accuracy, low work efficiency, high missed detection rate, low safety, long time consumption and high cost, the method cannot meet the huge detection needs of bridges in China at present. In order to more objectively reflect the real situation of the apparent cracks of the bridge structure and provide accurate and reliable crack data support for the bridge maintenance and management department, a scientific and efficient detection method must be used. With the explosive development of deep learning technology, it has made a major breakthrough in the field of computer vision and other engineering fields, and greatly promoted the development of industrial intelligence. Based on this, domestic and foreign researchers have carried out a series of researches on bridge structure apparent crack intelligent recognition algorithms based on deep learning, and promoted the development of bridge crack automatic detection. However, based on the complex and changeable service environment of real bridges, and the existence of oil stains, artificial lines, construction joints, water marks, graffiti and other background interference on the structure surface, the existing bridge structure apparent crack intelligent recognition algorithms based on deep learning have a high false detection rate, that is, it is easy to misidentify the interference features similar to the real crack features (such as construction joints and water marks) as cracks, resulting in that the overall robustness and generalization still have a large space for improvement. Therefore, how to realize the high-precision detection of the apparent cracks of the bridge structure under complex background is a key problem to be solved in bridge apparent detection. SUMMARY

[0004] In order to solve the key problem of how to realize high-precision detection of bridge structure apparent cracks in a complex background, the application provides a bridge crack intelligent identification method and system based on efficient sampling and multi-scale fusion.

[0005] In a first aspect, the embodiments of the application provide a bridge crack intelligent identification method based on efficient sampling and multi-scale fusion, which comprises:

[0006] S100, scanning the bridge structure appearance to be detected to obtain bridge structure appearance image data to be detected;

[0007] S101, reading the bridge structure appearance image data to be detected, and inputting the preprocessed bridge structure appearance image data to be detected into a built bridge crack segmentation model for identification to obtain the detection result of the bridge structure appearance image to be detected; wherein the bridge crack segmentation model takes U-Net as a baseline, replaces the original down-sampling and up-sampling operations of U-Net with an efficient down-sampling module and an efficient up-sampling module, and integrates a corrected coordinate attention module at the jump connection between the U-Net encoder and decoder; the efficient down-sampling module is used to extract rich crack feature information and reduce the height and width of the feature map; the efficient up-sampling module is used to retrieve dense crack feature details and expand the height and width of the feature map; and the corrected coordinate attention module is used to enhance the extraction of long-distance relationships of the feature map by the model, so as to learn the global channel and position information.

[0008] In an embodiment, the building method of the bridge crack segmentation model comprises:

[0009] S1, constructing bridge structure apparent crack image data and bridge structure apparent crack true value image data; the bridge structure apparent crack image data is obtained by manually selecting bridge structure apparent crack image data containing crack features from the collected bridge structure apparent image data and expanding the bridge structure apparent crack image data, and the bridge structure apparent crack true value image data is obtained by manually labeling bridge structure apparent crack features according to the constructed bridge structure apparent crack image data;

[0010] S2, performing binaryzation processing on the bridge structure apparent crack true value image data: in the bridge structure apparent crack true value image data, setting the pixel value of the bridge structure apparent background as 0 and the pixel value of the bridge structure apparent crack as 1 to digitally represent the bridge structure apparent crack true value image data;

[0011] S3, training the bridge crack segmentation model by using the bridge structure apparent crack image data and the binaryzation processed bridge structure apparent crack true value image data.

[0012] In an embodiment, step S1 comprises the following sub-steps:

[0013] S11, manually selecting bridge structure apparent image data containing crack features from historical bridge structure apparent image data obtained by using a bridge intelligent detection image sensor, i.e. original bridge structure apparent crack image data;

[0014] S12, performing left-right flipping, up-down flipping, diagonal flipping and mosaic enhancement processing on the selected bridge structure apparent crack image data to obtain expanded bridge structure apparent crack image data;

[0015] S13, aggregating the original bridge structure apparent crack image data and the expanded bridge structure apparent crack image data to construct bridge structure apparent crack ground truth data.

[0016] In an embodiment, step S12 comprises the following sub-steps:

[0017] S121, left-right flipping: taking the half-width value in the width direction of the original bridge structure apparent crack image data as the reference axis, exchanging the data values symmetrically about the axis to obtain expanded bridge structure apparent crack image data of the original size;

[0018] S122, up-down flipping: taking the half-height value in the height direction of the original bridge structure apparent crack image data as the reference axis, exchanging the data values symmetrically about the axis to obtain expanded bridge structure apparent crack image data of the original size;

[0019] S123, diagonal flipping: taking the half-height value in the height direction of the left-right flipped bridge structure apparent crack image data as the reference axis, exchanging the data values symmetrically about the axis to obtain expanded bridge structure apparent crack image data of the original size;

[0020] S124, mosaic enhancement: randomly selecting four original bridge structure apparent crack image data, center cropping each original bridge structure apparent crack image data according to the original height to width ratio to obtain one-fourth size bridge structure apparent crack image data, and then splicing the center cropped one-fourth size bridge structure apparent crack image data according to the original height to width ratio to obtain expanded bridge structure apparent crack image data of the original size.

[0021] In an embodiment, step S3 comprises the following sub-steps:

[0022] S31, performing convolution, batch normalization and activation processing in sequence on the bridge structure apparent crack image data to obtain a feature map of size D, where D represents the original size of the bridge structure apparent crack image data;

[0023] S32, performing EDSM, convolution, batch normalization and activation processing on the feature map with size D in turn to obtain a feature map with size D / 2; similarly, a feature map with size D / 4 is obtained in turn; wherein, EDSM represents an efficient down-sampling module; wherein, n is the number of down-sampling;

[0024] S33, performing MCAM, convolution, batch normalization and activation processing on the feature map with size D / 4 in S32 to obtain a feature map with size D / 8; wherein, MCAM represents a modified coordinate attention module;

[0025] S34, performing MCAM processing on the feature map with size D / 4 in S32 to obtain a feature map with size D / 8; performing EUSM processing on the feature map with size D / 8 in S33 to obtain a feature map with size D / 4; finally, performing splicing processing on the two feature maps with size D / 8 in the channel dimension to obtain a feature map with size D / 4; similarly, a feature map with size D / 8 is obtained in turn; wherein, EUSM represents an efficient up-sampling module;

[0026] S35, performing convolution, batch normalization and activation processing on the feature map with size D spliced in S34 to obtain a final prediction output feature map, then performing loss calculation on the final prediction output feature map and the input binary processed bridge structure apparent value image data, and performing back propagation with the calculated loss to update the model parameters; similarly, multiple batches of back propagation training network are performed until the optimal weight matrix is obtained, and the bridge crack segmentation model is obtained based on the optimal weight matrix.

[0027] In an embodiment, the EDSM processing in step S32 includes the following sub-steps:

[0028] S321, performing maximum pooling, convolution, batch normalization and activation processing on the input feature map in turn to obtain a feature map with height, width and channel number being half of the input feature Figure One

[0029] S322, performing convolution, batch normalization and activation processing on the input feature map in turn to obtain a feature map with height, width and channel number being half of the input feature Figure One

[0030] S323, performing convolution, batch normalization and activation processing on the input feature map in turn to obtain a feature map with height and width being half of the input feature Figure One ​​​​​​​​​​​​half the number of input feature maps;

[0031] S324, the feature maps obtained in S321, S322 and S323 are spliced in the channel dimension to obtain a feature map with a height and a width both being input feature Figure One half the number of input feature maps;

[0032] S325, the feature map obtained in S324 is subjected to convolution, batch normalization and activation processing in sequence to obtain a feature map with a height and a width both being input feature Figure One half the number of input feature maps.

[0033] In an embodiment, the MCAM processing in step S33 includes the following sub-steps:

[0034] S331, maximum value pooling in the width dimension and high-width dimension data transposition processing are performed on the input feature map to obtain a feature map with a height of 1, a width of the height of the input feature map and a channel number equal to the input feature map;

[0035] S332, maximum value pooling in the height dimension is performed on the input feature map to obtain a feature map with a height of 1, a width and a channel number both equal to the input feature map;

[0036] S333, the feature maps obtained in S331 and S332 are spliced in the width dimension, and then convolution, batch normalization and activation processing are performed in sequence to obtain a feature map with a height of 1, a width of the height and width of the input feature map and a channel number of input feature Figure Four half the number of input feature maps;

[0037] S334, the feature map obtained in S333 is cut in the width dimension to obtain a feature map with a height of 1, a width of the height of the input feature map and a channel number of input feature Figure Four half the number of input feature maps and a feature map with a height of 1, a width equal to the input feature map and a channel number of input feature Figure Four half the number of input feature maps;

[0038] S335, the feature map obtained in S334 with a height of 1, a width of the height of the input feature map and a channel number of input feature Figure Four half the number of input feature maps are subjected to high-width dimension data transposition, convolution, batch normalization and activation processing in sequence to obtain a feature map with a height equal to the input feature map, a width of 1 and a channel number equal to the input feature map;

[0039] S336, the feature map obtained in S334 with a height of 1, a width equal to the input feature map and a channel number of input feature Figure FourThe feature map with a height of 1, a width equal to the input feature map, and a channel number equal to the input feature map is obtained by sequentially performing convolution, batch normalization, and activation processing on the feature map with a height of 1 / 8, a width equal to the input feature map, and a channel number equal to the input feature map.

[0040] S337, performing matrix multiplication processing on the feature maps obtained in S335 and S336 to obtain a feature map with a height, a width, and a channel number equal to the input feature map.

[0041] S338, performing matrix multiplication processing on the feature map obtained in S337 and the original input feature map to obtain a feature map with a height, a width, and a channel number equal to the input feature map.

[0042] In an embodiment, the EUSM processing in step S34 includes the following sub-steps:

[0043] S341, sequentially performing bilinear up-sampling with a sampling factor of 2, convolution, batch normalization, and activation processing on the input feature map to obtain a feature map with a height and a width twice that of the input feature map and a channel number equal to the input feature map. Figure One

[0044] S342, first performing transpose convolution, batch normalization, and activation processing on the input feature map, and then performing convolution, batch normalization, and activation processing to obtain a feature map with a height and a width twice that of the input feature map and a channel number equal to the input feature map. Figure One

[0045] S343, first performing transpose convolution, batch normalization, and activation processing on the input feature map, and then performing convolution, batch normalization, and activation processing to obtain a feature map with a height and a width twice that of the input feature map and a channel number equal to the input feature map.

[0046] S344, concatenating the feature maps obtained in S341, S342, and S343 in the channel dimension to obtain a feature map with a height, a width, and a channel number twice that of the input feature map.

[0047] S345, sequentially performing convolution, batch normalization, and activation processing on the feature map obtained in S344 to obtain a feature map with a height and a width twice that of the input feature map and a channel number equal to the input feature map.

[0048] In a second aspect, the embodiments of the present application also provide a bridge crack intelligent identification system based on efficient sampling and multi-scale fusion, which is used to execute the method of any of the above embodiments and includes:

[0049] The acquisition module is configured to scan a to-be-detected bridge structure surface to obtain to-be-detected bridge structure surface image data.

[0050] ​​The identification module is used for reading bridge structure apparent image data to be detected, and inputting the bridge structure apparent image data to be detected after preprocessing to a built bridge crack segmentation model for identification to obtain a detection result of the bridge structure apparent image data to be detected.

[0051] In the construction of the convolutional neural network model, the image segmentation convolutional neural network U-Net is taken as a baseline, in the down-sampling process in the encoding stage, the designed efficient down-sampling module (EDSM) is used to extract rich crack feature information, and the height and width of the feature map are reduced; in the up-sampling process in the decoding stage, the designed efficient up-sampling module (EUSM) is used to search dense crack feature details, and the height and width of the feature map are expanded; the MCAM is integrated in the skip connection between the encoding structure and the decoding structure, the extraction of the long-distance relationship of the feature map is enhanced, and the learning of the global channel and the position information can be considered, so that the high-precision detection of the bridge structure apparent crack is realized. Further, the segmentation effect of the bridge crack segmentation model has good robustness, and the bridge structure apparent image data with many background interferences such as oil stains, artificial lines, construction joints, water marks and graffiti can capture crack local details and reconstruct global information well, reduce the interference of noise, reduce the misjudgment rate of predicted crack features, and effectively solve the problem of high-precision detection of bridge structure apparent cracks in a complex background. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0053] Figure 1 The step flow chart of the bridge crack intelligent identification method based on efficient sampling and multi-scale fusion of the present application;

[0054] Figure 2This is another step in the intelligent bridge crack identification method based on efficient sampling and multi-scale fusion of the present invention.

[0055] Figure 3 This is a schematic diagram of the overall structure of the bridge crack segmentation model of the present invention;

[0056] Figure 4 This is a schematic diagram of the overall structure of the high-efficiency downsampling module of the present invention;

[0057] Figure 5 This is a schematic diagram of the overall structure of the correction coordinate attention module of the present invention;

[0058] Figure 6 This is a schematic diagram of the overall structure of the high-efficiency upsampling module of the present invention. Detailed Implementation

[0059] To make the technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0060] Furthermore, in order to better describe the present invention and its specific embodiments, the following technical terms that may appear will be explained in advance:

[0061] Conv k×k (stride=s, padding=p): indicates a two-dimensional convolution operation with kernel size k, stride s, and padding p;

[0062] Conv-T k×k (stride=s, padding=p): This indicates a two-dimensional transpose convolution operation with kernel size k, stride s, and padding p.

[0063] BN: short for "Batch Normalization", is a method of standard regularization that makes the feature maps of the image after convolution satisfy a distribution pattern with a mean of 0 and a variance of 1.

[0064] ReLU: A type of activation function that combines linear and nonlinear properties to provide more sensitive activation and input, avoiding saturation;

[0065] Sigmoid: A type of activation function that utilizes the characteristics of its S-shaped curve to ensure that the probability value obtained after encoding and decoding is between (0, 1), resulting in more accurate predictions.

[0066] like Figure 1 As shown in the figure, this invention provides a method for intelligent identification of bridge cracks based on efficient sampling and multi-scale fusion. The method includes:

[0067] S100, scanning the appearance of the bridge structure to be detected to obtain bridge structure appearance image data to be detected.

[0068] In step S100, specifically, the bridge intelligent detection image sensor carried by the truss bridge detection vehicle is used to scan the appearance of the bridge structure to be detected to obtain bridge structure appearance image data to be detected. The detection can be real-time detection.

[0069] S101, reading the bridge structure appearance image data to be detected, pre-processing and inputting it into the built bridge crack segmentation model for identification to obtain the detection result of the bridge structure appearance image to be detected; wherein the bridge crack segmentation model takes U-Net as the baseline, replaces the original down-sampling and up-sampling operations of U-Net with high-efficiency down-sampling modules and high-efficiency up-sampling modules, and integrates a modified coordinate attention module at the jump connection between the U-Net encoder and decoder; the high-efficiency down-sampling module is used to extract rich crack feature information and reduce the height and width of the feature map; the high-efficiency up-sampling module is used to retrieve dense crack feature details and expand the height and width of the feature map; the modified coordinate attention module is used to enhance the model's extraction of long-distance relationships of the feature map, so as to learn the global channel and position information.

[0070] In step S101, the high-efficiency down-sampling module (Efficient Down Sampling Module, EDSM), the high-efficiency up-sampling module (Efficient Up Sampling Module, EUSM), and the modified coordinate attention module (Modified Coordinate Attention Module, MCAM) are all modules designed by the present application.

[0071] The application takes the image segmentation convolutional neural network U-Net as a baseline when constructing a convolutional neural network model, in the down-sampling process in the encoding stage, the sparse structure of the designed efficient down-sampling module (EDSM) is used to extract rich crack feature information, and the height and width of the feature map are reduced, specifically, the height and width can be reduced by two times; in the up-sampling process in the decoding stage, the sparse structure of the designed efficient up-sampling module (EUSM) is used to search dense crack feature details, and the height and width of the feature map are expanded, specifically, the height and width can be expanded by two times; the MCAM is integrated into the jump connection between the encoding structure and the decoding structure, the extraction of long-distance relationships of the feature map by the model is enhanced, the learning of global channels and position information can be considered, so that the high-precision detection of the apparent crack of the bridge structure is realized. Further, the segmentation effect of the bridge crack segmentation model of the application has good robustness, for the bridge structure apparent image data with many background interferences such as oil stains, artificial lines, construction joints, water marks, graffiti and the like, the crack local details can be well captured and the global information can be well reconstructed, the noise interference is reduced, the misjudgment rate of the predicted crack features is reduced, and the problem of high-precision detection of the apparent crack of the bridge structure under a complex background is effectively solved.

[0072] In step S101, the preprocessing includes the following processes:

[0073] The detected bridge structure apparent image is normalized according to the following formula:

[0074]

[0075] wherein, represents the row number of the detected bridge structure apparent image, represents the column number of the detected bridge structure apparent image, represents the pixel value corresponding to the original detected bridge structure apparent image, represents the pixel value corresponding to the processed bridge structure apparent image, the value range is 0-1.

[0076] As shown in Figure 2 in an embodiment, the building method of the bridge crack segmentation model comprises:

[0077] S1, constructing bridge structure apparent crack image data and bridge structure apparent crack true value image data; the bridge structure apparent crack image data is obtained by manually selecting bridge structure apparent crack image data containing crack features from the collected bridge structure apparent image data and expanding, and the bridge structure apparent crack true value image data is obtained by manually labeling bridge structure apparent crack features according to the constructed bridge structure apparent crack image data;

[0078] S2, binaryzation processing is performed on the bridge structure apparent crack true value image data: in the bridge structure apparent crack true value image data, the pixel value of the bridge structure apparent background is set to 0, and the pixel value of the bridge structure apparent crack is set to 1, so as to digitally represent the bridge structure apparent crack true value image data;

[0079] S3, a bridge crack segmentation model is trained by using the bridge structure apparent crack image data and the binaryzation processed bridge structure apparent crack true value image data.

[0080] In the embodiment of the present application, the final segmentation effect of the bridge crack segmentation model obtained by S1-S3 has good robustness. For bridge structure apparent image data with many background interferences such as oil stains, artificial lines, construction joints, water marks, graffiti and the like, the bridge crack segmentation model can well capture the local details of the cracks and reconstruct the global information, reduce the interference of noise, reduce the misjudgment rate of the predicted crack features, and effectively solve the problem of high-precision detection of bridge structure apparent cracks in a complex background.

[0081] In an embodiment, step S1 includes the following sub-steps:

[0082] S11, the bridge structure apparent image data containing crack features, i.e. the original bridge structure apparent crack image data, is manually selected from the historical bridge structure apparent image data obtained by using the bridge intelligent detection image sensor;

[0083] S12, the selected bridge structure apparent crack image data is processed by left-right flipping, up-down flipping, diagonal flipping and mosaic enhancement to obtain expanded bridge structure apparent crack image data;

[0084] S13, the original bridge structure apparent crack image data and the expanded bridge structure apparent crack image data are summarized to construct the bridge structure apparent crack true value data.

[0085] Generally, the quantity and quality of the sample data set participating in the model training determine the performance of the trained model. When the quality of the training sample data set is poor, the performance of the trained model has a high misrecognition rate regardless of the quantity and type of the training sample data set. When the quality of the training sample data set is good, the quantity and type of the training sample data set are insufficient, and the trained model is prone to underfitting. The type of the training sample data set is sufficient and the quantity of the training sample data set is excessive, and the trained model is prone to overfitting. For bridge structure apparent cracks, due to the complex and changeable real environment of the bridge itself, and the existence of oil stains, artificial lines, construction joints, water marks, graffiti and other background interference on the structure surface of the bridge, it is difficult to construct the quantity and quality of the bridge structure apparent crack image training sample data set and the bridge structure apparent crack true value image training sample data set. Based on this, in the embodiment, first, a small amount of bridge structure apparent image data containing crack features is manually selected from the historical bridge structure apparent image data obtained by using a bridge intelligent detection image sensor. Second, the selected bridge structure apparent crack image data is processed by left-right flipping, up-down flipping and diagonal line flipping to increase the quantity of the bridge structure apparent crack image data. Third, the selected bridge structure apparent crack image data is processed by mosaic enhancement to increase the type of the bridge structure apparent crack image data. Finally, the original bridge structure apparent crack image data and the expanded bridge structure apparent crack image data are summarized to construct the bridge structure apparent crack true value data. In addition, to ensure the quality of the constructed bridge structure apparent crack true value image training sample data set, the data label annotation work of the bridge structure apparent crack image data is checked and revised for three rounds in the embodiment: first, several trained labelers manually label cracks on the provided bridge structure apparent crack image using annotation software; second, the labelers correct and revise each other; and third, experts with years of research experience in bridge structure apparent crack image data intelligent detection technology perform the final check and revision.

[0086] In an embodiment, step S12 comprises the following sub-steps:

[0087] S121, left-right flipping: taking the half-width value in the width direction of the original bridge structure apparent crack image data as the reference axis, the data values are exchanged symmetrically about the axis to obtain expanded bridge structure apparent crack image data of the original size;

[0088] S122, up-down flipping: taking the half-height value in the height direction of the original bridge structure apparent crack image data as the reference axis, the data values are exchanged symmetrically about the axis to obtain expanded bridge structure apparent crack image data of the original size;

[0089] S123, diagonal flip: taking the half height value in the height direction of the bridge structure apparent crack image data after left-right flip as the reference axis, the data value is exchanged symmetrically about the axis to obtain the expanded bridge structure apparent crack image data of the original size;

[0090] S124, mosaic enhancement: randomly selecting four original bridge structure apparent crack image data, center cropping each original bridge structure apparent crack image data according to the original height and width ratio to obtain a quarter size bridge structure apparent crack image data, then centering the cropped quarter size bridge structure apparent crack image data according to the original height and width ratio to obtain the expanded bridge structure apparent crack image data of the original size.

[0091] In an embodiment, step S3 comprises the following sub-steps:

[0092] S31, convolution, batch normalization and activation processing are performed on the bridge structure apparent crack image data in turn to obtain a feature map with a size of D, D representing the original size of the bridge structure apparent crack image data;

[0093] S32, EDSM, convolution, batch normalization and activation processing are performed on the feature map with a size of D in turn to obtain a feature map with a size of D / 2; similarly, feature maps with sizes of D / 4, D / 8, …, D / 2n are obtained in turn; wherein EDSM represents an efficient downsampling module; wherein n is the number of downsampling;

[0094] S33, MCAM, convolution, batch normalization and activation processing are performed on the feature map with a size of D / 2n in S32 to obtain a feature map with a size of D / 2n+1; wherein MCAM represents a modified coordinate attention module; S34, MCAM processing is performed on the feature map with a size of D / 2n in S32 to obtain a feature map with a size of D / 2n+1; EUSM processing is performed on the feature map with a size of D / 2n+1 in S33 to obtain a feature map with a size of D / 2n+2, and finally the two feature maps with sizes of D / 2n+1 are spliced in the channel dimension to obtain a feature map with a size of D / 2n+2; similarly, feature maps with sizes of D / 2n+3, D / 2n+4, …, D / 2n+2n are obtained in turn; wherein EUSM represents an efficient upsampling module;

[0095]

[0096] ​​​​​​​​​​S35, performing convolution, batch normalization and activation processing on the feature map with the size of D obtained by splicing in S34 to obtain a final prediction output feature map, then performing loss calculation on the final prediction output feature map and the input binary processed bridge structure apparent value image data, and performing back propagation with the calculated loss to update the model parameters; similarly, a plurality of batches of back propagation training network are performed until the optimal weight matrix is obtained, and the bridge crack segmentation model is obtained based on the optimal weight matrix.

[0097] In an embodiment, n is 6, and the above n is 6 is described as follows. For step S32, EDSM, convolution, batch normalization and activation processing are sequentially performed on the feature map with size D to obtain a feature map with size D / 2; EDSM, convolution, batch normalization and activation processing are sequentially performed on the feature map with size D / 2 to obtain a feature map with size D / 4; EDSM, convolution, batch normalization and activation processing are sequentially performed on the feature map with size D / 4 to obtain a feature map with size D / 8; EDSM, convolution, batch normalization and activation processing are sequentially performed on the feature map with size D / 8 to obtain a feature map with size D / 16; EDSM, convolution, batch normalization and activation processing are sequentially performed on the feature map with size D / 16 to obtain a feature map with size D / 32; EDSM, convolution, batch normalization and activation processing are sequentially performed on the feature map with size D / 32 to obtain a feature map with size D / 64; wherein EDSM represents an efficient downsampling module. For step S33, MCAM, convolution, batch normalization and activation processing are sequentially performed on the feature map with size D / 64 in S32 to obtain a feature map with size D / 64; wherein MCAM represents a modified coordinate attention module.For step S34, the MCAM processing is performed on the feature map with the size of D / 32 in S32 to obtain a feature map with the size of D / 32; the EUSM processing is performed on the feature map with the size of D / 64 in S33 to obtain a feature map with the size of D / 32, and then the concatenation processing in the channel dimension is performed on the two feature maps with the size of D / 32 to obtain a feature map with the size of D / 32; the MCAM, convolution, batch normalization and activation processing are performed on the feature map with the size of D / 16 in S32 to obtain a feature map with the size of D / 16; the EUSM processing is performed on the D / 32 feature map obtained by concatenation in S34 to obtain a feature map with the size of D / 16, and then the concatenation processing in the channel dimension is performed on the two feature maps with the size of D / 16 to obtain a feature map with the size of D / 16; the MCAM, convolution, batch normalization and activation processing are performed on the feature map with the size of D / 8 in S32 to obtain a feature map with the size of D / 8; the EUSM processing is performed on the D / 16 feature map obtained by concatenation in S34 to obtain a feature map with the size of D / 8, and then the concatenation processing in the channel dimension is performed on the two feature maps with the size of D / 8 to obtain a feature map with the size of D / 8; the MCAM, convolution, batch normalization and activation processing are performed on the feature map with the size of D / 4 in S32 to obtain a feature map with the size of D / 4; the EUSM processing is performed on the D / 8 feature map obtained by concatenation in S34 to obtain a feature map with the size of D / 4, and then the concatenation processing in the channel dimension is performed on the two feature maps with the size of D / 4 to obtain a feature map with the size of D / 4; the MCAM, convolution, batch normalization and activation processing are performed on the feature map with the size of D / 2 in S32 to obtain a feature map with the size of D / 2; the EUSM processing is performed on the D / 4 feature map obtained by concatenation in S34 to obtain a feature map with the size of D / 2, and then the concatenation processing in the channel dimension is performed on the two feature maps with the size of D / 2 to obtain a feature map with the size of D / 2; the MCAM, convolution, batch normalization and activation processing are performed on the feature map with the size of D in S32 to obtain a feature map with the size of D; the EUSM processing is performed on the D / 2 feature map obtained by concatenation in S34 to obtain a feature map with the size of D, and then the concatenation processing in the channel dimension is performed on the two feature maps with the size of D to obtain a feature map with the size of D; wherein, the EUSM represents the efficient up-sampling module.

[0098] More specifically, in the above step S31, specifically, the bridge structure crack image data and the bridge structure apparent crack true value image data matched therewith are read, and both of them are converted into feature matrix vectors to obtain a bridge structure apparent crack image feature matrix vector F1 and a bridge structure apparent crack true value image feature matrix vector T; the bridge structure apparent crack image feature matrix vector F1 is input to the MCAM, convolution, batch normalization and activation processing as shown in the following formula (1) to obtain a feature matrix vector F2. Figure 3In the bridge crack segmentation model shown, two "Conv 3x3 (stride=1) + BN + ReLU" processes are performed to preliminarily extract features of the input picture to obtain a feature map F2. The size of the input F1 is (512, 1024, 1), and the size of the output F2 is (512, 1024, 16). In the step S32, F2 in S31 is input into an efficient down sampling module (EDSM), which extracts rich crack feature information by using its sparse structure and reduces the height and width of F2, and then two "Conv 3x3 (stride=1, padding=1) + BN + ReLU" processes are performed to extract features to obtain a feature map F3 with a size of (256, 512, 32). Similarly, a feature map F4 (F4 is obtained based on F3), a feature map F5 (F5 is obtained based on F4), a feature map F6 (F6 is obtained based on F5), a feature map F7 (F7 is obtained based on F6), and a feature map F8 (F8 is obtained based on F7) are sequentially obtained, and the corresponding sizes are (128, 256, 64), (64, 128, 128), (32, 64, 256), (16, 32, 512), and (8, 16, 1024), respectively. In the step S33, the feature map F8 with a size of (8, 16, 1024) in S32 is input into a modified coordinate attention module (MCAM), which enhances the extraction of long-distance relationships of the feature map by the model, so that it can learn global channel and position information, and then two "Conv 3x3 (stride=1, padding=1) + BN + ReLU" processes are performed to extract features to obtain a feature map F9 with a size of (8, 16, 1024).In step S34, the feature map F7 with the size of (16, 32, 512) in S32 is input into the MCAM to obtain a feature map (16, 32, 512), the feature map F9 with the size of (8, 16, 1024) in S33 is input into an efficient up sampling module (EUSM) to search for dense crack feature details by using the sparse structure of the EUSM and expand the height and width of the feature map F9 to obtain a feature map (16, 32, 1024), the feature map (16, 32, 512) and the feature map (16, 32, 1024) are spliced in the channel dimension once to obtain a feature map (16, 32, 1536), and then two times of “Conv 3x3 (stride = 1, padding = 1) + BN + ReLU” processing are performed to extract features to obtain a feature map F10 with the size of (16, 32, 512). Similarly, a feature map F11 (F11 is obtained based on F10), a feature map F12 (F12 is obtained based on F11), a feature map F13 (F13 is obtained based on F12), a feature map F14 (F14 is obtained based on F13), and a feature map F15 (F15 is obtained based on F14) are sequentially obtained, and the corresponding sizes are (32, 64, 256), (64, 128, 128), (128, 256, 64), (256, 512, 32), and (512, 1024, 16), respectively. In step S35, one time of “Conv 1x1 (stride = 1, padding = 0) + Sigmoid” processing is performed on the feature map F15 with the size of (512, 1024, 16) obtained in S34 to obtain a final prediction output feature matrix vector O, then the feature map is subjected to Dice Loss calculation with an input bridge structure apparent crack true value image feature matrix vector T, and the loss obtained by the calculation is subjected to back propagation to update the model parameters. Similarly, a plurality of batches of back propagation training of the network is performed until an optimal weight matrix is obtained, and a bridge crack segmentation model is obtained based on the optimal weight matrix.

[0099] In the network back propagation training in the embodiments of the present application, the weight of the non-target pixel gradually decreases, and the weight of the target object pixel gradually increases until an optimal weight matrix is reached. The Dice Loss is a loss function used to monitor the coincidence degree between the result of network recognition and the bridge structure apparent true value image. The smaller the value of the loss function, the closer the result of network recognition is to the bridge structure apparent true value image.

[0100] In an embodiment, the EDSM processing in step S32 includes the following sub-steps:

[0101] S321, maximum value pooling, convolution, batch normalization and activation processing are performed on the input feature map in turn to obtain a feature map with the height, width and channel number of the input feature map; Figure One half the input feature map;

[0102] S322, convolution, batch normalization and activation processing are performed on the input feature map in turn to obtain a feature map with the height, width and channel number of the input feature map; Figure One half the input feature map;

[0103] S323, convolution, batch normalization and activation processing are performed on the input feature map in turn to obtain a feature map with the height and width of the input feature map; Figure One half and the channel number is equal to that of the input feature map;

[0104] S324, the feature maps obtained in S321, S322 and S323 are spliced in the channel dimension to obtain a feature map with the height and width of the input feature map; Figure One half and the channel number is equal to that of the input feature map;

[0105] S325, convolution, batch normalization and activation processing are performed on the feature map obtained in S324 in turn to obtain a feature map with the height and width of the input feature map; Figure One half and the channel number is equal to that of the input feature map.

[0106] As Figure 4As shown, in the step S321, specifically, the input feature map (H, W, C) is first subjected to a maximum pooling processing with a pooling kernel of 2 and a step of 2, and then subjected to a "Conv 1x1 (stride=1, padding=0) + BN + ReLU" processing, to obtain a feature map (H / 2, W / 2, C / 2); in the step S322, specifically, the input feature map (H, W, C) is sequentially subjected to a "Conv 3x3 (stride=2, padding=1) + BN + ReLU", a "Conv 3x3 (stride=1, padding=1) + BN + ReLU", and a "Conv 1x1 (stride=1, padding=0) + BN + ReLU" processing, to obtain a feature map (H / 2, W / 2, C / 2); in the step S323, specifically, the input feature map (H, W, C) is sequentially subjected to a "Conv 3x3 (stride=2, padding=1) + BN + ReLU" and a "Conv 1x1 (stride=1, padding=0) + BN + ReLU" processing, to obtain a feature map (H / 2, W / 2, C); in the step S324, specifically, the feature maps (H / 2, W / 2, C / 2), (H / 2, W / 2, C / 2), and (H / 2, W / 2, C) obtained in the steps S321, S322, and S323 are spliced in the channel dimension, to obtain a feature map (H / 2, W / 2, 2C); in the step S325, specifically, the (H / 2, W / 2, 2C) obtained in the step S324 is subjected to a "Conv 1x1 (stride=1, padding=0) + BN + ReLU" processing, to obtain an output feature map (H / 2, W / 2, C); thus, the EDSM processing is completed.

[0107] In an embodiment, the MCAM processing in the step S33 includes the following sub-steps:

[0108] S331, performing a maximum pooling in the width dimension and a data transposition in the height and width dimensions on the input feature map, to obtain a feature map with a height of 1, a width equal to the height of the input feature map, and a channel number equal to that of the input feature map;

[0109] S332, performing a maximum pooling in the height dimension on the input feature map, to obtain a feature map with a height of 1, a width, and a channel number both equal to that of the input feature map;

[0110] S333, perform splicing on the feature maps obtained in S331 and S332 in the width dimension, and then perform convolution, batch normalization and activation processing in sequence to obtain a feature map with a height of 1, a width equal to the height and width of the input feature map, and a channel number equal to the input feature map; Figure Four

[0111] S334, perform cutting on the feature map obtained in S333 in the width dimension to obtain a feature map with a height of 1, a width equal to the height of the input feature map, and a channel number equal to the input feature map Figure Four Figure Four

[0112] S335, perform high-width dimension data transposition, convolution, batch normalization and activation processing in sequence on the feature map with a height of 1, a width equal to the height of the input feature map, and a channel number equal to the input feature map Figure Four

[0113] S336, perform convolution, batch normalization and activation processing in sequence on the feature map with a height of 1, a width equal to the input feature map, and a channel number equal to the input feature map Figure Four

[0114] S337, perform matrix multiplication processing on the feature maps obtained in S335 and S336 to obtain a feature map with a height, a width, and a channel number all equal to the input feature map;

[0115] S338, perform matrix multiplication processing on the feature map obtained in S337 and the original input feature map to obtain a feature map with a height, a width, and a channel number all equal to the input feature map.

[0116] As Figure 5 ​​​​​As shown, in the step S331, specifically, the input feature map (H, W, C) is sequentially subjected to the maximum value pooling in the width dimension and the data transposition processing in the height and width dimensions to obtain the feature map (1, H, C); in the step S332, specifically, the input feature map (H, W, C) is subjected to the maximum value pooling in the height dimension to obtain the feature map (1, W, C); in the step S333, specifically, the feature map (1, W, C) and the feature map (1, H, C) obtained in the step S341 are spliced in the width dimension to obtain the feature map (1, (H+W), C), and the feature map (1, (H+W), C) is subjected to the processing of “Conv 3x3 (stride=1, padding=1) + BN + ReLU” to obtain the feature map (1, (H+W), C / 4); in the step S334, the feature map (1, (H+W), C / 4) obtained in the step S333 is subjected to the cutting processing in the width dimension to obtain the feature map (1, H, C / 4) and the feature map (1, W, C / 4), respectively; in the step S335, the feature map (1, H, C / 4) obtained in the step S334 is subjected to the data transposition processing in the height and width dimensions to obtain the feature map (H, 1, C / 4), and then subjected to the processing of “Conv 3x3 (stride=1, padding=1) + BN + Sigmoid” to obtain the feature map (H, 1, C); in the step S336, the feature map (1, W, C / 4) obtained in the step S334 is subjected to the processing of “Conv 3x3 (stride=1, padding=1) + BN + Sigmoid” to obtain the feature map (1, W, C); in the step S337, the feature map (H, 1, C) obtained in the step S335 and the feature map (1, W, C) obtained in the step S336 are subjected to the matrix multiplication processing to obtain the feature map (H, W, C); in the step S338, the feature map (H, W, C) obtained in the step S337 and the input feature map (H, W, C) are subjected to the matrix multiplication processing to obtain the output feature map (H, W, C). Thus, the MCAM processing of the feature map is completed.

[0117] In an embodiment, the EUSM processing in the step S34 includes the following sub-steps:

[0118] S341, the input feature map is sequentially subjected to the bilinear up-sampling with a sampling factor of 2, the convolution, the batch normalization and the activation processing to obtain the feature map with the height and the width being half of the input feature map and the channel number being Figure One half of the input feature map;

[0119] S342, performing transpose convolution, batch normalization and activation processing on the input feature map first, and then performing convolution, batch normalization and activation processing, to obtain a feature map with the height and width of the input feature map and the channel number equal to that of the input feature map; Figure One half the feature map of the input feature map;

[0120] S343, performing transpose convolution, batch normalization and activation processing on the input feature map first, and then performing convolution, batch normalization and activation processing, to obtain a feature map with the height and width of the input feature map and the channel number equal to that of the input feature map;

[0121] S344, performing feature splicing on the feature maps obtained in S341, S342 and S343 in the channel dimension, to obtain a feature map with the height, width and channel number of the input feature map;

[0122] S345, performing convolution, batch normalization and activation processing on the feature map obtained in S344, to obtain a feature map with the height and width of the input feature map and the channel number equal to that of the input feature map.

[0123] As Figure 6As shown, in the step S341, specifically, the input feature map (H, W, C) is first subjected to a bilinear up-sampling processing with a sampling factor of 2, and then subjected to a “Conv 1x1 (stride=1, padding=0) + BN + ReLU” processing, to obtain a feature map (2H, 2W, C / 2); in the step S342, specifically, the input feature map (H, W, C) is sequentially subjected to a “Conv_T 3x3 (stride=2, padding=1) + BN + ReLU”, a “Conv 3x3 (stride=1, padding=1) + BN + ReLU”, and a “Conv 1x1 (stride=1, padding=0) + BN + ReLU” processing, to obtain a feature map (2H, 2W, C / 2); in the step S343, specifically, the input feature map (H, W, C) is sequentially subjected to a “Conv_T 3x3 (stride=2, padding=1) + BN + ReLU” and a “Conv 1x1 (stride=1, padding=0) + BN + ReLU” processing, to obtain a feature map (2H, 2W, C); in the step S344, specifically, the feature maps (2H, 2W, C / 2), (2H, 2W, C / 2) and (2H, 2W, C) obtained in the steps S341, S342 and S343 are spliced in the channel dimension, to obtain a feature map (2H, 2W, 2C); in the step S345, specifically, the (2H, 2W, 2C) obtained in the step S344 is subjected to a “Conv 1x1 (stride=1, padding=0) + BN + ReLU” processing, to obtain an output feature map (2H, 2W, C); thus, the EUSM processing is completed.

[0124] The scheme of the present application is based on the original U-Net "encoding-decoding" structure, and an efficient down sampling module (EDSM), a modified coordinate attention module (MCAM) and an efficient up sampling module (EUSM) are designed and added. At the same time, the original U-Net is also deeper (considering more deep feature extraction); wherein the designed EDSM replaces the maximum pooling operation with a pooling kernel of 2 and a step of 2 in the original U-Net, uses its sparse structure to extract rich crack feature information, avoids the possible gradient disappearance problem caused by the single maximum pooling operation, and reduces the height and width of the feature map; the designed EUSM replaces the transpose convolution operation with a convolution kernel of 3, a convolution step of 2 and a padding of 1 in the original U-Net, uses its sparse structure to retrieve and restore dense crack feature details, avoids the possible loss of local detail information caused by the single transpose convolution operation, and expands the height and width of the feature map; the designed MCAM is integrated into the skip connection between the encoding structure and the decoding structure of the original U-Net to enhance the extraction of long-distance relationships of the feature map by the model, so that it can learn the global channel and position information, thereby realizing high-precision detection of the apparent cracks of the bridge structure.

[0125] In order to verify the advantages of the present application, relevant comparative experiments were carried out, specifically, based on the traditional U-Net algorithm model, DeepLabv3+ algorithm model, SegFormer algorithm model, Swin-Unet algorithm model and bridge crack segmentation model of the embodiment of the present application, 500 measured bridge structure apparent image data were tested. The overall indicators of each algorithm model on the 500 bridge structure apparent image data are shown in Table 1.

[0126] Table 1

[0127]

[0128] Among them, four indicators most representative in the field of semantic segmentation algorithm, namely precision (Precision), recall (Recall), F-measure and intersection over union (IOU), are adopted. The specific calculation of each indicator is as follows:

[0129]

[0130]

[0131]

[0132]

[0133] Wherein, TP is the number of true positives, FP is the number of false positives, FN is the number of false negatives, and further, the model in S3 predicts a crack condition: when the predicted image pixel value is greater than or equal to 0.5, the prediction result is a crack, otherwise it is considered as background.

[0134] It is worth mentioning that, The indicator is And The harmonic mean of and can more comprehensively reflect the excellent performance of the algorithm network. As can be seen from the above table, compared with the current mainstream network model: U-Net, DeepLabv3+, SegFormer and Swin-Unet, the bridge crack segmentation model proposed in the application has obvious advantages in identifying bridge surface cracks.

[0135] The embodiment of the application also provides a bridge crack intelligent identification system based on efficient sampling and multi-scale fusion, which is used to execute the method of any one of the above embodiments. The system comprises:

[0136] The acquisition module is used to scan the surface of the bridge structure to be detected to obtain surface image data of the bridge structure to be detected.

[0137] The identification module is used to read the surface image data of the bridge structure to be detected, and input the preprocessed data into the built bridge crack segmentation model for identification to obtain the detection result of the surface image of the bridge structure to be detected. The bridge crack segmentation model takes U-Net as the baseline, replaces the original down-sampling and up-sampling operations of U-Net with an efficient down-sampling module and an efficient up-sampling module, and integrates a modified coordinate attention module at the jump connection between the U-Net encoder and the decoder. The efficient down-sampling module is used to extract rich crack feature information and reduce the height and width of the feature map. The efficient up-sampling module is used to retrieve dense crack feature details and expand the height and width of the feature map. The modified coordinate attention module is used to enhance the extraction of long-distance relationships of the feature map by the model, so as to learn the global channel and position information.

[0138] It should be understood that the above methods are applicable to the system of the application, therefore, the embodiments of the application will not be described in detail here.

[0139] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, databases, or other media in this application and in the embodiments refers to both non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0140] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, devices, articles, or methods that include a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, devices, articles, or methods. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of other identical elements in the process, device, article, or method that includes the element.

[0141] The above specific embodiments further illustrate the purpose, technical solutions, and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A bridge crack intelligent identification method based on efficient sampling and multi-scale fusion, characterized by, The method comprises: S100, scanning a to-be-detected bridge structure appearance to obtain to-be-detected bridge structure appearance image data; S101, reading the to-be-detected bridge structure appearance image data, pre-processing the to-be-detected bridge structure appearance image data, and inputting the pre-processed to-be-detected bridge structure appearance image data into a built bridge crack segmentation model to obtain a detection result of the to-be-detected bridge structure appearance image; wherein the bridge crack segmentation model takes U-Net as a baseline, replaces original down-sampling and up-sampling operations of U-Net with a high-efficiency down-sampling module and a high-efficiency up-sampling module, and integrates a modified coordinate attention module at a skip connection between an encoder and a decoder of U-Net; the high-efficiency down-sampling module is used to extract rich crack feature information by using a sparse structure thereof and reduce a height and a width of a feature map; the high-efficiency up-sampling module is used to search dense crack feature details by using a sparse structure thereof and expand the height and the width of the feature map; and the modified coordinate attention module is used to enhance extraction of long-distance relationships of the feature map by the model, so as to learn global channels and position information. The modified coordinate attention module processing comprises the following sub-steps: S331, performing maximum value pooling in a width dimension and high-width dimension data transposition processing on an input feature map HxWxC to obtain a feature map of 1xHxC; S332, performing maximum value pooling in a height dimension on the input feature map to obtain a feature map of 1xWxC; S333, splicing the feature maps obtained in S331 and S332 in the width dimension, and then sequentially performing convolution, batch normalization and activation processing to obtain a feature map of 1x(H+W)xC / 4; S334, cutting the feature map obtained in S333 in the width dimension to obtain a feature map of 1xHxC / 4 and a feature map of 1xWxC / 4; S335, sequentially performing high-width dimension data transposition, convolution, batch normalization and activation processing on the feature map of 1xHxC / 4 obtained in S334 to obtain a feature map of Hx1xC; S336, sequentially performing convolution, batch normalization and activation processing on the feature map of 1xWxC / 4 obtained in S334 to obtain a feature map of 1xWxC; S337, performing matrix multiplication processing on the feature maps obtained in S335 and S336 to obtain a feature map of HxWxC; S338, performing matrix multiplication processing on the feature map obtained in S337 and the original input feature map to obtain a feature map of HxWxC.

2. The bridge crack intelligent identification method based on efficient sampling and multi-scale fusion according to claim 1, characterized in that, The method for building the bridge crack segmentation model comprises: S1, constructing bridge structure appearance crack image data and bridge structure appearance crack true value image data; the bridge structure appearance crack image data is obtained by artificially selecting bridge structure appearance crack image data containing crack features from collected bridge structure appearance image data and expanding the bridge structure appearance crack image data; and the bridge structure appearance crack true value image data is obtained by artificially labeling bridge structure appearance crack features according to the constructed bridge structure appearance crack image data; S2, binaryzation processing is performed on the bridge structure apparent crack true value image data; in the bridge structure apparent crack true value image data, the pixel value of the bridge structure apparent background is set to 0, and the pixel value of the bridge structure apparent crack is set to 1, so as to digitally represent the bridge structure apparent crack true value image data; S3, a bridge crack segmentation model is trained by using the bridge structure apparent crack image data and the binaryzation processed bridge structure apparent crack true value image data.

3. The bridge crack intelligent identification method based on efficient sampling and multi-scale fusion according to claim 2, characterized in that, Step S1 includes the following sub-steps: S11, the bridge structure apparent crack image data containing crack features is manually selected from the historical bridge structure apparent image data obtained by using the bridge intelligent detection image sensor, that is, the original bridge structure apparent crack image data; S12, the selected bridge structure apparent crack image data is processed by left-right flipping, up-down flipping, diagonal flipping and mosaic enhancement to obtain expanded bridge structure apparent crack image data; S13, the original bridge structure apparent crack image data and the expanded bridge structure apparent crack image data are summarized to construct the bridge structure apparent crack true value data.

4. The bridge crack intelligent identification method based on efficient sampling and multi-scale fusion according to claim 3, characterized in that, Step S12 includes the following sub-steps: S121, left-right flipping: taking the half width value in the width direction of the original bridge structure apparent crack image data as the reference axis, the data values are exchanged symmetrically about the axis to obtain the expanded bridge structure apparent crack image data with the original size; S122, up-down flipping: taking the half height value in the height direction of the original bridge structure apparent crack image data as the reference axis, the data values are exchanged symmetrically about the axis to obtain the expanded bridge structure apparent crack image data with the original size; S123, diagonal flipping: taking the half height value in the height direction of the left-right flipped bridge structure apparent crack image data as the reference axis, the data values are exchanged symmetrically about the axis to obtain the expanded bridge structure apparent crack image data with the original size; S124, mosaic enhancement: four original bridge structure apparent crack image data are randomly selected, and each original bridge structure apparent crack image data is center cropped according to the original height to width ratio to obtain a quarter size bridge structure apparent crack image data, and then the center cropped quarter size bridge structure apparent crack image data is spliced according to the original height to width ratio to obtain the expanded bridge structure apparent crack image data with the original size.

5. The bridge crack intelligent identification method based on efficient sampling and multi-scale fusion according to claim 2, characterized in that, Step S3 includes the following sub-steps: S31, convolution, batch normalization and activation processing are performed on the bridge structure apparent crack image data in sequence to obtain a feature map with a size of D, wherein D represents the original size of the bridge structure apparent crack image data; S32, EDSM, convolution, batch normalization and activation processing are performed on the feature map with a size of D in sequence to obtain a feature map with a size of D / 2; Similarly, the feature maps with sizes of are obtained in turn; wherein, EDSM represents a high-efficiency down-sampling module; wherein, n represents the number of down-sampling; S33, performing MCAM, convolution, batch normalization and activation processing on the feature map with the size of in S32 to obtain a feature map with the size of ; wherein, MCAM represents a modified coordinate attention module; S34, performing MCAM processing on the feature map with the size of in S32 to obtain a feature map with the size of ; performing EUSM processing on the feature map with the size of in S33 to obtain a feature map with the size of , and finally performing splicing processing on the channel dimension of the two feature maps with the size of to obtain a feature map with the size of ; similarly, a feature map with the size of is obtained in sequence; wherein EUSM represents an efficient up-sampling module. S35, performing convolution, batch normalization and activation processing on the feature map with the size of D obtained by splicing in S34 to obtain a final prediction output feature map, then performing loss calculation on the final prediction output feature map and the input binary processed bridge structure apparent value image data, and performing back propagation with the calculated loss to update the model parameters; similarly, performing multi-batch back propagation training network until the optimal weight matrix is obtained, and obtaining the bridge crack segmentation model based on the optimal weight matrix.

6. The bridge crack intelligent identification method based on efficient sampling and multi-scale fusion according to claim 5, characterized in that, The EDSM processing in step S32 includes the following sub-steps: S321, performing maximum pooling, convolution, batch normalization and activation processing on the input feature map in sequence to obtain a feature map with half the height, width and channel number of the input feature map; S322, performing convolution, batch normalization and activation processing on the input feature map in sequence to obtain a feature map with half the height, width and channel number of the input feature map; S323, performing convolution, batch normalization and activation processing on the input feature map in sequence to obtain a feature map with half the height and width of the input feature map and the same channel number as the input feature map; S324, splicing the feature maps obtained in S321, S322 and S323 in the channel dimension to obtain a feature map with half the height and width of the input feature map and twice the channel number of the input feature map; S325, performing convolution, batch normalization and activation processing on the feature map obtained in S324 in sequence to obtain a feature map with half the height and width of the input feature map and the same channel number as the input feature map.

7. The bridge crack intelligent identification method based on efficient sampling and multi-scale fusion according to claim 5, characterized in that, The EUSM processing in step S34 includes the following sub-steps: S341, performing bilinear upsampling with a sampling factor of 2, convolution, batch normalization and activation processing on the input feature map in sequence to obtain a feature map with twice the height and width of the input feature map and half the channel number of the input feature map; S342, performing transposed convolution, batch normalization and activation processing on the input feature map first, and then performing convolution, batch normalization and activation processing to obtain a feature map with twice the height and width of the input feature map and half the channel number of the input feature map; S343, performing transposed convolution, batch normalization and activation processing on the input feature map first, and then performing convolution, batch normalization and activation processing to obtain a feature map with twice the height and width of the input feature map and the same channel number as the input feature map; S344, splicing the feature maps obtained in S341, S342 and S343 in the channel dimension to obtain a feature map with twice the height, width and channel number of the input feature map; S345, performing convolution, batch normalization and activation processing on the feature map obtained in S344 in sequence to obtain a feature map with twice the height and width of the input feature map and the same channel number as the input feature map.

8. A bridge crack intelligent identification system based on efficient sampling and multi-scale fusion, characterized in that, The system is used to perform the method of any one of claims 1-7, comprising: a collection module for scanning a to-be-detected bridge structure to obtain to-be-detected bridge structure apparent image data; The identification module is used for reading apparent image data of a bridge structure to be detected, pre-processing the apparent image data, and inputting the pre-processed apparent image data into a built bridge crack segmentation model for identification to obtain a detection result of the apparent image data of the bridge structure to be detected; wherein the bridge crack segmentation model takes U-Net as a baseline, replaces original down-sampling and up-sampling operations of U-Net with a high-efficiency down-sampling module and a high-efficiency up-sampling module, and integrates a correction coordinate attention module at a skip connection between an encoder and a decoder of U-Net; the high-efficiency down-sampling module is used for extracting rich crack feature information and reducing a height and a width of a feature map; the high-efficiency up-sampling module is used for searching dense crack feature details and expanding the height and the width of the feature map; and the correction coordinate attention module is used for enhancing extraction of long-distance relationships of the feature map by the model, so that learning of global channels and position information is taken into account.

Images