Road crack detection method for inspection vehicle under overexposure background

Abstract

The invention belongs to the technical field of road management, and particularly relates to a road crack detection method for an inspection vehicle under an overexposure background, and the method comprises the following steps: S1, collecting a road crack picture under the overexposure background through the inspection vehicle; s2, carrying out two-dimensional space-time segmentation on the collected pictures, constructing space-time association between cracks, making a data set, and dividing the data set into a training set and a test set; and S3, constructing a road crack feature extraction module, a feature fusion and complementation module and a feature reconstruction module, combining the three modules by using a U-Net network, constructing a road crack detection system network model based on an overexposure background, and complementing interrupted and incomplete cracks caused by overexposure influence. According to the method, the situation that crack detection results are interrupted and hollow when actual road pictures collected by an inspection vehicle under the overexposure background are detected can be avoided, and the integrity of the crack detection results is guaranteed.

Description

Technical Field

[0001] The invention relates to the technical field of highway management, and in particular to a method for detecting road cracks under an overexposure background of an inspection vehicle. Background Art

[0002] Existing road crack detection methods are divided into two categories: one is the crack detection method based on digital images, which includes methods based on threshold segmentation, edge detection, region and matching, and the other is the detection method based on deep learning. The deep learning method generally uses image classification algorithm and image segmentation algorithm to detect cracks. For the actual road crack pictures collected by the inspection vehicle, the pictures are often overexposed. Under the irradiation of strong light, the shallower cracks will disappear, and the deeper cracks will remain, resulting in discontinuous cracks. The road crack detection technology based on digital images is often used in the case of a single background without too much interference noise. It has certain limitations, the image processing process is complex, and it is easily affected by the environment. The robustness and generalization ability are poor. The existing technology of road crack detection based on deep learning can automatically learn parameters from the data set, has fast detection speed and strong robustness, but the detection of road cracks under overexposed background often results in crack cavities and incomplete detection. Therefore, we propose a road crack detection method under overexposed background for inspection vehicles to solve the above problems. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] In view of the deficiencies in the prior art, the present invention provides a method for detecting road cracks under an overexposed background for an inspection vehicle, which solves the problems raised in the above-mentioned background technology.

[0005] (II) Technical solution

[0006] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0007] A method for detecting road cracks under an overexposed background of an inspection vehicle comprises the following steps:

[0008] S1: Use inspection vehicles to collect road crack images with overexposed backgrounds;

[0009] S2: Segment the collected images into two-dimensional space-time, construct the space-time correlation between the cracks, and create a data set, which is divided into a training set and a test set;

[0010] S3: Construct a road crack feature extraction module, a feature fusion and completion module, and a feature reconstruction module, and combine the three modules using the U-Net network to build a road crack detection system network model based on overexposure background, and complete the cracks that are interrupted and incomplete due to overexposure;

[0011] S4: Based on the data set, the network model of the road crack detection system is trained, and the trained model is used to detect road crack images under overexposed backgrounds, and the detection results are saved.

[0012] Furthermore, an industrial camera is installed on the top of the inspection vehicle in S1. The camera head of the industrial camera should be at a certain angle to the road surface. It is required to be able to clearly capture the road surface and not capture too many roadside interference factors. The data collected by the industrial camera is uploaded to the computer, and the software system completes the road surface data storage, processing, analysis and other tasks.

[0013] Furthermore, the specific content of constructing the spatiotemporal correlation between the cracks in S2 is as follows: first, in order to construct the temporal continuity of the cracks, the crack image at time t1 is used as the reference in the video collected by the inspection vehicle, and the image is taken every t seconds. Δ Select a picture, a total of M frames, M is an odd number, and then in order to construct the continuity of the crack in space, the selected M frames, the picture size is B×B, use the sliding window of size A×A to slide along the horizontal direction of the crack picture, A is the factor of B, the sliding step is A, then a crack picture is divided into n×n ( ) small patches, the M frame image will be divided into M×n×n patches of size A×A, so that each small patch can learn the feature information of the cracks in the same frame and the feature information of the cracks at the same position in different frames during training.

[0014] Furthermore, in S2, the segmented images are made into a data set, 80% of the images are used as a training set for training, and 20% of the images are used as a test set for testing the model.

[0015] Furthermore, the road crack feature extraction module in S3 has a total of 5 Encode blocks and 4 pooling layers. The crack image under the exposure background is used to extract crack features of different depths through 5 Encode blocks, and the feature map is downsampled through 4 pooling layers to compress the crack feature information. The specific process is: M×n×n pictures with a size of A×A with time continuity are input into the first Encode block, and the pictures are feature extracted through the Encode block, and the output is a feature map F1 with a size of A×A; then the feature map F1 is subjected to maximum pooling with a step size of 2, the crack feature information is compressed, and it is input into the second Encode block, and the output size is The feature map F2 is max-pooled to compress the crack feature information and input into the third Encode block. The output size is The feature map F3 is pooled to compress the crack feature information and input into the fourth Encode block. The output size is The feature map F4 is pooled to compress the crack feature information and input into the fifth Encode block. The output size is These feature maps have undergone different numbers of Encode blocks, so they contain crack information of different depths, which are used by the next road crack feature completion module and reconstruction module.

[0016] Furthermore, in the Encode block, the input feature map is first subjected to two 3*3 convolutions, one 1*1 convolution and one 3*3 convolution, to extract the features of the cracks, and then each layer is passed through the Relu activation layer and the batch normalization layer to update the feature maps of each layer, the new feature map is subjected to 1*1 convolution and batch normalization operation is performed to update the feature map, and then the updated feature map is added to the feature map output by the second 3*3 convolution to fuse the crack features and output a new feature map. Finally, the fused new feature map enters the CBAM attention module, which contains a channel attention module and a spatial attention module. The channel feature map output by the channel attention module is multiplied by the spatial feature map output by the spatial attention module to obtain the entire feature map F1 updated by the CBAM attention module; similarly, each time an Encode block is passed, the crack features will be extracted more deeply, and new feature maps, i.e., F2, F3, F4, and F5, will be obtained. The obtained feature maps will be used for the next feature fusion and completion module.

[0017] Furthermore, the feature fusion and completion module uses the upper-level coding fusion method to perform feature fusion on the feature map output by the road crack feature extraction module, enhance the detail information of the cracks, and add the Conv-Lstm module in the jump connection to complete the incomplete and missing cracks in the overexposed background; the specific process is: the feature map F1 with time continuity output by the road crack feature extraction is input into the Conv-Lstm module, in the Conv-Lstm module, the input gate determines how much crack information in the input feature map F1 is saved in the memory unit, the forget gate determines how much interference information should be forgotten, and the output gate determines the current memory. The memory state can be used to output the predicted information, so the feature map F6 output by the Conv-Lstm module can fuse the crack information of the previous and next frames, thereby predicting the missing or incomplete crack information caused by the overexposed background; then, the feature map F1 and the feature map F2 are added to enhance the detail information of the cracks, and the fused feature map is input into the Conv-Lstm module to obtain the feature map F7 with the predicted crack information; similarly, the feature maps F2 and F3 are fused, and the feature map F8 is output by the Conv-Lstm module, and the feature map F3 and F4 are fused, and the feature map F9 is output by the Conv-Lstm module.

[0018] Furthermore, the road crack reconstruction module has a total of 4 Up blocks and 4 Fuse blocks. The Up block can upsample the feature map output by the road crack feature extraction module to restore the size of the feature map to the previous size and enhance the details of the cracks. The Fuse block can fuse the feature map output by the road crack fusion and completion module with the feature map output by the Up block, and extract deeper crack information from the fused feature map to facilitate completion and segmentation of finer cracks. The specific process is: the size of the output of the road crack feature extraction module is The feature map F5 of is input to the Up block, and is upsampled by the Up block to obtain a size of The feature map F 10 ; The feature map F 10 It is fused with the feature map F9 output by the road crack completion module through the Fuse block and input into the Up block to obtain a size of The feature map F 11 ; The feature map F 11 It is fused with the feature map F8 output by the road crack completion module through the Fuse block and input into the Up block to obtain a size of The feature map F 12 ; The feature map F 12 The feature map F7 output by the road crack completion module is fused through the Fuse block and input into the Up block to obtain a feature map F with a size of A×A. 13; Finally, the feature map F 13 The feature map F6 output by the road crack completion module is fused through the Fuse block to obtain the final feature map F 14 , we get the final category of whether each pixel value in each crack image represents a crack or background.

[0019] Furthermore, in the Up block, the input feature map is first subjected to a 7*7 depthwise separable convolution, and the depthwise separable convolution updates the feature map through a depthwise convolution and a stationary point convolution; then the updated feature map is input into the batch-one layer, the linear layer, the Gelu layer, and the linear layer, and the feature maps output by these layers are fused with the feature map first input to update the feature map; then the feature map is input into the ConvTranspose2d layer for upsampling, so that the resolution of the feature map becomes larger, which is convenient for feature fusion and enhances the crack detail information; finally, the updated feature map is then input into the 7*7 depthwise separable convolution layer, the batch-one layer, the linear layer, the Gelu layer, and the linear layer, and the features are fused to output a feature map with increased resolution and more crack details;

[0020] In the Fuse block, the feature map F output by the Up module is first 10 The feature map F9 output by the feature fusion and completion module is spliced ​​through the Cat operator, and a 1*1 convolution is performed to update the feature map; then the feature map is split into two features through the Spilt operator Figure X 1. X2, the feature Figure X 2 is input into InvertResidualBlock, and its output feature map is compared with the feature Figure X 2 performs feature fusion; finally, the updated feature map is input into InvertedResidualBlock and combined with the feature Figure X 1. Perform feature fusion to obtain a new feature map. Similarly, the feature maps output by each layer of the Up module and the feature maps output by the feature fusion and completion module must pass through the Fuse module. In the InvertedResidualBlock, the input feature map is first input into the 1*1 convolution layer to expand the number of channels, thereby increasing the dimension of the feature map. Then, it passes through the normalization layer, the activation function layer, and the ReflectionPad2d two-dimensional reflection filling layer to fill the feature map in the network, protect the edge information of the crack image, and obtain a new feature map. Then, the new feature map is input into the 3*3 depth-separable convolution layer, and passes through the normalization layer and the activation function layer. Finally, a 1*1 point-by-point convolution is used to project the number of channels of the feature map to the original number of channels. In the Fuse module, the amount of parameters and the amount of calculation are greatly reduced, so that the cracks can be better segmented and reconstructed at a lower computational cost.

[0021] Furthermore, in S4, the network model of the road crack detection system based on the overexposed background is trained, the test set data is predicted using the trained model, and the prediction result images are spliced ​​and combined. First, a blank image with the same size as the original image is created to store the prediction result images, and then the pixel values ​​of each prediction result image are copied to the blank image to ensure that the position and size of each prediction result are consistent when combined. Finally, the combined road crack results are saved, and the detection results are saved.

[0022] (III) Beneficial effects

[0023] Compared with the prior art, the present invention provides a method for detecting road cracks under an overexposed background of an inspection vehicle, which has the following beneficial effects:

[0024] The present invention divides the collected images into two-dimensional time and space, constructs the time and space association between cracks, and constructs a road crack feature extraction module, a feature fusion and completion module, and a feature reconstruction module. The three modules are combined using a U-Net network to construct a road crack detection system network model based on an overexposed background, so that the model can better process cracks in the overexposed background, and can combine the context information of the image to complete the discontinuities and cavities that appear in the cracks in the overexposed background. Therefore, it is possible to avoid the discontinuities and cavities in the crack detection results when detecting the actual road images under the overexposed background collected by the inspection vehicle, thereby ensuring the integrity of the crack detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a schematic diagram of the process of the present invention;

[0026] Figure 2 Schematic diagram of collecting pictures for the inspection vehicle of the present invention

[0027] Figure 3 This is a schematic diagram of M consecutive crack images of the present invention;

[0028] Figure 4 This is a crack image segmentation effect diagram of the present invention;

[0029] Figure 5 This is a schematic diagram of a road crack feature extraction module of the present invention;

[0030] Figure 6 It is a schematic diagram of the structure of the Encode block of the present invention;

[0031] Figure 7 This is a schematic diagram of the feature fusion and completion module of the present invention;

[0032] Figure 8 This is a schematic diagram of a road crack reconstruction module of the present invention;

[0033] Fig. 9 This is a schematic diagram of the Up block structure of the present invention;

[0034] Fig.10 This is a schematic diagram of the structure of the Fuse block of the present invention;

[0035] Fig.11 A schematic diagram of a network model of a road crack detection system according to the present invention;

[0036] Fig.12 It is a schematic diagram of the test results of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Example

[0039] like Figure 1-11 As shown, a method for detecting road cracks under an overexposed background of an inspection vehicle according to an embodiment of the present invention includes the following steps:

[0040] S1: Use inspection vehicles to collect road crack images with overexposed backgrounds;

[0041] like Figure 2 As shown, an industrial camera is installed on the top of the inspection vehicle. The camera head of the industrial camera should be at a certain angle to the road surface. It is required to be able to take clear pictures of the road surface and not take too many interference factors on the roadside. The data collected by the industrial camera is uploaded to the computer. The camera model is the Hikvision MV-CU060-UM industrial camera. The pixel of this camera is 6 million. It can take pictures with high road surface image quality and meet the requirements of the detection algorithm. The maximum frame rate reaches 59.6FPS. It can collect continuous cracks to ensure the integrity of crack collection. It has a USB interface for easy connection with the computer. The collected road surface images are transmitted to the computer, and the software system completes the road surface data storage, processing, analysis and other tasks.

[0042] S2: Segment the collected images into two-dimensional space-time, construct the space-time correlation between the cracks, and create a data set, which is divided into a training set and a test set;

[0043] The specific contents of the temporal and spatial correlation between structural cracks are as follows: First, in order to construct the temporal continuity of the structural cracks, the crack image at time t1 is used as the reference in the video collected by the inspection vehicle, and the image is recorded every t seconds.Δ Select a picture, a total of M frames, M is an odd number, such as Figure 3 As shown. Secondly, in order to construct the continuity of the crack in space, the selected M frames of images with a size of B×B are slid along the horizontal direction of the crack image using a sliding window of size A×A, where A is the factor of B and the sliding step is A. Then a crack image is divided into n×n ( ) small patch blocks, the M frame image will be divided into M×n×n patch blocks of size A×A, such as Figure 4 As shown. Each small patch block can learn the feature information of the cracks in the same frame and the feature information of the cracks in the same position in different frames during training; it should be further explained here that the segmented pictures are made into a data set, 80% of the pictures are used as training sets for training, and 20% of the pictures are used as test sets for testing the model.

[0044] S3: Construct a road crack feature extraction module, a feature fusion and completion module, and a feature reconstruction module, and combine the three modules using the U-Net network to build a road crack detection system network model based on overexposure background, and complete the cracks that are interrupted and incomplete due to overexposure;

[0045] The road crack feature extraction module has a total of 5 Encode blocks and 4 pooling layers. The crack image under the exposure background is extracted through 5 Encode blocks to extract crack features of different depths. The feature map is downsampled through 4 pooling layers to compress the crack feature information and reduce the calculation complexity, so as to better fuse the features of different layers of crack information and complete the missing cracks.

[0046] Road crack feature extraction module Figure 5 As shown in Figure 1, M×n×n images of size A×A with temporal continuity are input into the first Encode block, and features are extracted from the images through the Encode block, and the output is a feature map F1 of size A×A; then the feature map F1 is subjected to maximum pooling with a step size of 2, to compress the crack feature information, and is input into the second Encode block, and the output size is The feature map F2 is max-pooled to compress the crack feature information and input into the third Encode block. The output size is The feature map F3 is pooled to compress the crack feature information and input into the fourth Encode block. The output size is The feature map F4 is pooled to compress the crack feature information and input into the fifth Encode block. The output size is These feature maps have undergone different numbers of Encode blocks, so they contain crack information of different depths, which are used by the next step of road crack feature completion module and reconstruction module.

[0047] The Encode block structure is as follows Figure 6 As shown in the figure, in the Encode block, the input feature map is first subjected to two 3*3 convolutions, one 1*1 convolution and one 3*3 convolution, to extract the crack features. Then, each layer is updated through the Relu activation layer and the batch normalization layer. The new feature map is subjected to 1*1 convolution and batch normalization operation to update the feature map. Then, the updated feature map is added to the feature map output by the second 3*3 convolution to fuse the crack features and output a new feature map. Finally, the fused new feature map enters the CBAM attention module, which contains a channel attention module and a spatial attention module. The channel feature map output by the channel attention module is multiplied by the spatial feature map output by the spatial attention module to obtain the entire feature map F1 updated by the CBAM attention module. Similarly, each time an Encode block is passed, the crack features will be extracted more deeply, and new feature maps will be obtained, namely F2, F3, F4, and F5. The obtained feature maps will be used for the next feature fusion and completion modules.

[0048] In the feature fusion and completion module, the feature map output by the road crack feature extraction module is fused using the upper-level encoding fusion method to enhance the detail information of the cracks, and the Conv-Lstm module is added in the jump connection to complete the incomplete and missing cracks in the overexposed background. Figure 7 As shown in the figure, in the feature fusion and completion module, the feature map F1 with time continuity output by the road crack feature extraction is input into the Conv-Lstm module. In the Conv-Lstm module, the input gate determines how much crack information in the input feature map F1 is saved in the memory unit, the forget gate determines how much interference information should be forgotten, and the output gate determines how much prediction information will be output in the current memory state. Therefore, the feature map F6 output by the Conv-Lstm module can fuse the crack information of the previous and next frames, thereby predicting the missing or incomplete crack information caused by the overexposure background. Then, the feature map F1 and the feature map F2 are added to enhance the detail information of the cracks, and the fused feature map is input into the Conv-Lstm module to obtain the feature map F7 with predicted crack information. Similarly, the feature maps F2 and F3 are fused and output by the Conv-Lstm module to obtain the feature map F8, and the feature maps F3 and F4 are fused and output by the Conv-Lstm module to obtain the feature map F9.

[0049] There are 4 Up blocks and 4 Fuse blocks in the road crack reconstruction module. The Up block can upsample the feature map output by the road crack feature extraction module to restore the feature map to its previous size and enhance the details of the cracks. The Fuse block can fuse the feature map output by the road crack fusion and completion module with the feature map output by the Up block, and extract deeper crack information from the fused feature map to facilitate completion and segmentation of finer cracks.

[0050] Road crack reconstruction module Figure 8 As shown, the size of the road crack feature extraction module output is The feature map F5 of is input to the Up block, and is upsampled by the Up block to obtain a size of The feature map F 10 ; The feature map F 10 It is fused with the feature map F9 output by the road crack completion module through the Fuse block and input into the Up block to obtain a size of The feature map F 11 ; The feature map F 11 It is fused with the feature map F8 output by the road crack completion module through the Fuse block and input into the Up block to obtain a size of The feature map F 12 ; The feature map F 12 The feature map F7 output by the road crack completion module is fused through the Fuse block and input into the Up block to obtain a feature map F with a size of A×A. 13 ; Finally, the feature map F 13 The feature map F6 output by the road crack completion module is fused through the Fuse block to obtain the final feature map F 14 , we get the final category of whether each pixel value in each crack image represents a crack or background.

[0051] Among them, an Up block such as Fig. 9 As shown in the figure, in the Up block, the input feature map is first subjected to a 7*7 depthwise separable convolution, and the depthwise separable convolution updates the feature map through a depthwise convolution and a stationary point convolution; then the updated feature map is input into the batch-one layer, the linear layer, the Gelu layer, and the linear layer, and the feature maps output by these layers are fused with the feature map input first to update the feature map; then the feature map is input into the ConvTranspose2d layer for upsampling to increase the resolution of the feature map, facilitate feature fusion, and enhance the crack detail information; finally, the updated feature map is then input into the 7*7 depthwise separable convolution layer, the batch-one layer, the linear layer, the Gelu layer, and the linear layer, and the features are fused to output a feature map with increased resolution and more crack details.

[0052] Among them, a Fuse block such as Fig.10 As shown, in the Fuse block, the feature map F output by the Up module is first 10 The feature map F9 output by the feature fusion and completion module is spliced ​​through the Cat operator, and a 1*1 convolution is performed to update the feature map; then the feature map is split into two features through the Spilt operator Figure X 1. X2, the feature Figure X 2 is input into InvertResidualBlock, and its output feature map is compared with the feature Figure X 1 performs feature fusion; finally, the updated feature map is input into InvertedResidualBlock and combined with the feature Figure X 3. Perform feature fusion to obtain a new feature map. Similarly, the feature maps output by each layer of the Up module and the feature maps output by the feature fusion and completion module must pass through the Fuse module. In InvertedResidualBlock, the input feature map is first input into the 1*1 convolution layer to expand the number of channels, increasing the dimension of the feature map; then it passes through the normalization layer, activation function layer, and ReflectionPad2d two-dimensional reflection filling layer to fill the feature map in the network, protect the edge information of the crack image, and obtain a new feature map; then the new feature map is input into the 3*3 depth-separable convolution layer, and passes through the normalization layer and activation function layer; finally, a 1*1 point-by-point convolution is used to project the number of channels of the feature map to the original number of channels. In the Fuse module, the number of parameters and calculations are greatly reduced, and the cracks can be better segmented and reconstructed at a lower computational cost.

[0053] S4: Based on the data set, the network model of the road crack detection system is trained, and the trained model is used to detect road crack images under overexposed backgrounds, and the detection results are saved;

[0054] The specific content is: train the network model of the road crack detection system based on overexposed background, use the trained model to predict the test set data, and splice and combine the prediction result pictures. First, create a blank image of the same size as the original image to store the prediction result picture, and then copy the pixel value of each prediction result picture to the blank image to ensure that each prediction result is consistent in position and size when combined, and finally save the combined road crack results.

[0055] In addition, the following indicators can be used to evaluate the improved model of the present invention, F1-score, Recall (R), mean Intersection over Union (mIoU) and Precision (P). TP is a true positive example, FP is a false positive example, TN is a true negative example, and FN is a false negative example. The calculation indicators are as follows:

[0056]

[0057]

[0058]

[0059]

[0060] In order to verify the feasibility of the present invention, photos taken by a patrol car on October 13, 2023 on the road section from Taojia to Liuyang in Gongzhuling City, Jilin Province were selected, and the photos were made into a data set to test the performance of the model. The test results are as follows:

[0061]

[0062] The above table shows the prediction performance of the classic semantic segmentation network models U-Net, SegNet and the model of the present invention on the data set. By comparison, the model proposed by the present invention performs better in terms of precision, recall, F1 score and MioU indicators.

[0063] Fig.12 This is the test result of the model of the present invention on the data set. It can be seen that the network model of the road crack detection system based on the overexposure background can solve the problems of discontinuity and cavity in the process of road crack detection under the overexposure background.

[0064] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting road cracks under overexposure background of a patrol vehicle, characterized by: The steps include: S1: Use inspection vehicles to collect road crack images with overexposed backgrounds; S2: Segment the collected images into two-dimensional space-time, construct the space-time correlation between the cracks, and create a data set, which is divided into a training set and a test set; S3: Construct a road crack feature extraction module, a feature fusion and completion module, and a feature reconstruction module, and combine the three modules using the U-Net network to build a road crack detection system network model based on overexposure background, and complete the cracks that are interrupted and incomplete due to overexposure; S4: Based on the data set, the network model of the road crack detection system is trained, and the trained model is used to detect road crack images under overexposed backgrounds, and the detection results are saved.

2. A method for detecting road cracks under overexposure background for a patrol vehicle according to claim 1, characterized in that: An industrial camera is installed on the top of the inspection vehicle in S1. The camera head of the industrial camera should be at a certain angle to the road surface. It is required to be able to clearly capture the road surface and not capture too many roadside interference factors. The data collected by the industrial camera is uploaded to the computer, and the software system completes the road surface data storage, processing, analysis and other tasks.

3. A method for detecting road cracks under overexposure background for a patrol vehicle according to claim 1, characterized in that: The specific content of constructing the temporal and spatial association between the cracks in S2 is as follows: first, in order to construct the temporal continuity of the cracks, the crack image at time t1 is used as the reference in the video collected by the inspection vehicle, and the image is taken every t before and after. Δ Select a picture with M frames, where M is an odd number. In order to construct the continuity of the crack in space, select the M frames with a size of B×B, and slide the sliding window with a size of A×A along the horizontal direction of the crack picture. A is the factor of B, and the sliding step is A. Then a crack picture is divided into n×n The M-frame image will be divided into M×n×n patches of size A×A, so that each small patch can learn the feature information of the cracks in the same frame and the feature information of the cracks at the same position in different frames during training.

4. A method for detecting road cracks under overexposure background for a patrol vehicle according to claim 1, characterized in that: In S2, the segmented images are made into a data set, 80% of the images are used as a training set for training, and 20% of the images are used as a test set for testing the model.

5. The method for detecting road cracks under overexposure background of a patrol vehicle according to claim 3, characterized in that: The road crack feature extraction module in S3 has a total of 5 Encode blocks and 4 pooling layers. The crack image under the exposure background is extracted through 5 Encode blocks to extract crack features of different depths, and the feature map is downsampled through 4 pooling layers to compress the crack feature information; The specific process is as follows: M×n×n images of size A×A with temporal continuity are input into the first Encode block, and features are extracted from the images through the Encode block, and the output is a feature map F1 of size A×A; then the feature map F1 is subjected to maximum pooling with a step size of 2 to compress the crack feature information, and is input into the second Encode block, and the output size is The feature map F2 is max-pooled to compress the crack feature information and input into the third Encode block. The output size is The feature map F3 is pooled to compress the crack feature information and input into the fourth Encode block. The output size is The feature map F4 is pooled to compress the crack feature information and input into the fifth Encode block. The output size is These feature maps have undergone different numbers of Encode blocks, so they contain crack information of different depths, which are used by the next road crack feature completion module and reconstruction module.

6. A method for detecting road cracks under overexposure background for a patrol vehicle according to claim 5, characterized in that: In the Encode block, the input feature map is first subjected to two 3*3 convolutions, one 1*1 convolution and one 3*3 convolution, to extract the features of the cracks. Then, each layer is passed through the Relu activation layer and the batch normalization layer to update the feature maps of each layer. The new feature map is subjected to 1*1 convolution and batch normalization operation is performed to update the feature map. Then, the updated feature map is added to the feature map output by the second 3*3 convolution to fuse the crack features and output a new feature map. Finally, the fused new feature map enters the CBAM attention module, which contains a channel attention module and a spatial attention module. The channel feature map output by the channel attention module is multiplied by the spatial feature map output by the spatial attention module to obtain the entire feature map F1 updated by the CBAM attention module. Similarly, each time an Encode block is passed, the crack features will be extracted more deeply, and new feature maps, namely F2, F3, F4, and F5, will be obtained. The obtained feature maps will be used for the next feature fusion and completion modules.

7. A method for detecting road cracks under overexposure background for a patrol vehicle according to claim 6, characterized in that: The feature fusion and completion module uses the upper-level coding fusion method to perform feature fusion on the feature map output by the road crack feature extraction module, enhance the detail information of the cracks, and add the Conv-Lstm module in the jump connection to complete the incomplete and missing cracks in the overexposed background. The specific process is as follows: the feature map F1 with time continuity output by the road crack feature extraction is input into the Conv-Lstm module. In the Conv-Lstm module, the input gate determines how much crack information in the input feature map F1 is saved in the memory unit, the forget gate determines how much interference information should be forgotten, and the output gate determines the current memory state. Therefore, the feature map F6 output by the Conv-Lstm module can fuse the crack information of the previous and next frames, thereby predicting the missing or incomplete crack information caused by the overexposed background; then, the feature map F1 and the feature map F2 are added to enhance the detail information of the cracks, and the fused feature map is input into the Conv-Lstm module to obtain the feature map F7 with the predicted crack information; similarly, the feature maps F2 and F3 are fused and output through the Conv-Lstm module to obtain the feature map F8, and the feature maps F3 and F4 are fused and output through the Conv-Lstm module to obtain the feature map F9.

8. A method for detecting road cracks under overexposure background for a patrol vehicle according to claim 7, characterized in that: The road crack reconstruction module has a total of 4 Up blocks and 4 Fuse blocks. The Up blocks can upsample the feature map output by the road crack feature extraction module to restore the size of the feature map to the previous size and enhance the details of the cracks. The Fuse block can fuse the feature map output by the road crack fusion and completion module with the feature map output by the Up block, and extract deeper crack information from the fused feature map to facilitate completion and segmentation of finer cracks; The specific process is: the size of the road crack feature extraction module output is The feature map F5 of is input to the Up block, and is upsampled by the Up block to obtain a size of The feature map F 10 ; The feature map F 10 It is fused with the feature map F9 output by the road crack completion module through the Fuse block and input into the Up block to obtain a size of The feature map F 11 ; The feature map F 11 It is fused with the feature map F8 output by the road crack completion module through the Fuse block and input into the Up block to obtain a size of The feature map F 12 ; The feature map F 12 The feature map F7 output by the road crack completion module is fused through the Fuse block and input into the Up block to obtain a feature map F with a size of A×A. 13 ; Finally, the feature map F 13 It is fused with the feature map F6 output by the road crack completion module through the Fuse block to obtain the final feature map F 14 , we get the final category of whether each pixel value in each crack image represents a crack or background.

9. A method for detecting road cracks under overexposure background for a patrol vehicle according to claim 8, characterized in that: In the Up block, first, the input feature map is subjected to a 7*7 depthwise separable convolution, and the depthwise separable convolution updates the feature map through a depthwise convolution and a stationary point convolution; then the updated feature map is input into the batch-one layer, the linear layer, the Gelu layer, and the linear layer, and the feature maps output by these layers are fused with the feature map first input to update the feature map; then the feature map is input into the ConvTranspose2d layer for upsampling, so that the resolution of the feature map becomes larger, which is convenient for feature fusion and enhances the crack detail information; finally, the updated feature map is then input into the 7*7 depthwise separable convolution layer, the batch-one layer, the linear layer, the Gelu layer, and the linear layer, and the features are fused to output a feature map with increased resolution and more crack details; In the Fuse block, the feature map F output by the Up module is first 10 The feature map F9 output by the feature fusion and completion module is spliced ​​through the Cat operator, and a 1*1 convolution is performed to update the feature map; then the feature map is split into two feature maps X1 and X2 through the Spilt operator, and the feature map X2 is input into the InvertResidualBlock, and the feature map output by it is fused with the feature map X2; finally, the updated feature map is input into the InvertedResidualBlock, and the feature map is fused with the feature map X1 to obtain a new feature map. Similarly, the feature map output by each layer of the Up module and the feature map output by the feature fusion and completion module are All the images must pass through the Fuse module; in InvertedResidualBlock, the input feature map is first input into a 1*1 convolution layer to expand the number of channels and increase the dimension of the feature map; then it passes through the normalization layer, the activation function layer, and the ReflectionPad2d two-dimensional reflection filling layer to fill the feature map in the network, protect the edge information of the crack image, and obtain a new feature map; then the new feature map is input into a 3*3 depth-separable convolution layer, and passes through the normalization layer and the activation function layer; finally, a 1*1 point-by-point convolution is used to project the number of channels of the feature map to the original number of channels.

10. The method for detecting road cracks under overexposure background of a patrol vehicle according to claim 1, characterized in that: In the S4, the network model of the road crack detection system based on the overexposed background is trained, the test set data is predicted using the trained model, and the prediction result images are spliced ​​and combined. First, a blank image with the same size as the original image is created to store the prediction result images, and then the pixel values ​​of each prediction result image are copied to the blank image to ensure that the position and size of each prediction result are consistent when combined. Finally, the combined road crack results are saved, and the detection results are saved.

Images