A lightweight detection method, device, medium and equipment for road surface defects

Through drone video acquisition and improved Conv-LSTM network processing, the stability problem of road surface disease detection is solved, efficient and low-cost multi-lane detection is achieved, and energy consumption and carbon emissions are reduced.

CN120032282BActive Publication Date: 2025-08-26SHAANXI ZHONGHUIHE AMMONIA HYDROGEN TECHNOLOGY ENGINEERING CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510145227.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-02-10
Publication Date
2025-08-26
Estimated Expiration
2045-02-10

AI Technical Summary

Technical Problem

In the prior art, road surface disease detection is susceptible to road surface materials and external lighting environment, resulting in insufficient detection stability. In addition, vehicle-mounted detection methods require repeated driving to cover multiple lanes, high energy consumption and large carbon emissions.

Method used

UAV video acquisition data is used, image sequence processing is performed through keyframe extraction and improved Conv-LSTM network, and combined with GPS path data, a road table disease detection model is built to realize automated detection.

Benefits of technology

It improves the stability and accuracy of road surface disease detection, reduces energy consumption and carbon emissions, is suitable for multi-lane detection, has high automation and low cost.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120032282B_ABST
    Figure CN120032282B_ABST
Patent Text Reader

Abstract

The present invention discloses a lightweight detection method, device, medium and equipment for road surface defects, which relate to the field of road detection. The method comprises the following steps: obtaining drone video data and GPS path data of a road surface to be detected; performing key frame extraction on the drone video data using a key frame extraction algorithm to determine a road surface image sequence data set; constructing an improved convolutional long short-term memory (Conv‑LSTM) network; training the improved convolutional long short-term memory (Conv‑LSTM) network to obtain a road surface defect detection model for detecting road surface defects; inputting the road surface image sequence data set to be detected into the road surface defect detection model to obtain a road surface defect detection result; and obtaining a road surface defect detection report based on the road surface defect detection result and in combination with the GPS path data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of road detection, and in particular to a lightweight detection method, device, medium and equipment for road surface defects. Background Art

[0002] Road surface defects are a series of surface damages that appear in the early stages of road use, including cracks, patches, and potholes. Measuring the number and geometric parameters of road surface defects is a key indicator for evaluating pavement performance and has a significant impact on road safety. Because road surface defects are somewhat random in time and space, detection typically involves regular inspections, recording the locations of defects, and then conducting specific tests in areas where defects are concentrated. This method is susceptible to subjective influence from testing personnel, resulting in uncertainty in detection stability.

[0003] Currently, there are two main methods for detecting road surface defects: two-dimensional image detection using industrial high-definition cameras and three-dimensional image detection using three-dimensional lasers. Industrial high-definition cameras are installed on road inspection vehicles and collect data as the vehicle moves, capturing two-dimensional images of the road surface within a lane and recording the corresponding location information. The two-dimensional image data is then processed using visual processing algorithms or deep neural networks to determine the specific location and extent of the defect. However, these methods are susceptible to the influence of road surface materials and external lighting conditions, resulting in limited stability in the results of road surface defect detection. Summary of the Invention

[0004] The present invention provides a lightweight detection method, device, medium and equipment for road surface defects, which are used to solve the above-mentioned problem existing in the prior art, namely, how to improve the stability of road surface defect detection in the prior art. The present invention provides a lightweight detection method for road surface defects, which includes:

[0005] The road surface to be inspected is segmented and the drone video collection nodes are determined. When the inspection begins, the drone flies in an uplink and downlink manner and starts GPS positioning to obtain drone video data and GPS path data of the road surface to be inspected.

[0006] The key frame extraction algorithm is used to extract key frames from the UAV video data to determine the road surface image sequence dataset;

[0007] An improved convolutional long short-term memory (Conv-LSTM) network is constructed; the encoder of the original Conv-LSTM network is replaced with a local feature extraction module, and the upsampling in the decoder of the original Conv-LSTM network is replaced with a bilinear interpolation method; the local feature extraction module is composed of a convolutional layer, a global self-attention mechanism layer Global-SA, a multi-layer perceptron (MLP), a local self-attention mechanism layer Local-SA, and an MLP connected in sequence;

[0008] Train the improved convolutional long short-term memory (Conv-LSTM) network to obtain a road surface defect detection model for detecting road surface defects.

[0009] The road surface image sequence dataset to be detected is input into the road surface disease detection model to obtain the results of the road surface disease detection. According to the results of the road surface disease detection and combined with the GPS path data, a road surface disease detection report is obtained.

[0010] Optionally, extracting key frames from the drone video data using a key frame extraction algorithm to determine a road surface image sequence dataset specifically includes:

[0011] Key frame extraction is performed on the drone video data using a key frame extraction algorithm to obtain image data. An image dataset is constructed based on the image data, and the image dataset is divided into several subsets. The several subsets constitute the road table image sequence dataset.

[0012] Optionally, the key frame extraction algorithm is used to extract key frames from the drone video data to obtain image data, and an image dataset is constructed based on the image data. The image dataset is divided into a plurality of subsets, and the plurality of subsets constitute the road table image sequence dataset, specifically including:

[0013] By using the key frame extraction algorithm at a frame rate of 10FPS, key frames are extracted from the video to obtain image data, and the image dataset is obtained using the following formula:

[0014] ,

[0015] in, is an image dataset, is the image data, is the timestamp corresponding to the image data, is the pixel position, is the pixel position Corresponding RGB values;

[0016] The image dataset It is further divided into several subsets, each subset is an image data sequence, containing 10 ordered image data. The specific calculation formula is as follows:

[0017] ,

[0018] ,

[0019] in, is a subset, is the total number of subsets, is the mth image in the 10 image data i, and n is the sequence number of the subset;

[0020] A plurality of image data sequences constitute the road table image sequence data set.

[0021] Optionally, the road surface defect detection report is obtained based on the results of the road surface defect detection in combination with GPS path data, specifically including:

[0022] According to the results of road surface damage detection, the timestamp of the surface damage location is obtained using the following formula:

[0023] ,

[0024] in, is the timestamp of the surface disease location, n is the total number of subsets in the road surface image sequence dataset, and m is the mth image of the subset in the road surface image sequence dataset;

[0025] Based on the timestamp of the surface defect location and combined with GPS path data, the defect location is determined and a road surface defect detection report is obtained.

[0026] Optionally, before extracting key frames from the drone video data using the key frame extraction algorithm, the road section range of the detection position in the drone video data is marked according to the GPS path data, and the drone video data is numbered, and each numbered road section is treated as an independent data set.

[0027] Optionally, abnormal data removal processing is performed on the image data sequence dataset.

[0028] The present invention provides a lightweight road surface disease detection device, comprising:

[0029] The acquisition module is used to segment the road surface to be inspected and determine the drone video collection nodes. When the inspection begins, the drone flies in an uplink and downlink manner and starts GPS positioning to obtain drone video data and GPS path data of the road surface to be inspected;

[0030] The key frame extraction module is used to extract key frames from the UAV video data using a key frame extraction algorithm to determine the road surface image sequence dataset;

[0031] A construction module for constructing an improved convolutional long short-term memory (Conv-LSTM) network; replacing the encoder of the original Conv-LSTM network with a local feature extraction module, and replacing the upsampling in the decoder of the original Conv-LSTM network with a bilinear interpolation method; the local feature extraction module is composed of a convolutional layer, a global self-attention mechanism layer (Global-SA), a multi-layer perceptron (MLP), a local self-attention mechanism layer (Local-SA), and an MLP connected in sequence;

[0032] A training module is used to train the improved convolutional long short-term memory (Conv-LSTM) network to obtain a road surface defect detection model for detecting road surface defects;

[0033] The detection module is used to input the road surface image sequence data set to be detected into the road surface disease detection model, obtain the results of the road surface disease detection, and obtain a road surface disease detection report based on the results of the road surface disease detection combined with the GPS path data.

[0034] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned lightweight detection method for road surface diseases is implemented.

[0035] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned lightweight detection method for road surface defects is implemented.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention provides a lightweight detection method for road surface defects, which obtains a continuous image sequence by extracting key frames from the acquired drone video data. When the lighting environment changes, mainly because there are shadows on the road, it will cause serious information loss problems for some viewing angles. However, the continuous image sequence makes it possible to obtain image data from different angles, and through the construction of an improved Conv-LSTM network to accurately extract the correlation features of adjacent images, accurate detection can be achieved, and lightweight detection of roads at all levels can be achieved, thereby improving the stability of road surface defect detection; compared with vehicle-mounted industrial cameras and three-dimensional lasers, the method proposed by the present invention is applicable to more engineering situations and has high utilization rate And the cost is low; then, the method has a high degree of automation, and in the process of data processing, computer automation processing of the entire process can be realized, which effectively saves time cost and improves the accuracy of detection; this method avoids the problem that the detection vehicle needs to repeatedly drive on a section of road multiple times to cover multiple lanes during the detection process of the intelligent detection vehicle, especially in the multi-lane detection process, and effectively reduces the amount of energy consumption in the detection process without affecting the existing traffic operation, and reduces the carbon emissions of the road detection project; in addition, in the collection stage, through reasonable road segmentation, node selection and the drone collecting video data by uplink and downlink round-trip flights, comprehensive coverage of multiple lanes is achieved, which provides a solid foundation for subsequent road surface detection and analysis work. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0038] Figure 1 A flowchart of a lightweight detection method for road surface defects provided by an embodiment of the present invention;

[0039] Figure 2 A flowchart for obtaining road surface defects provided by an embodiment of the present invention;

[0040] Figure 3 Flowchart of the improved Conv-LSTM network provided by an embodiment of the present invention;

[0041] Figure 4 Flowchart of the encoding module of the improved Conv-LSTM network provided by an embodiment of the present invention;

[0042] Figure 5 A schematic diagram of a computer device for the lightweight road surface defect detection method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0044] The following describes in detail the technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems using specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following embodiments of the present invention are described in conjunction with the accompanying drawings.

[0045] like Figure 1 and Figure 2 As shown, this embodiment shows a lightweight detection method for road surface defects, including:

[0046] S1: The road surface to be inspected is segmented and the drone video collection nodes are determined. When the inspection begins, the drone flies in an uplink and downlink manner and starts GPS positioning to obtain drone video data and GPS path data of the road surface to be inspected.

[0047] For example, before obtaining drone video data of the road surface to be inspected, the road surface to be inspected is segmented; when the inspection begins, the inspection route is tracked and located by starting GPS positioning, and the inspection position is calibrated.

[0048] For example, the preparation of the drone equipment can be carried out in an open area near the pre-inspection section, including assembling drone accessories and replacing the power supply and data storage card; then the drone is started, connected to the tablet, and the wireless connection between the drone and the tablet is debugged. After flying to a predetermined height, the camera's focal length is calibrated and fixed; after adjusting the picture to be clear, enter the pre-inspection section for a pre-flight, the pre-flight distance is approximately 1 kilometer, and the flight speed is adjusted between 1m / s and 5m / s. During the flight, the flight path setting and flight altitude are adjusted to ensure that the pre-inspection area appears completely within the lens, and the flight is carried out as much as possible along the center line of the road; return after completing all calibration and calibration work.

[0049] For example, before the formal data collection, the road section to be inspected can be segmented. Generally, the single data collection distance does not exceed 10 km. The node position of each segment should be selected in an open area suitable for take-off and landing. The ground operator can monitor the data collection process of the drone. The data collection is collected by uplink and downlink round-trip flight. The data collection can cover multiple lanes during the flight in each direction. The coverage range is 0 to 10 meters, and the number of covered lanes should not exceed three. After completing the inspection of a section of road, it can move to the next node position and repeat the operation to continue the inspection of the next section of road.

[0050] At the beginning of each inspection, the drone is started at a pre-selected node position and enters the inspection section. It can start recording video in 4K mode with a resolution of 4096×2160. The flight speed and altitude are kept unchanged during the process. GPS positioning is started to track and locate the inspection route and calibrate the inspection position. The drone can transmit the inspection image to the tablet computer for monitoring. The collected video data of the inspection road surface is stored in the data card. After the uplink is completed, the video collection is stopped, and the direction is turned to enter the downlink, and then the video collection is restarted.

[0051] After the drone completes the collection and returns, remove the data card, upload the collected video MOV file to the computer, and note the road section range; number the collected video data in sequence, for example, the uplink number can be S1, S2, S3,..., and the downlink number can be X1, X2, X3,..., each numbered section is an independent data set, and then export the GPS path data file, name it accordingly and annotate it, and store it in the computer.

[0052] In this step, the use of drones to collect road surface inspection video data not only improves collection efficiency but also ensures the accuracy and completeness of the data. At the same time, through meticulous debugging and pre-flight verification, the stability and accuracy of the drone are ensured. In addition, during the collection phase, through reasonable road segmentation, node selection, and collection methods, comprehensive coverage of multiple lanes is achieved, providing a solid foundation for subsequent road surface inspection and analysis.

[0053] S2: Extract key frames from the drone video data using a key frame extraction algorithm to determine the road surface image sequence dataset.

[0054] Optionally, performing key frame extraction on the drone video data using a key frame extraction algorithm to obtain image data, and constructing an image dataset based on the image data; dividing the image dataset into a plurality of subsets, wherein the plurality of subsets constitute the road surface image sequence dataset, includes the following steps:

[0055] By using the key frame extraction algorithm at a frame rate of 10FPS, key frames are extracted from the video to obtain image data, and the image dataset is obtained using the following formula:

[0056] ;

[0057] in, is an image dataset, is the image data, is the timestamp corresponding to the image data, is the pixel position, is the pixel position Corresponding RGB values;

[0058] The image dataset It is further divided into several subsets, each subset is an image data sequence, containing 10 ordered image data. The specific calculation formula is as follows:

[0059] ;

[0060] ;

[0061] in, is a subset, is the total number of subsets, is the mth image in the 10 image data i, and n is the sequence number of the subset;

[0062] A plurality of image data sequences constitute the road table image sequence data set.

[0063] Exemplarily, the image data sequence data set is preprocessed through image processing algorithms and statistical algorithms to eliminate abnormal data, including converting the RGB values ​​of the image into grayscale values ​​and calculating the average and standard deviation of the grayscale values ​​at each pixel position of each image data; based on the average and standard deviation of each image data in each image data sequence, a consistency check is performed to eliminate image data that does not meet the consistency requirements; for image sequences with problematic data, the timestamp starting point is adjusted, and the operation key frame selection is restarted, and the processing is re-performed to supplement the sequence.

[0064] For example, all images are processed by LabelMe software, and each road surface disease appearing in the image is marked with a box label, and then saved as a JSON data file. Each processed set of data samples includes image data and corresponding JSON data; based on the previous image data sequence, the processed images and corresponding JSON files are merged into a sequence of 10, and output as two NPY files: image sequence and mask sequence; in order to reduce the computational cost and time cost of network training, the image can be segmented into , a total of 32 regions, each region size is ; Then, the following formula is used to normalize the data in all datasets:

[0065] ;

[0066] in, , It is the mean and standard deviation of the pixel values ​​at all locations in each channel of the RGB image.

[0067] S3: Construct an improved convolutional long short-term memory (Conv-LSTM) network; replace the encoder of the original Conv-LSTM network with a local feature extraction module, and replace the upsampling in the decoder of the original Conv-LSTM network with a bilinear interpolation method; the local feature extraction module consists of a convolutional layer, a global self-attention mechanism layer (Global-SA), a multi-layer perceptron (MLP), a local self-attention mechanism layer (Local-SA), and an MLP connected in sequence.

[0068] like Figure 3 As shown in Figure 1, the improved convolutional long short-term memory Conv-LSTM network includes an encoding module, a decoding module, and a fully connected layer.

[0069] like Figure 4 As shown in the figure, the encoding module first inputs a convolutional layer Conv-Layer, then inputs the global self-attention mechanism layer Global-SA connected to a multi-layer perceptron MLP, and the local self-attention mechanism layer Local-SA connected to a multi-layer perceptron, and finally obtains the output through the Conv-LSTM layer.

[0070] S4: Train the improved convolutional long short-term memory (Conv-LSTM) network to obtain a road surface defect detection model for detecting road surface defects.

[0071] For example, the WANDB toolkit can be used to monitor the loss function and stop training after it reaches a stable state. The improved convolutional long short-term memory (Conv-LSTM) network is trained based on a pre-built dataset. The specific methods for collecting and processing the dataset can refer to steps S1-S2 above.

[0072] For example, the detected pavement defects may include transverse cracks, longitudinal cracks, cracks, potholes, transverse repairs, longitudinal repairs, block repairs, and looseness.

[0073] S5: Inputting the road surface image sequence data set to be detected into the road surface disease detection model to obtain the results of the road surface disease detection, and obtaining a road surface disease detection report based on the results of the road surface disease detection and combined with the GPS path data.

[0074] For example, all road surface image sequence data sets to be detected are input into the road surface defect detection model, batch processed, and the detection results are obtained and saved in RGB image format. For images with defects, the corresponding image number is output. , save the summary as a CSV file; calculate the timestamp of the location where the surface disease is detected , and then obtain the specific location of the disease according to the collected GPS path data; by obtaining the type and location of all diseases, and counting the area and number of areas involved in the diseases, the road surface disease detection report is output.

[0075] Optionally, use the following formula to obtain the timestamp of the surface disease location:

[0076] ;

[0077] Among them, among them, is the timestamp of the surface disease location, n is the total number of subsets in the road surface image sequence dataset, and m is the mth image of the subset in the road surface image sequence dataset.

[0078] The above is a lightweight detection method for road surface defects provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding lightweight detection device for road surface defects, including:

[0079] The acquisition module is used to segment the road surface to be inspected and determine the drone video collection nodes. When the inspection begins, the drone flies in an uplink and downlink manner and starts GPS positioning to obtain drone video data and GPS path data of the road surface to be inspected;

[0080] The key frame extraction module is used to extract key frames from the UAV video data using a key frame extraction algorithm to determine the road surface image sequence dataset;

[0081] A construction module for constructing an improved convolutional long short-term memory (Conv-LSTM) network; replacing the encoder of the original Conv-LSTM network with a local feature extraction module, and replacing the upsampling in the decoder of the original Conv-LSTM network with a bilinear interpolation method; the local feature extraction module is composed of a convolutional layer, a global self-attention mechanism layer (Global-SA), a multi-layer perceptron (MLP), a local self-attention mechanism layer (Local-SA), and an MLP connected in sequence;

[0082] A training module is used to train the improved convolutional long short-term memory (Conv-LSTM) network to obtain a road surface defect detection model for detecting road surface defects;

[0083] The detection module is used to input the road surface image sequence data set to be detected into the road surface disease detection model, obtain the results of the road surface disease detection, and obtain a road surface disease detection report based on the results of the road surface disease detection combined with the GPS path data.

[0084] The specific limitations of the lightweight road surface defect detection device can be found in the limitations of the lightweight road surface defect detection method described above and will not be further elaborated here. Each module within the lightweight road surface defect detection device described above can be implemented in whole or in part via software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor within a computer device in hardware form, or stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0085] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program can be used to execute the above-mentioned lightweight detection method for road surface defects.

[0086] The present invention also provides Figure 5 The structural diagram of the computer equipment shown in FIG. Figure 5 As shown, at the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for its operations. The processor reads the corresponding computer program from the non-volatile storage into the internal memory and then runs it to implement the lightweight road surface defect detection method provided in the above embodiment.

[0087] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A lightweight detection method for road surface defects, characterized in that: include: The road surface to be inspected is segmented and the drone video collection nodes are determined. When the inspection begins, the drone flies in an uplink and downlink manner and starts GPS positioning to obtain drone video data and GPS path data of the road surface to be inspected. The key frame extraction algorithm is used to extract key frames from the UAV video data to determine the road surface image sequence dataset; Construct an improved convolutional long short-term memory Conv-LSTM network; The encoder of the original Conv-LSTM network is replaced with a local feature extraction module, and the upsampling in the decoder of the original Conv-LSTM network is replaced with a bilinear interpolation method; the local feature extraction module is composed of a convolutional layer, a global self-attention mechanism layer Global-SA, a multi-layer perceptron MLP, a local self-attention mechanism layer Local-SA, and an MLP connected in sequence; Train the improved convolutional long short-term memory (Conv-LSTM) network to obtain a road surface defect detection model for detecting road surface defects. Input the road surface image sequence dataset to be detected into the road surface defect detection model to obtain the results of the road surface defect detection. Based on the results of the road surface defect detection and combined with the GPS path data, a road surface defect detection report is obtained; The road surface disease detection report is obtained based on the results of the road surface disease detection and combined with the GPS path data, specifically including: According to the results of road surface damage detection, the timestamp of the surface damage location is obtained using the following formula: , in, is the timestamp of the surface disease location, n is the total number of subsets in the road surface image sequence dataset, and m is the mth image of the subset in the road surface image sequence dataset; Based on the timestamp of the surface defect location and combined with GPS path data, the defect location is determined and a road surface defect detection report is obtained.

2. The lightweight detection method for road surface defects according to claim 1, characterized in that: The key frame extraction algorithm is used to extract key frames from the UAV video data to determine the road surface image sequence data set, specifically including: Key frame extraction is performed on the drone video data using a key frame extraction algorithm to obtain image data. An image dataset is constructed based on the image data, and the image dataset is divided into several subsets. The several subsets constitute the road table image sequence dataset.

3. The lightweight detection method for road surface defects according to claim 2, characterized in that: The key frame extraction algorithm is used to extract key frames from the drone video data to obtain image data. An image dataset is constructed based on the image data. The image dataset is divided into several subsets, and the several subsets constitute the road table image sequence dataset, specifically including: By using the key frame extraction algorithm at a frame rate of 10FPS, key frames are extracted from the video to obtain image data, and the image dataset is obtained using the following formula: , in, is an image dataset, is the image data, is the timestamp corresponding to the image data, is the pixel position, is the pixel position Corresponding RGB values; The image dataset It is further divided into several subsets, each subset is an image data sequence, containing 10 orderly arranged image data. The specific calculation formula is as follows: , , in, is a subset, is the total number of subsets, is the mth image in the 10 image data i, and n is the sequence number of the subset; A plurality of image data sequences constitute the road table image sequence data set.

4. The lightweight detection method for road surface defects according to claim 1, characterized in that: Before extracting key frames from the drone video data using the key frame extraction algorithm, the road section range of the detection position in the drone video data is marked according to the GPS path data, and the drone video data is numbered, and each numbered road section is used as an independent data set.

5. The lightweight detection method for road surface defects according to claim 1, characterized in that: Abnormal data removal processing is performed on the image data sequence data set.

6. A lightweight detection device for road surface defects, characterized in that: include: The acquisition module is used to segment the road surface to be inspected and determine the drone video collection nodes. When the inspection begins, the drone flies in an uplink and downlink manner and starts GPS positioning to obtain drone video data and GPS path data of the road surface to be inspected; The key frame extraction module is used to extract key frames from the UAV video data using a key frame extraction algorithm to determine the road surface image sequence dataset; Building module for constructing improved convolutional long short-term memory Conv-LSTM network; The encoder of the original Conv-LSTM network is replaced with a local feature extraction module, and the upsampling in the decoder of the original Conv-LSTM network is replaced with a bilinear interpolation method; the local feature extraction module is composed of a convolutional layer, a global self-attention mechanism layer Global-SA, a multi-layer perceptron MLP, a local self-attention mechanism layer Local-SA, and an MLP connected in sequence; A training module is used to train the improved convolutional long short-term memory (Conv-LSTM) network to obtain a road surface defect detection model for detecting road surface defects; The detection module is used to input the road surface image sequence data set to be detected into the road surface disease detection model, obtain the results of the road surface disease detection, and obtain a road surface disease detection report based on the results of the road surface disease detection and combined with the GPS path data; The road surface disease detection report is obtained based on the results of the road surface disease detection and combined with the GPS path data, specifically including: According to the results of road surface damage detection, the timestamp of the surface damage location is obtained using the following formula: , in, is the timestamp of the surface disease location, n is the total number of subsets in the road surface image sequence dataset, and m is the mth image of the subset in the road surface image sequence dataset; Based on the timestamp of the surface defect location and combined with GPS path data, the defect location is determined and a road surface defect detection report is obtained.

7. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the lightweight detection method for road surface defects according to any one of claims 1 to 5 is implemented.

8. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for lightweight detection of road surface defects as claimed in any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Pavement disease detection method and device, terminal equipment and storage medium

    CN113255605A

  • Multi-dimensional pavement disease data set construction method

    CN115329109A