Road damage detection method, device, system, equipment, product and medium

Abstract

The present disclosure provides a road damage detection method, device, system, equipment, product and medium, the method comprises: acquiring at least one group of multi-source detection data;Each group of multi-source detection data corresponds to each detection position in the road, and each group of multi-source detection data comprises at least one of pavement data, road surface data and sub-road data;Projecting each group of multi-source detection data to obtain a detection three-dimensional model of the road;Based on the pre-constructed damage detection model, the detection three-dimensional model is identified to obtain the damage detection result corresponding to each detection position in the road, so that the final damage identification result is more comprehensive, which is convenient for operators to observe and meets the core demand of modern traffic network for efficient, accurate and comprehensive detection.

Description

Technical Field

[0001] This disclosure relates to the field of road inspection, and in particular to a method, apparatus, system, equipment, product and medium for road damage detection. Background Technology

[0002] Road infrastructure is a crucial component of the transportation system, and its structural condition and service performance directly affect public transportation safety and operational efficiency. As roads age, under the combined effects of traffic loads and natural environmental factors, the road surface and its internal structural layers are prone to damage such as cracks, potholes, voids, and structural defects. Therefore, periodic damage monitoring of roads is a vital technical means to ensure safe road operation.

[0003] However, traditional damage detection technologies are often not comprehensive enough for detecting road damage and are difficult to meet the core requirements of modern transportation networks for efficient, accurate and comprehensive detection. Summary of the Invention

[0004] The technical problem to be solved by this disclosure is that traditional damage detection technologies in the prior art are not comprehensive enough for road damage detection, and provides a road damage detection method, device, system, equipment, product and medium.

[0005] This disclosure solves the above-mentioned technical problems through the following technical solution:

[0006] In a first aspect, this disclosure provides a road damage detection method, the method comprising:

[0007] Acquire at least one set of multi-source detection data; each set of multi-source detection data corresponds to each detection location in the road, and each set of multi-source detection data includes at least one of road surface data, road surface data, and subsurface data;

[0008] Projecting each set of multi-source detection data yields a detection 3D model of the road;

[0009] The damage detection model is identified based on a pre-built damage detection model to obtain the damage detection results corresponding to each detection location in the road.

[0010] Optionally, acquiring at least one set of multi-source detection data includes:

[0011] Acquire all multi-source detection data for the road; each multi-source detection data has a corresponding detection time node;

[0012] All multi-source detection data are grouped according to the same detection time point to obtain at least one set of multi-source detection data.

[0013] Optionally, the road surface data is acquired by lidar, the road surface data is acquired by camera, and the subsurface data is acquired by ground-penetrating radar.

[0014] The lidar, the camera, and the ground-penetrating radar are mounted on the same detection vehicle, and the mounting reference planes of the lidar, the camera, and the ground-penetrating radar are parallel to the road surface.

[0015] Optionally, the method further includes:

[0016] Before triggering the lidar, camera, and ground-penetrating radar to collect multi-source detection data, the lidar, camera, and ground-penetrating radar are calibrated;

[0017] Optionally, the method further includes:

[0018] The road surface data is denoised using a median filtering algorithm; or the road surface data is denoised using a wavelet transform algorithm; or the road surface data is denoised using an outlier removal algorithm.

[0019] Optionally, the step of projecting each set of multi-source detection data to obtain a detection 3D model of the road includes:

[0020] All the multi-source detection data in each group of multi-source detection data are spliced ​​together to obtain the spliced ​​detection data corresponding to each group of multi-source detection data.

[0021] The spliced ​​detection data corresponding to each group of multi-source detection data is projected to obtain the detection three-dimensional model of the road.

[0022] Optionally, the step of splicing all the multi-source detection data in each group of multi-source detection data to obtain each group of spliced ​​detection data includes:

[0023] Based on the environmental data, generate the weight corresponding to each multi-source detection data in each group of multi-source detection data;

[0024] Each set of spliced ​​detection data is obtained by concatenating each multi-source detection data and its corresponding weight.

[0025] Optionally, the method further includes:

[0026] For the damage detection results that characterize the detection location where damage exists, the detection 3D model is labeled according to the corresponding damage detection results, and the labeled detection 3D model is displayed.

[0027] Optionally, the method further includes:

[0028] For the detection location where damage is characterized by the damage detection results, the local three-dimensional model corresponding to the detection location is determined from the detection three-dimensional model;

[0029] The local 3D model is annotated based on the corresponding damage detection results, and the annotated local 3D model is then displayed.

[0030] Secondly, this disclosure provides a road damage detection device, the device comprising:

[0031] The acquisition module is used to acquire at least one set of multi-source detection data; each set of multi-source detection data corresponds to each detection location in the road, and each set of multi-source detection data includes at least one of road surface data, road surface data, and subsurface data.

[0032] The projection module is used to project each set of multi-source detection data to obtain a detection three-dimensional model of the road.

[0033] The identification module is used to identify the detection three-dimensional model based on the pre-built damage detection model, and obtain the damage detection result corresponding to each detection location in the road.

[0034] Thirdly, this disclosure provides a road damage detection system, which includes a lidar, a ground-penetrating radar, a camera, and a server;

[0035] The lidar, the ground-penetrating radar, and the camera are all mounted on the same detection vehicle and are all connected to the server.

[0036] The server is used to execute the road damage detection method as described in any one of the first aspects.

[0037] Fourthly, this disclosure provides an electronic device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the road damage detection method as described in any of the first aspects.

[0038] Fifthly, this disclosure provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the road damage detection method as described in any of the first aspects.

[0039] In a sixth aspect, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the road damage detection method as described in any one of the first aspects.

[0040] The positive and progressive effects of this disclosure are as follows:

[0041] This disclosure constructs a three-dimensional detection model using multi-source detection data, including road surface data, road surface data, and subsurface data. This overcomes the limitations of single-dimensional data for road damage identification. The three-dimensional detection model can correlate road damage from multiple dimensions, such as road surface, road surface, and subsurface, making the final damage identification results more comprehensive and easier for operators to observe. It meets the core requirements of modern transportation networks for efficient, accurate, and comprehensive detection. At the same time, the multi-source detection data is classified according to the same detection location, making it easier for operators to observe the actual situation at each detection location.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0043] Figure 1 A schematic diagram illustrating an application scenario provided for an exemplary embodiment of this disclosure;

[0044] Figure 2 A schematic flowchart of a road damage detection method provided as an exemplary embodiment of this disclosure;

[0045] Figure 3 A flowchart illustrating step S201 of a road damage detection method provided for an exemplary embodiment of this disclosure;

[0046] Figure 4 A schematic flowchart of step S203 of a road damage detection method provided for an exemplary embodiment of this disclosure;

[0047] Figure 5 This is a schematic diagram of the structure of a detection three-dimensional model provided in an exemplary embodiment of the present disclosure;

[0048] Figure 6 A schematic diagram of a road damage detection device provided as an exemplary embodiment of this disclosure;

[0049] Figure 7 A schematic diagram of the structure of a road damage detection system provided as an exemplary embodiment of this disclosure;

[0050] Figure 8 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of the present disclosure. Detailed Implementation

[0051] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0052] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0053] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided by an embodiment of the present disclosure, which includes a vehicle 101 and a server 102.

[0054] In an optional embodiment, the vehicle 101 is equipped with multiple sensors, such as lidar, cameras, and ground-penetrating radar, with the mounting reference planes of the sensors parallel to the road surface. The lidar is used to collect road surface data, the camera to collect surface data, and the ground-penetrating radar to collect subsurface data. The sensors transmit the collected data to the server 102 for processing. In traditional damage detection technologies, individual sensors are limited by the mobility of the equipment, making continuous data acquisition impossible while the vehicle is in motion. This embodiment achieves continuous data acquisition while the vehicle is in motion by mounting multiple sensors on the vehicle 101.

[0055] In one optional embodiment, the server 102 may be an on-board server installed in the vehicle 101, which includes a road damage detection model used to determine the damage detection results at each detection location in the road based on multi-source detection data. In another optional embodiment, the vehicle 101 may have its own on-board server, and this on-board server and... Figure 1The server 102 shown is not the same server. After the vehicle-mounted server transmits the obtained data to server 102, the server can complete the subsequent steps and finally obtain the corresponding damage detection results. Hereinafter, the vehicle-mounted server involved in the first scenario and the server involved in the second scenario will be collectively referred to as the server.

[0056] Specifically, server 102 acquires at least one set of multi-source detection data; each set of multi-source detection data corresponds to each detection location in the road, and each set of multi-source detection data includes at least one of road surface data, road surface data, and roadbed data; each set of multi-source detection data is projected to obtain a three-dimensional detection model of the road; the three-dimensional detection model is identified based on the pre-built damage detection model to obtain the damage detection result corresponding to each detection location in the road.

[0057] The following describes a road damage detection method provided by an embodiment of this disclosure. Figure 2 This is a flowchart illustrating a road damage detection method according to an embodiment of this disclosure. This specification provides the method operation steps as shown in the embodiments or flowchart, but based on conventional or non-inventive methods, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In practice, when the system or server product executes the method, it can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 2 As shown, the method may include:

[0058] S201. Obtain at least one set of multi-source detection data.

[0059] Each set of multi-source detection data corresponds to each detection location on the road, and each set of multi-source detection data includes at least one of road surface data, road surface data, and roadbed data.

[0060] Specifically, road surface data includes, but is not limited to, parameters such as road surface smoothness (IRI index), rut depth (accuracy ≤ 0.1 mm), and road alignment (cross slope, longitudinal slope). Road surface data includes, but is not limited to, road surface texture (texture depth) and cracks (width ≥ 0.2 mm). Subsurface data includes, but is not limited to, voids in the subsurface structural layer (area ≥ 0.1 m²). 2Damage detection methods include identifying defects such as cracks (length ≥ 0.5m) and abnormal moisture content. Traditional damage detection techniques rely on single-dimensional data for single-dimensional damage detection, such as identifying only damage present in surface data. This fragmented approach leads to a lack of correlation between surface damage detection results (e.g., cracks, potholes) and subsurface damage detection results (e.g., subgrade voids, interlayer stripping). Technicians must conduct multiple phased inspections and cross-device data comparisons to preliminarily infer the correlation, significantly extending the detection cycle and resulting in incomplete damage detection results. This embodiment, however, correlates multi-source detection data, establishing a correlation between multiple sources at the same detection location. This allows the final damage detection results to reflect the condition from the subsurface to the surface, resulting in more comprehensive damage detection, significantly reducing the detection cycle, and improving detection efficiency.

[0061] Furthermore, road surface data, road surface data, and under-road data are collected from different sensors. For example, road surface data is collected by LiDAR, road surface data by camera, and under-road data by ground-penetrating radar. This is one example of a multi-source detection data acquisition method, but it is not limited to this. The LiDAR, camera, and ground-penetrating radar are installed on the same detection vehicle, and their mounting reference planes are parallel to the road surface. While the detection vehicle is moving on the road, the LiDAR, camera, and ground-penetrating radar simultaneously collect multi-source detection data. The movement of the detection vehicle overcomes the limitations of the individual sensors' mobility, enabling continuous data acquisition and improving the efficiency of road damage detection.

[0062] In one embodiment, since the multi-source detection data are collected from different sensors, the parameter settings of these different sensors can lead to problems in subsequent processing. For example, for a crack at the same detection location, the road surface data may show a crack width of 5 cm, while the actual crack width is only 0.1 cm. Furthermore, the sensors lack an adaptive calibration mechanism. As usage time increases, factors such as wear and tear on equipment components, vibration interference, and changes in temperature and humidity can cause data deviations to accumulate. If manual calibration is not performed in a timely manner, the deviations will continue to widen, ultimately resulting in severely distorted damage detection results that cannot provide a reliable basis for road inspection decisions. Therefore, the method further includes:

[0063] Before triggering the acquisition of multi-source detection data by the lidar, camera, and ground-penetrating radar, the lidar, camera, and ground-penetrating radar are calibrated. After calibration, the acquisition of multi-source detection data can be triggered. For example, the lidar and camera can be calibrated via pixel calibration, which can be implemented using a standard calibration board; the ground-penetrating radar can be calibrated via phase calibration, which can be implemented using a metal calibration rod. The error between the final road surface data or road surface data and the actual data is less than or equal to a first error threshold, and the error between the under-road data and the actual data is less than or equal to a second error threshold. The specific settings of the first and second error thresholds are determined according to the actual situation and are not specifically limited in this embodiment. This embodiment ensures the reliability of the acquired multi-source detection data by calibrating the sensors before acquiring multi-source detection data, avoiding the drawback of operators needing to manually annotate and verify large amounts of multi-source detection data.

[0064] In one embodiment, to improve the accuracy of multi-source detection data collected by different sensors and thus improve the accuracy of damage detection, the method further includes:

[0065] Noise reduction processing is performed on all multi-source detection data in each group of multi-source detection data.

[0066] Due to the differences in the format and sensors of the multi-source detection data, this embodiment provides different noise reduction methods for different types of multi-source detection data in each group of multi-source detection data. For example, median filtering algorithm is used to denoise the road surface data to remove road surface texture interference; or wavelet transform algorithm is used to denoise the under-road data to suppress clutter signals; or outlier removal algorithm is used to denoise the road surface data to effectively suppress noise points in 3D modeling. The median filtering algorithm, wavelet transform algorithm, or outlier removal algorithm mentioned above can all refer to the implementation methods in the prior art, and will not be elaborated in this embodiment. In addition, those skilled in the art can also choose the corresponding noise reduction method according to the type of multi-source detection data actually collected.

[0067] In an optional implementation, the method further includes:

[0068] Inertial navigation data is used to correct all multi-source detection data in each group to offset positional shifts caused by vehicle vibrations. The navigation inertial data is collected by a positioning module installed on the detection vehicle.

[0069] In one embodiment, multi-source detection data can be acquired by different sensors mounted on the detection vehicle. Each sensor continuously collects multi-source detection data on the road as the vehicle travels. Undoubtedly, this generates a large amount of multi-source detection data during a single road damage detection process. Mixing multi-source detection data from different detection time points introduces displacement errors into the 3D model, leading to misalignment or artifacts in the fused product. To align multiple multi-source detection data corresponding to the same detection location and avoid affecting the generation of the 3D model, thereby impacting the damage detection results, see [reference needed]. Figure 3 Step S201 specifically includes:

[0070] S2011. Obtain all multi-source detection data for the road.

[0071] Step S2011 includes all multi-source detection data collected by all sensors during a road damage detection process. Each multi-source detection data has a corresponding detection time node, which is output as a timestamp by each sensor. The detection time node can be obtained from the output of the acceleration sensor installed on the detection vehicle.

[0072] S2013. Group all multi-source detection data according to the same detection time node to obtain at least one set of multi-source detection data.

[0073] In this embodiment, since all sensors are installed on the same moving detection vehicle, when collecting data at the same detection location, the detection time nodes corresponding to the multi-source detection data output by each sensor should be the same. The multi-source detection data is grouped by the detection time nodes to ensure that each group of multi-source detection data is consistent in the time and spatial dimensions, so as to avoid affecting the generation of the three-dimensional model and thus avoid affecting the damage detection results.

[0074] In one embodiment, each multi-source detection data in each group of multi-source detection data also has corresponding spatial coordinates and the pose information of the detection vehicle. The spatial coordinates and pose information are obtained by the output of the acceleration sensor installed on the detection vehicle, ensuring spatiotemporal reference consistency and facilitating the construction of the detection 3D model.

[0075] S203. Project each set of multi-source detection data to obtain a three-dimensional detection model of the road.

[0076] Since road surface data is typically in the form of spatial point cloud data, road surface data is typically in the form of image data, and under-road data is typically in the form of radar echo signals, one implementation method for step S203 is provided. First, a basic 3D model can be constructed using the road surface data. Specifically: since spatial point cloud data is inherently in a real spatial coordinate system, with one point corresponding to one spatial coordinate, the discrete point cloud can be transformed into a continuous road surface geometry, for example, by generating the corresponding basic 3D model through triangular meshing or surface fitting. Next, the road surface data can be projected onto the basic 3D model. Specifically: each pixel in the image data can be mapped to its corresponding position in the basic 3D model based on spatial positional relationships. Finally, the under-road data can be projected onto the basic 3D model. Specifically, radar echo signals, after time-depth conversion, can form a radar profile image, and each pixel in the radar profile image can be projected onto the basic 3D model based on spatial positional relationships. After all projections are completed, a detection 3D model is obtained. The spatial positional relationships can be constructed based on the spatial coordinates calibrated by sensors (cameras or ground-penetrating radar) and the pose information of the detected vehicle.

[0077] In addition to the methods mentioned above, those skilled in the art can also construct the detection 3D model according to any of the existing 3D model construction methods. This embodiment also provides another implementation method for step S203 for reference. See [link to relevant documentation]. Figure 4 ,include:

[0078] S2031. All the multi-source detection data in each group of multi-source detection data are spliced ​​together to obtain the spliced ​​detection data corresponding to each group of multi-source detection data.

[0079] Step S2031 specifically includes: generating weights corresponding to each multi-source detection data in each group of multi-source detection data based on environmental data, and splicing each multi-source detection data and its corresponding weight to obtain each group of spliced ​​detection data. The environmental data includes, but is not limited to, temperature and humidity data. Environmental data can be acquired by temperature or humidity sensors installed on the detection vehicle. Typically, only one set of environmental data is needed for a single road damage detection, but it is not limited to acquiring corresponding environmental data at each detection time point for the sake of splicing accuracy.

[0080] The purpose of adjusting the weights of each multi-source detection data point in each group of multi-source detection data during the stitching process using environmental data is to address the impact of temperature or humidity data exceeding preset ranges on the collected multi-source detection data. For example, in rainy, foggy, or low-temperature weather, sensors are prone to condensation, and the transmission of detection signals is hindered, leading to a decrease in the acquisition accuracy of multi-source detection data by more than 30%, and even data failure in some extreme cases. Therefore, the participation of multi-source detection data with low reliability in the generation of the detection 3D model should be reduced. The weights can be extracted from each multi-source detection data point under the environmental data using an attention model. The construction of the attention model can be found in existing implementation methods, and this embodiment does not impose any specific limitations.

[0081] Detecting 3D models, such as Figure 5 As shown, the 3D model reveals subsurface voids 51 and pavement cracks 52 at the detection location. The local 3D model also shows the specific road structure at the detection location, including the base layer 53, lower layer 54, intermediate layer 55, and upper layer 56. The 3D model obtained according to this embodiment can reflect the actual road conditions as a whole.

[0082] S2033. Project the spliced ​​detection data corresponding to each group of multi-source detection data to obtain the detection three-dimensional model of the road.

[0083] Regarding the construction of the 3D detection model in step S2032, the above-mentioned method of first constructing a basic model using road surface data and then projecting it based on road surface data and subsurface data can be referred to. In an optional embodiment, a fusion model can also be used. The base model of the fusion model is a deep learning model, and the input to the fusion model is multi-source detection data. The output of the fusion model is the detected 3D model. The specific training process can be found in the training process of deep learning models in existing technologies, and is not specifically limited in this embodiment.

[0084] S205. Based on the pre-built damage detection model, the detection three-dimensional model is identified to obtain the damage detection results corresponding to each detection location in the road.

[0085] The damage detection model is trained on a base model using historical 3D detection models as input and historical damage detection results as output. The base model can be a machine learning model, a deep learning model, etc., and the training process can be supervised training, unsupervised training, etc., without particular limitation in this embodiment. Furthermore, the damage detection results include, but are not limited to, damage type, detection location, crack size, void depth and area, correlation, and damage level.

[0086] In one embodiment, the method further includes:

[0087] For the damage detection results indicating the location of damage, the corresponding 3D detection model is annotated based on the damage detection results, and the annotated 3D detection model is then displayed. The annotated 3D detection model provides a global view of the road and its damage detection results, facilitating operators' observation of the overall road condition and preventing omissions. Compared to existing technologies that only output damage detection results from a single dimension, this embodiment constructs a holistic framework from the road surface, road surface, and subsurface, thereby linking damage from different dimensions and enabling those skilled in the art to better observe the specific details of the damage.

[0088] In one embodiment, the method further includes:

[0089] For each damage detection location identified by the damage detection results, a corresponding local 3D model is determined from the detection 3D model. The local 3D model is then annotated based on the corresponding damage detection results, and the annotated local 3D model is displayed. This local 3D model allows users to zoom in and view the damage details at the corresponding detection location within the 3D model.

[0090] In one embodiment, in addition to displaying the detected 3D model and / or a local 3D model, the method further includes:

[0091] It displays basic road information, sensor parameters, data acquisition logs, damage distribution maps, 3D detection models, and damage detection result tables.

[0092] In addition, in the above embodiments, the information to be displayed can be sent to the vehicle-mounted terminal on the testing vehicle for display, or sent to other terminals of the operator for display.

[0093] In one embodiment, the method further includes:

[0094] Multi-source detection data, damage detection results, and 3D models of the road at all detection locations are stored. The storage device can be a solid-state drive or a cloud storage terminal, facilitating data traceability and statistical analysis later. Data for each road segment can be stored in a format specifying the detection location and time point, enabling operators to easily trace the data.

[0095] In one embodiment, the method further includes:

[0096] Damage detection results indicating the location of damage are used to assess the damage level. The damage level assessment is based on the "Highway Technical Condition Assessment Standard" (JTG 5210-2018).

[0097] The following example uses a section of a national highway, K500-K520, for road inspection. The road is 20km long, designed for a speed of 100km / h, and has an asphalt concrete pavement. It has been in service for 8 years, and historical maintenance records show that it underwent partial crack sealing treatment 3 years ago. The road damage detection method provided in this embodiment will be described in detail below:

[0098] 1. Preparation stage:

[0099] (1) A light van with a wheelbase of 2.8m can be selected as the inspection vehicle. The camera, ground-penetrating radar, lidar, temperature sensor, humidity sensor and other sensors are fixed on a special bracket in the rear compartment of the vehicle to ensure that the installation reference surface of all sensors is parallel to the road surface and the installation error is ≤ ±0.5°. The sensors can be powered by the vehicle's 12V battery + backup lithium battery dual power supply mode to ensure the stability of continuous operation.

[0100] (2) Accuracy calibration: The camera or lidar is calibrated by pixel calibration using a standard calibration board, with a first error threshold of 0.01 mm; the phase of the ground-penetrating radar antenna is calibrated using a metal calibration rod, with a second error threshold of 0.02 mm.

[0101] (3) Parameter configuration: Input the detection task into the on-board terminal of the detection vehicle, including the road name "National Highway K500-K520 section", the road surface structure type "Asphalt concrete: 4cm top layer + 6cm middle layer + 8cm bottom layer + 18cm base layer", and the design axle load of 100kN; set the driving speed of the detection vehicle to 100km / h (matching the design speed of the road); set the sampling frequency of the camera to 10 frames / second; set the ground penetrating radar to 500MHz main frequency and sampling interval to 0.01m; select 128-line lidar with a scanning frequency of 200Hz and a point cloud density of ≥200 points / m. 2 .

[0102] (4) Safety inspection: Check the braking system, tire pressure and lighting status of the test vehicle, affix "Highway Inspection Operation" warning signs to the front and rear of the test vehicle, and arrange a guide vehicle to provide early warning 500m ahead to ensure road traffic safety during the data collection process.

[0103] 2. Data collection phase:

[0104] The detection vehicle travels at a constant speed of 100 km / h, and all sensors are triggered synchronously at the same detection time point. Details are as follows:

[0105] (1) Road surface data acquisition: The camera (preferably with a lens focal length of 25mm and a field of view of 60°, but not limited to this) acquires road surface images in real time. The image resolution reaches 1920×1080 pixels, and the damage detection results can be identified by the changes in image grayscale values. When the detection vehicle travels to K510+200, a transverse crack is captured on the road surface. According to the camera's built-in algorithm, the crack width is 0.8mm, the length is 3.2m, and the depth is 0.3mm. The crack direction is perpendicular to the driving direction, and there is no loosening or chipping at the crack edge.

[0106] (2) Road surface data acquisition: The lidar collects road surface data in real time, including parameters such as road surface smoothness (IRI index), rut depth (accuracy ≤ 0.1 mm), and road alignment (cross slope, longitudinal slope).

[0107] (3) Subsurface Data Acquisition: Ground-penetrating radar synchronously emits electromagnetic waves. After the electromagnetic waves penetrate the road surface structure, the characteristics of the subsurface medium are analyzed by the amplitude and phase changes of the reflected signals. At K510+200, an abnormal area of ​​reflected signals is shown at a depth of 0.4m (at the interface between the base course and the subbase course). According to the radar's built-in algorithm, the shape of the void is approximately elliptical, with a major axis of 1.2m and a minor axis of 0.53m. The void area is calculated to be 0.2m². 2 The density of the medium surrounding the vacuolated area is normal, and there is no tendency for it to expand.

[0108] 3. Processing stage:

[0109] (1) Data preprocessing: Noise reduction processing is performed on all multi-source detection data. Median filtering algorithm is used to remove road surface texture interference for road surface data, wavelet transform algorithm is used to suppress clutter signals for under-road data, and outlier points are removed from road surface data according to outlier removal algorithm. The positional offset caused by vehicle bumps can also be corrected by detecting the inertial navigation data output by the vehicle, making the correspondence of multi-source detection data more accurate.

[0110] (2) Multi-source data fusion: The road surface data, road surface data, and subsurface data corresponding to each detection time node are correlated to obtain the multi-source detection data corresponding to each detection location. Based on the multi-source detection data corresponding to all detection locations in the road, a three-dimensional detection model of the road is generated through a fusion model. The detection three-dimensional model is identified through a damage detection model, and the damage detection results corresponding to the detection location K510+200 are identified. The damage detection results indicate that there is "road surface crack + subsurface void" damage at the detection location, and the direction of crack extension is perpendicular to the center of the void area. The local three-dimensional model corresponding to the detection location K510+200 is determined from the detection three-dimensional model. The model resolution of the local three-dimensional model is 0.05m×0.05m×0.02m, which clearly shows the distribution pattern of cracks on the road surface and the spatial location of voids under the road.

[0111] (3) Damage Level Assessment: The damage level was assessed based on the damage detection results according to the "Highway Technical Condition Assessment Standard" (JTG 5210-2018). Damage detection results showed: transverse crack width 0.8 mm, classified as a minor crack; subgrade void area 0.2 m². 2 The injury is classified as moderate detachment, and the overall assessment classifies it as Grade II, with an automatic damage risk factor of 0.65.

[0112] (4) Data storage: All data is stored in the format of “detection location + detection time node” to solid-state drive to ensure that the data is not lost and is traceable.

[0113] 4. Output stage:

[0114] A detection report is generated and displayed. The core content of the report includes: basic road information, sensor parameters, data acquisition logs, damage distribution map, 3D detection model, and damage detection result table. The 3D detection model at location K510+200 also includes damage detection results, such as: damage type, station number, crack size, depth and area of ​​voids, correlation, and damage level.

[0115] This embodiment demonstrates that by integrating multi-source detection data, it achieves simultaneous spatiotemporal acquisition of three types of data: road surface, road surface, and subsurface data, and constructs a 3D detection model based on this data, breaking through the limitations of fragmented detection. A detection vehicle enables mobile damage detection, balancing detection efficiency and data accuracy. A damage identification model shortens the data processing cycle, rapidly outputting detection reports and reducing manual intervention. Optimized sensor adaptive calibration and anti-interference design ensure detection stability under complex environments such as strong light and light rain, enhancing the practicality and reliability of the technology system and providing core support for road damage detection.

[0116] Corresponding to the embodiments of the aforementioned road damage detection method, this disclosure also provides embodiments of a road damage detection device.

[0117] Figure 6 An exemplary embodiment of this disclosure provides a road damage detection device, the device comprising:

[0118] The acquisition module 61 is used to acquire at least one set of multi-source detection data; each set of multi-source detection data corresponds to each detection location in the road, and each set of multi-source detection data includes at least one of road surface data, road surface data, and roadbed data.

[0119] Projection module 63 is used to project each set of multi-source detection data to obtain a three-dimensional detection model of the road.

[0120] The identification module 65 is used to identify the detection 3D model based on the pre-built damage detection model to obtain the damage detection result corresponding to each detection location in the road.

[0121] In one embodiment, the acquisition module 61 is further configured to:

[0122] Acquire all multi-source detection data for the road. Each multi-source detection data point has a corresponding detection time point.

[0123] All multi-source detection data are grouped according to the same detection time point to obtain at least one set of multi-source detection data.

[0124] In one embodiment, road surface data is acquired using lidar, road surface data is acquired using a camera, and under-road data is acquired using ground-penetrating radar.

[0125] The lidar, camera, and ground-penetrating radar are mounted on the same detection vehicle, and the mounting reference plane of the lidar, camera, and ground-penetrating radar is parallel to the road surface.

[0126] In one embodiment, the apparatus further includes:

[0127] The calibration module is used to calibrate the lidar, camera, and ground-penetrating radar before triggering the acquisition of multi-source detection data.

[0128] In one embodiment, the apparatus further includes:

[0129] The noise reduction module is used to reduce noise in road surface data using a median filtering algorithm, or in subsurface data using a wavelet transform algorithm, or in road surface data using an outlier removal algorithm.

[0130] In one embodiment, the projection module 63 is further configured to:

[0131] All multi-source detection data in each group of multi-source detection data are spliced ​​together to obtain the spliced ​​detection data corresponding to each group of multi-source detection data.

[0132] The spliced ​​detection data corresponding to each set of multi-source detection data is projected to obtain a three-dimensional detection model of the road.

[0133] In one embodiment, the projection module 63 is further configured to:

[0134] The weights for each multi-source detection data point in each group of multi-source detection data are generated based on the environmental data.

[0135] Each set of spliced ​​detection data is obtained by concatenating each multi-source detection data and its corresponding weight.

[0136] In one embodiment, the apparatus further includes:

[0137] The display module is used to characterize the detection location where damage exists based on the damage detection results, annotate the detection 3D model according to the corresponding damage detection results, and display the annotated detection 3D model.

[0138] In one embodiment, the apparatus further includes:

[0139] The display module is used to characterize the detection location of damage based on the damage detection results, determine the local 3D model corresponding to the detection location from the detection 3D model, annotate the local 3D model according to the corresponding damage detection results, and display the annotated local 3D model.

[0140] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this disclosure according to actual needs.

[0141] An exemplary embodiment of this disclosure also provides a road damage detection system, see [link to example]. Figure 7 The detection system includes a lidar 74, a ground-penetrating radar 75, a camera 73, and a server.

[0142] The lidar 74, ground-penetrating radar 75, and camera 73 are mounted on the same detection vehicle 71 (the detection vehicle 71 can be connected to...). Figure 1 Vehicle 101 in the text corresponds to the server (the server can be connected to...). Figure 1 (Corresponding to server 102 in the above embodiments) Communication connection. The server is used to execute the road damage detection method as described in any of the above embodiments. The server is preferably an edge computing server, configured with an 8-core CPU + 24GB GPU memory.

[0143] Specifically, the detection vehicle 71 is a modified light van (curb weight ≤ 3.5t), with a sensor integrated bracket 72 (anti-shake level: ≤ 0.1° / s angular vibration) installed on the top of the vehicle. The bracket 72 has a built-in positioning module (positioning accuracy: horizontal ≤ 2cm, vertical ≤ 5cm), and is also equipped with a vehicle speed / acceleration sensor to synchronously collect pose information, spatial coordinates, and inertial navigation data during detection. The camera 73 preferably uses a high-frequency 3D camera 73, set with a frame rate ≥ 120fps, a resolution of 1920×1080, and is equipped with an 850nm infrared structured light laser (power 5W, spot diameter ≤ 0.3mm) to collect road surface data. Furthermore, the camera 73 also has a built-in light intensity sensor; when the ambient light intensity is ≥ 50000 lux, a polarization filter is automatically activated to reduce interference from strong light reflection on the road surface data. The lidar 74 is set with 128 lines, a scanning frequency of 200Hz, and a point cloud density ≥ 200 points / m². 2 The ground-penetrating radar 75 is set with a center frequency of 1GHz, a scanning speed of ≥100 lines / s, and a detection depth of 1.5m to collect road surface data.

[0144] In one embodiment, the road damage detection system further includes a panoramic camera 77, a temperature sensor 76, and a humidity sensor (not shown). The panoramic camera 77, temperature sensor 76, and humidity sensor are all communicatively connected to a server.

[0145] Specifically, the panoramic camera 77 is equipped with four lenses stitched together, offering a 360° field of view and 8-megapixel resolution, recording information such as road signs and the surrounding environment. The temperature sensor 76 collects temperature data, ranging from -20℃ to 60℃. The humidity sensor collects ambient temperature and humidity data, ranging from 0% to 100%RH.

[0146] In one embodiment, the on-board terminal 78 in the detection vehicle 71 is connected to a server for displaying information sent by the server.

[0147] In one embodiment, an external terminal communicates with a server to display information sent by the server.

[0148] An exemplary embodiment of this disclosure also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and used to run on the processor. When the processor executes the computer program, it implements the road damage detection method of any of the above embodiments. Figure 8 The electronic device 80 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0149] like Figure 8As shown, the electronic device 80 can be manifested as a general-purpose computing device, such as a server device. The components of the electronic device 80 may include, but are not limited to: at least one processor 81, at least one memory 82, and a bus 83 connecting different system components (including memory 82 and processor 81).

[0150] Bus 83 includes a data bus, an address bus, and a control bus.

[0151] The memory 82 may include volatile memory, such as random access memory (RAM) 821 and / or cache memory 822, and may further include read-only memory (ROM) 823.

[0152] The memory 82 may also include a program tool 825 (or utility) having a set (at least one) program unit 824, such program unit 824 including but not limited to: an operating system, one or more application programs, other program units and program data, each or some combination of these examples may include an implementation of a network environment.

[0153] The processor 81 executes various functional applications and data processing by running computer programs stored in the memory 82, such as the road damage detection method provided in any of the above embodiments.

[0154] Electronic device 80 can also communicate with one or more external devices 84 (e.g., keyboard, pointing device, etc.). This communication can be performed through input / output (I / O) interface 85. Furthermore, electronic device 80 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public network, such as the Internet) via network adapter 86. As shown, network adapter 86 communicates with other units of electronic device 80 via bus 83. It should be understood that, although not shown in the figure, other hardware and / or software units can be used in conjunction with electronic device 80, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0155] It should be noted that although several units / components or sub-units of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / components described above can be embodied in one unit / component. Conversely, the features and functions of one unit / component described above can be further divided and embodied by multiple units / components.

[0156] An exemplary embodiment of this disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the road damage detection method provided in any of the above embodiments.

[0157] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0158] An exemplary embodiment of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the road damage detection method described above.

[0159] An exemplary embodiment of this disclosure also provides a computer program that, when executed by a processor, implements the road damage detection method described above.

[0160] The program code for executing the computer program product disclosed herein can be written in any combination of one or more programming languages. The program code can be executed entirely on a user device, partially on a user device, as a stand-alone software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0161] While specific embodiments of this disclosure have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this disclosure is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this disclosure, but all such changes and modifications fall within the scope of protection of this disclosure.

Claims

1. A method for detecting road damage, characterized in that, The method includes: Acquire at least one set of multi-source detection data; each set of multi-source detection data corresponds to each detection location in the road, and each set of multi-source detection data includes at least one of road surface data, road surface data, and subsurface data; Projecting each set of multi-source detection data yields a detection 3D model of the road; The damage detection model is identified based on a pre-built damage detection model to obtain the damage detection results corresponding to each detection location in the road.

2. The road damage detection method as described in claim 1, characterized in that, The acquisition of at least one set of multi-source detection data includes: Acquire all multi-source detection data for the road; each multi-source detection data has a corresponding detection time node; All multi-source detection data are grouped according to the same detection time point to obtain at least one set of multi-source detection data.

3. The road damage detection method as described in claim 1, characterized in that, The road surface data is obtained by lidar, the road surface data is obtained by camera, and the subsurface data is obtained by ground penetrating radar. The lidar, the camera, and the ground-penetrating radar are mounted on the same detection vehicle, and the mounting reference planes of the lidar, the camera, and the ground-penetrating radar are parallel to the road surface.

4. The road damage detection method as described in claim 3, characterized in that, The method further includes: Before triggering the lidar, camera, and ground-penetrating radar to collect multi-source detection data, the lidar, camera, and ground-penetrating radar are calibrated; And / or, the method further includes: The road surface data is denoised using a median filtering algorithm; or the road surface data is denoised using a wavelet transform algorithm; or the road surface data is denoised using an outlier removal algorithm.

5. The road damage detection method as described in claim 1, characterized in that, The process of projecting each set of multi-source detection data to obtain a detection 3D model of the road includes: All the multi-source detection data in each group of multi-source detection data are spliced ​​together to obtain the spliced ​​detection data corresponding to each group of multi-source detection data. The spliced ​​detection data corresponding to each group of multi-source detection data is projected to obtain the detection three-dimensional model of the road.

6. The road damage detection method as described in claim 5, characterized in that, The step of stitching together all the multi-source detection data in each group of multi-source detection data to obtain each group of stitched detection data includes: Based on environmental data, generate weights corresponding to each multi-source detection data in each group of multi-source detection data; Each set of spliced ​​detection data is obtained by concatenating each multi-source detection data and its corresponding weight.

7. The road damage detection method as described in claim 1, characterized in that, The method further includes: For the damage detection results that characterize the detection location where damage exists, the detection 3D model is labeled according to the corresponding damage detection results, and the labeled detection 3D model is displayed. And / or, for the detection location where damage is characterized by the damage detection results, determine the local three-dimensional model corresponding to the detection location from the detection three-dimensional model; The local 3D model is annotated based on the corresponding damage detection results, and the annotated local 3D model is then displayed.

8. A road damage detection device, characterized in that, The device includes: The acquisition module is used to acquire at least one set of multi-source detection data; each set of multi-source detection data corresponds to each detection location in the road, and each set of multi-source detection data includes at least one of road surface data, road surface data, and subsurface data. The projection module is used to project each set of multi-source detection data to obtain a detection three-dimensional model of the road. The identification module is used to identify the detection three-dimensional model based on the pre-built damage detection model, and obtain the damage detection result corresponding to each detection location in the road.

9. A road damage detection system, characterized in that, The detection system includes lidar, ground-penetrating radar, cameras, and servers; The lidar, the ground-penetrating radar, and the camera are all mounted on the same detection vehicle and are all connected to the server. The server is used to execute the road damage detection method as described in any one of claims 1-7.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the road damage detection method as described in any one of claims 1-7.

11. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the road damage detection method as described in any one of claims 1-7.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the road damage detection method as described in any one of claims 1-7.

Images