A method, system, electronic device and product for detecting roadbed defects

Abstract

This invention belongs to the field of road inspection technology, and its purpose is to provide a method, system, electronic device, and product for detecting roadbed defects. The method includes: real-time acquisition of multi-source inspection data of a specified roadbed and data fusion processing to obtain fused inspection data; crack detection processing of roadbed surface images to obtain crack feature data of the specified roadbed; surface settlement analysis processing of laser point cloud data to obtain settlement feature data of the specified roadbed; deep defect analysis processing of ultrasonic detection data and electromagnetic wave reflection information to obtain deep defect feature data of the specified roadbed; construction of a three-dimensional defect model of the specified roadbed based on crack feature data, settlement feature data, and deep defect feature data; and visualization processing of the three-dimensional defect model. This invention can achieve comprehensive defect detection of both the surface and deep layers of the roadbed, with a high degree of intelligence.

Description

Technical Field

[0001] This invention belongs to the field of road inspection technology, specifically relating to a method, system, electronic equipment, and product for detecting roadbed defects. Background Technology

[0002] With the rapid development of highways, railways, and urban infrastructure, the health of the roadbed, as the fundamental load-bearing structure of roads and tracks, directly affects the safety and service life of road traffic. However, due to the combined effects of natural factors (such as rainwater erosion and geological activity) and human factors (such as overloaded vehicles and improper construction), roadbeds are prone to various defects, including surface cracks, settlement, and potholes, as well as deep structural voids and material loosening. If these defects are not detected and repaired in a timely manner, they will lead to a decrease in the roadbed's load-bearing capacity and may even cause major safety accidents. Therefore, the rapid and accurate detection and analysis of roadbed defects are crucial for road maintenance and management.

[0003] Traditional roadbed inspection relies on manual labor, primarily using engineers to visually inspect, tap, and measure pavement cracks, settlement, and other defects on-site. While this method is inexpensive and simple to operate, its drawbacks are significant: the results depend on the engineers' experience, are highly subjective, and their accuracy and reliability are difficult to guarantee; furthermore, manual inspection is inefficient, unsuitable for large-area roadbed inspections, and incapable of detecting deeper defects.

[0004] To overcome the problems of manual inspection, technologies such as optical image inspection, lidar inspection, and ground-penetrating radar and ultrasonic inspection have emerged. Optical image inspection utilizes high-definition cameras or drones to capture images of the road surface and uses image processing algorithms (such as edge detection and deep learning models) to identify road surface defects such as cracks and spalling. This technology has high detection accuracy for surface defects and is suitable for rapid screening of large areas. Lidar inspection technology mainly collects high-precision three-dimensional point cloud data of the roadbed surface, and can identify deformation defects such as settlement and depressions. Ground-penetrating radar and ultrasonic technologies can detect cavities, cracks, and other abnormal features in the deep structure of the roadbed. Ground-penetrating radar obtains underground information by analyzing the reflected electromagnetic wave signals, and is suitable for identifying hidden dangers inside the roadbed. Ultrasonic inspection calculates the depth and extent of defects by using echo delay time and signal amplitude.

[0005] The aforementioned prior art has played an important role in the field of roadbed defect detection. However, in using the prior art, the inventors have discovered at least the following problems, making it difficult to meet practical needs:

[0006] Existing single technologies are typically designed for specific types of defects. For example, optical imaging and lidar are mainly used for surface crack detection, while ground-penetrating radar and ultrasound are more suitable for deep structure detection. These detection methods are independent of each other and lack a unified detection scheme, making it impossible to cover both surface and deep defects simultaneously, thus limiting the detection range. In addition, the detection data from existing technologies usually require post-processing by professionals, which is time-consuming and complex, making it difficult to meet the needs of rapid diagnosis and real-time analysis, and the level of intelligence is insufficient. Summary of the Invention

[0007] The present invention aims to solve the above-mentioned technical problems to at least a certain extent, and provides a method, system, electronic device and product for detecting roadbed defects.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides a method for detecting roadbed defects, comprising:

[0010] Real-time acquisition of multi-source detection data of a designated roadbed; wherein, the multi-source detection data includes roadbed surface images acquired by a camera, laser point cloud data acquired by a lidar, ultrasonic detection data acquired by an ultrasonic sensor, and electromagnetic wave reflection information acquired by a ground-penetrating radar;

[0011] The multi-source detection data is fused to obtain fused detection data;

[0012] Crack detection processing is performed on the roadbed surface image in the fused detection data to obtain crack feature data of the specified roadbed;

[0013] Surface settlement analysis is performed on the laser point cloud data in the fused detection data to obtain the settlement characteristic data of the specified roadbed;

[0014] The ultrasonic detection data and electromagnetic wave reflection information in the fused detection data are subjected to in-depth disease analysis and processing to obtain the in-depth disease characteristic data of the specified roadbed.

[0015] Based on the crack feature data, the settlement feature data, and the deep disease feature data, a three-dimensional disease model of the specified roadbed is constructed.

[0016] The three-dimensional model of the disease is then visualized.

[0017] In one possible design, the multi-source detection data is fused to obtain fused detection data, including:

[0018] The sampling time of each sensor in the multi-source detection data is obtained, and a reference time is obtained based on the sampling time of each sensor; wherein each sensor includes lidar, camera, ultrasonic sensor and ground-penetrating radar, and the reference time is:

[0019] ;

[0020] In the formula, T k For the first k Sampling time of each sensor, K The total number of the sensors;

[0021] The multi-source detection data is time-synchronized using the reference time to obtain time-synchronized multi-source detection data.

[0022] Spatial alignment processing is performed on the time-synchronized multi-source detection data to obtain fused detection data.

[0023] In one possible design, the subgrade surface image in the fused detection data is subjected to crack detection processing to obtain crack feature data of the specified subgrade, including:

[0024] The roadbed surface image from the fused detection data is input into a pre-trained crack detection model so as to obtain crack image information in the roadbed surface image based on the crack detection model; wherein, the crack detection model adopts a convolutional neural network model;

[0025] Crack feature data of the specified roadbed is calculated based on the crack image information; wherein, the crack feature data includes crack length and crack width, and the crack length is:

[0026] ;

[0027] In the formula, ( x i , y i ) is the center line of the crack image information. i The actual coordinates of each pixel. x i = u i · R , y i = v i ·R , ( u i , v i) is the center line of the crack image information. i The pixel coordinates of each pixel R The ratio between the pixel size and the actual size of the pixels in the crack image information is given by the following formula: x i+1 , y i+1 ) is the center line of the crack image information. i +1 pixel's actual coordinates n The total number of pixels in the center line of the crack image information;

[0028] The width of the crack is:

[0029] ;

[0030] In the formula, W i The center line of the crack image information is the first i The actual width of each pixel W i =△ u i ·R , △ u i The center line of the crack image information is the first i The pixel width of each pixel in the normal direction of the center line of the crack image information.

[0031] In one possible design, the RANSAC algorithm is used to fit the laser point cloud data only to a reference surface, and then the settlement characteristic data of the specified roadbed is calculated based on the reference surface position information obtained from the fitting.

[0032] In one possible design, the deep-seated defect feature data includes defect depth data and information on abnormal material areas; correspondingly, the ultrasonic detection data and electromagnetic wave reflection information in the fused detection data are subjected to deep-seated defect analysis processing to obtain the deep-seated defect feature data of the specified roadbed, including:

[0033] Obtain the ultrasonic echo delay data from the ultrasonic detection data, and calculate the defect depth data of the specified roadbed based on the ultrasonic detection data; wherein, the defect depth data is:

[0034] d=v·t' / 2;

[0035] In the formula, v The speed at which the ultrasonic wave emitted by the ultrasonic sensor propagates in the designated roadbed. t’ The echo time of the ultrasonic detection data;

[0036] The electromagnetic wave reflection information is subjected to Fourier transform analysis to obtain spectral feature information;

[0037] The material anomaly region information of the specified roadbed is obtained based on the spectral feature information.

[0038] In one possible design, a point cloud reconstruction algorithm is used to generate a 3D model of the disease.

[0039] In one possible design, after generating a 3D model of the defect, the method further includes:

[0040] A historical three-dimensional model of the roadbed is obtained, and based on the historical three-dimensional model and the current three-dimensional model of the roadbed, the roadbed is subjected to disease expansion trend prediction processing to obtain disease feature prediction data at a future specified time.

[0041] In a second aspect, the present invention provides a roadbed defect detection system for implementing a roadbed defect detection method as described in any one of the above claims; the roadbed defect detection system includes:

[0042] The data acquisition module is used to collect multi-source detection data of a specified roadbed in real time; wherein, the multi-source detection data includes roadbed surface images collected by a camera, laser point cloud data collected by a lidar, ultrasonic detection data collected by an ultrasonic sensor, and electromagnetic wave reflection information collected by a ground-penetrating radar;

[0043] The data fusion module is communicatively connected to the data acquisition module and is used to perform data fusion processing on the multi-source detection data to obtain fused detection data.

[0044] The road damage feature identification module is communicatively connected to the data fusion module. It is used to perform crack detection processing on the roadbed surface image in the fused detection data to obtain crack feature data of the specified roadbed; to perform surface settlement analysis processing on the laser point cloud data in the fused detection data to obtain settlement feature data of the specified roadbed; and to perform deep disease analysis processing on the ultrasonic detection data and electromagnetic wave reflection information in the fused detection data to obtain deep disease feature data of the specified roadbed.

[0045] The model building module is communicatively connected to the road damage feature identification module and is used to construct a three-dimensional model of the roadbed damage based on the crack feature data, the settlement feature data and the deep disease feature data.

[0046] The visualization processing module is communicatively connected to the model building module and is used to perform visualization processing on the three-dimensional model of the disease.

[0047] Thirdly, the present invention provides an electronic device, comprising:

[0048] Memory, used to store computer program instructions; and,

[0049] A processor is configured to execute the computer program instructions to perform the operation of a roadbed defect detection method as described in any of the preceding claims.

[0050] Fourthly, the present invention provides a computer program product, including a computer program or instructions, wherein the computer program or instructions, when executed by a computer, implement a roadbed defect detection method as described in any of the above claims.

[0051] The beneficial effects of this invention are as follows:

[0052] This invention discloses a method, system, electronic equipment, and product for detecting roadbed defects, enabling comprehensive detection of defects on both the surface and deep layers of the roadbed with a high degree of intelligence. Specifically, this invention employs a multi-sensor collaborative detection approach to collaboratively collect surface and deep defect data of a specified roadbed. Utilizing multi-source data fusion technology, the detection results from multiple sensors are comprehensively analyzed. Based on the fused detection data, defect features are extracted to obtain crack feature data, settlement feature data, and deep defect feature data. Finally, a three-dimensional defect model is constructed based on this data, thereby quantifying the surface and deep defect characteristics. In this process, this invention integrates multiple sensors, including cameras, lidar, ultrasonic sensors, and ground-penetrating radar, for defect detection, achieving comprehensive coverage of the detection range. Furthermore, during implementation, this invention identifies and classifies different types of defect features from the fused detection data, constructs a three-dimensional defect model based on this, and performs visualization processing, enabling real-time processing and visualization of the detection results. This provides maintenance personnel with intuitive analysis results, enhancing the level of intelligence and providing a solid guarantee for road safety management.

[0053] Other beneficial effects of the present invention will be further explained in the specific embodiments. Attached Figure Description

[0054] Figure 1 This is a flowchart of a roadbed defect detection method in one of the embodiments;

[0055] Figure 2 This is a block diagram of a roadbed defect detection system in one embodiment;

[0056] Figure 3 This is a block diagram of an electronic device in one embodiment. Detailed Implementation

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in conjunction with the accompanying drawings and descriptions of the embodiments or the prior art. Obviously, the following description of the structure of the accompanying drawings is only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0058] Example 1:

[0059] This embodiment discloses a method for detecting roadbed defects, which can be executed, but is not limited to, by a computer device or virtual machine with certain computing resources, such as a personal computer, smartphone, personal digital assistant or wearable device, or by a virtual machine.

[0060] like Figure 1 As shown, a method for detecting roadbed defects may include, but is not limited to, the following steps:

[0061] S1. Real-time acquisition of multi-source detection data for a designated roadbed; wherein, the multi-source detection data includes roadbed surface images acquired by a camera, laser point cloud data acquired by a lidar, ultrasonic detection data acquired by an ultrasonic sensor, and electromagnetic wave reflection information acquired by a ground-penetrating radar; it should be understood that the camera is a high-resolution camera, and the roadbed surface images acquired by it are used to assist in the detection of roadbed cracks; the ultrasonic detection data is used to indicate deep structural defects in the roadbed, such as cracks and cavities, and includes information such as sound wave propagation time and signal amplitude; the ground-penetrating radar acquires data by emitting electromagnetic wave signals to the roadbed and receiving reflected signals, and the electromagnetic wave reflection information includes its echo time and intensity, which can be used to reflect abnormal conditions of the roadbed's internal materials, such as changes in water content. During implementation, the multi-source detection data can be rapidly transmitted and processed through a 5G (fifth-generation mobile communication technology) network or edge computing devices, thereby facilitating rapid data analysis and processing.

[0062] In this embodiment, after acquiring the multi-source detection data of the specified roadbed, the multi-source detection data is further denoised. Specifically, for the roadbed surface image, image enhancement technology such as CLAHE (Contrast Limiting Adaptive Histogram Equalization) is applied to improve image quality. For the laser point cloud data, filtering algorithms such as radius filtering or statistical filtering are used to remove noise. For the ultrasonic detection data and the electromagnetic wave reflection information, Kalman filtering is used to smooth the data, thereby obtaining preprocessed multi-source detection data. Then, subsequent data processing and analysis are performed based on the preprocessed multi-source detection data.

[0063] S2. Perform data fusion processing on the multi-source detection data to obtain fused detection data. It should be noted that, due to temporal and spatial differences in data collected by different sensors, the multi-source detection data is fused before disease feature identification. The fused detection data can reflect the three-dimensional location and morphology of the disease in a unified spatial coordinate system. This fully utilizes the advantages of different sensors, ensures data consistency, avoids inaccurate disease location and extent corresponding to the detection results, further improves the accuracy of subsequent analysis, and enhances disease localization capabilities.

[0064] Specifically, in step S2, the multi-source detection data is subjected to data fusion processing to obtain fused detection data, including:

[0065] S201. Obtain the sampling time of each sensor in the multi-source detection data, and obtain a reference time based on the sampling time of each sensor; wherein, each sensor includes lidar, camera, ultrasonic sensor and ground-penetrating radar, and the reference time is:

[0066] ;

[0067] In the formula, T k For the first k Sampling time of each sensor, K The total number of the sensors;

[0068] It should be noted that this benchmark time calculation scheme can be directly applied to application scenarios where the sampling frequency of multi-source detection data is the same. If the sampling frequency of multi-source detection data is different, the detection data can be pre-processed by data interpolation or downsampling, and then the above calculation formula can be applied to calculate the benchmark time.

[0069] S202. Use the reference time to perform time synchronization processing on the multi-source detection data to obtain time-synchronized multi-source detection data; it should be noted that using the reference time to perform time synchronization processing on the multi-source detection data, that is, using the reference time to perform time alignment processing on all detection data, can ensure the consistency of the detection data in time.

[0070] It should be noted that each sensor records a timestamp when collecting data, indicating the sampling time at which the data is collected. In this embodiment, a reference time is obtained based on the sampling time of each sensor, and the reference time is used to perform time synchronization processing on the multi-source detection data. That is, the difference between the sampling time of the multi-source detection data and the reference time is calculated respectively, and then these time differences are interpolated or compensated so that all detection data are unified to the same time point.

[0071] S203. Perform spatial alignment processing on the time-synchronized multi-source detection data to obtain fused detection data. It should be noted that spatial alignment processing involves transforming the coordinate systems of multiple sensors to unify the various detection data into a global coordinate system. The representation of the detection data of a certain sensor in the preset global coordinate system is as follows: P = R 1· P raw + R 2, where, R 1 represents the rotation matrix, used to describe the orientation change of the coordinate system of a certain sensor relative to the global coordinate system. R 2 represents the translation matrix, used to describe the displacement between the origin of the coordinate system where a sensor is located and the origin of the global coordinate system. P raw This represents the representation of a sensor's detection data in its own coordinate system.

[0072] S3. Perform crack detection processing on the roadbed surface image in the fused detection data to obtain crack feature data of the specified roadbed.

[0073] Specifically, in step S3 of this embodiment, crack detection processing is performed on the roadbed surface image in the fused detection data to obtain crack feature data of the specified roadbed, including:

[0074] S301. Input the subgrade surface image from the fused detection data into a pre-trained crack detection model to obtain crack image information in the subgrade surface image based on the crack detection model; wherein, the crack detection model adopts a convolutional neural network model; specifically, in the process of image processing by the convolutional neural network, features such as crack edges and textures can be extracted in advance, and then downsampling is performed to retain the main features and reduce computational complexity. Then, the ReLU activation function is used to enhance the nonlinear expression ability of the model, and finally the crack image information is obtained, which is output in the form of a binary segmentation map to facilitate subsequent calculations such as crack length.

[0075] S302. Calculate the crack feature data of the designated roadbed based on the crack image information; wherein, the crack feature data includes crack length and crack width, and the crack length is:

[0076] ;

[0077] In the formula, ( x i , y i ) is the center line of the crack image information.i The actual coordinates of each pixel. x i = u i · R , y i = v i ·R , ( u i , v i ) is the center line of the crack image information. i The pixel coordinates of each pixel R The ratio between the pixel size and the actual size of the pixels in the crack image information is given by the following formula: x i+1 , y i+1 ) is the center line of the crack image information. i +1 pixel's actual coordinates n The total number of pixels in the center line of the crack image information;

[0078] The width of the crack is:

[0079] ;

[0080] In the formula, W i The center line of the crack image information is the first i The actual width of each pixel W i =△ u i ·R , △ u i The center line of the crack image information is the first i The width of a pixel in the normal direction of the center line of the crack image information. It should be noted that, for each pixel on the center line of the crack image information, the crack width in the normal direction is calculated, and the widths of all pixels are summed to obtain the crack width, thereby simplifying the calculation of the crack width.

[0081] Furthermore, the crack feature data may also include crack depth, wherein the crack depth is:

[0082] H= △ I / (r·cosθ );

[0083] In the formula, △ IThe light intensity variation information of the crack image information. r The preset optical constants, θ The angle of incidence of light from the camera is denoted as .

[0084] S4. Perform surface settlement analysis on the laser point cloud data in the fused detection data to obtain the settlement characteristic data of the specified roadbed.

[0085] Specifically, in step S4 of this embodiment, the RANSAC algorithm (RAndom SAmple Consensus) is used to fit the laser point cloud data only to the reference surface, and then the offset depth, that is, the settlement characteristic data of the specified roadbed, is calculated based on the reference surface position information obtained by fitting.

[0086] S5. Perform in-depth defect analysis on the ultrasonic detection data and electromagnetic wave reflection information in the fused detection data to obtain the in-depth defect characteristic data of the specified roadbed.

[0087] Specifically, in this embodiment, the deep-seated defect feature data includes defect depth data and material anomaly area information; correspondingly, in step S5, the ultrasonic detection data and electromagnetic wave reflection information in the fused detection data are subjected to deep-seated defect analysis processing to obtain the deep-seated defect feature data of the specified roadbed, including:

[0088] S501. Obtain the ultrasonic echo delay data from the ultrasonic detection data, and calculate the defect depth data of the specified roadbed based on the ultrasonic detection data; wherein, the defect depth data is:

[0089] d=v·t' / 2;

[0090] In the formula, v The speed at which the ultrasonic wave emitted by the ultrasonic sensor propagates in the designated roadbed. t’ The echo time of the ultrasonic detection data is the time difference between the emission of the ultrasonic wave and the receipt of the ultrasonic detection data. This parameter is recorded by the ultrasonic sensor.

[0091] S502. Perform Fourier transform analysis on the electromagnetic wave reflection information to obtain spectral feature information; wherein, the spectral feature information is:

[0092] ;

[0093] In the formula, f ( t The electromagnetic wave reflection information is time. t The function, ωThe frequency of the electromagnetic wave reflection information, i It is the symbol for imaginary numbers.

[0094] It should be understood that the electromagnetic wave reflection signal is a time-domain signal, and the obtained spectral feature information is a frequency-domain signal, which includes the spectral features of the electromagnetic wave reflection signal. Anomalies such as cracks and cavities inside the foundation material will cause changes in the spectral features of the electromagnetic wave reflection signal (such as a decrease in frequency peak value or amplitude). Based on this, the material anomalies of the specified roadbed can be identified through the spectral feature information.

[0095] S503. Obtain the material anomaly area information of the designated roadbed based on the spectral feature information. Specifically, since material anomalies cause a decrease in the amplitude of reflected signals, during implementation, spectral change data in the spectral feature information can be acquired to confirm the material anomaly. The material anomaly depth can be calculated based on the echo time of the electromagnetic wave reflected signal and the propagation speed of the electromagnetic wave transmitted by the ground-penetrating radar in the designated roadbed (the calculation method is the same as the calculation of the disease depth). Furthermore, the spatial range of the anomaly area can be located by combining the scanning path of the ground-penetrating radar.

[0096] It should be noted that, in this embodiment, after obtaining the crack feature data, the settlement feature data, and the deep disease feature data, the above analysis results can be further integrated into a report in HTML (HyperText Markup Language) or other formats, which makes it easier for users to view and provides a reference for maintenance decisions.

[0097] S6. Based on the crack feature data, the settlement feature data, and the deep disease feature data, a three-dimensional model of the specified roadbed is constructed. It should be understood that, based on the three-dimensional model, information such as the location, depth, extent, and type of the disease on the specified roadbed can be obtained. Specifically, in this embodiment, the three-dimensional model of the specified roadbed is reconstructed in advance based on the laser point cloud data, ultrasonic detection data, and electromagnetic wave reflection information. A point cloud interpolation algorithm is used to generate the roadbed surface, and the crack feature data is superimposed onto the three-dimensional model as the surface texture features. Then, based on the deep disease feature data, a three-dimensional structure of the disease area is constructed using voxels or parametric geometric models (such as spheres or cuboids). Finally, the three-dimensional structure of the disease area and the settlement feature data are integrated into the three-dimensional model to form a complete three-dimensional model of the disease.

[0098] Specifically, in step S6 of this embodiment, a three-dimensional model of the disease is generated using point cloud reconstruction algorithms such as the Poisson Surface Reconstruction algorithm, based on the crack feature data, the settlement feature data, and the deep disease feature data.

[0099] Existing technologies generally lack the function of intelligently predicting the future development trend of road defects based on their current state. Furthermore, existing technologies typically only provide single-detection results, failing to comprehensively assess historical data, defect expansion trends, and the overall condition of the roadbed, thus making it difficult to provide a comprehensive basis for road maintenance decisions. Therefore, in step S6 of this embodiment, after generating the three-dimensional defect model, the method further includes:

[0100] A. Obtain a historical 3D model of the defects of the designated roadbed, and based on the historical 3D model and the defect 3D model, perform defect expansion trend prediction processing on the designated roadbed to obtain defect feature prediction data for a future specified time. Specifically, in this embodiment, a time series prediction model (such as LSTM) can be used to perform defect expansion trend prediction processing on the designated roadbed, and the defect feature prediction data may include data such as crack propagation rate and settlement aggravation trend.

[0101] In this embodiment, the disease change trend can be further displayed in a three-dimensional view based on the result of the disease expansion trend prediction processing of the specified roadbed. For example, dashed lines can be used in the three-dimensional view to mark the disease feature prediction data at a specified future time.

[0102] It should be noted that this embodiment, by combining a time series analysis model, enables the prediction of the trend of disease expansion, which can further provide early warning for road maintenance and assist in the formulation of maintenance plans.

[0103] S7. Visualize the three-dimensional model of the disease. Specifically, by visualizing the three-dimensional model of the disease, an interactive three-dimensional view can be generated, displaying information such as the location of the disease.

[0104] This embodiment enables comprehensive detection of surface and deep roadbed defects, achieving a high level of intelligence. Specifically, it employs a multi-sensor collaborative detection method to collaboratively collect surface and deep defect data of a designated roadbed. Utilizing multi-source data fusion technology, the detection results from multiple sensors are comprehensively analyzed. Based on the fused detection data, defect features are extracted to obtain crack feature data, settlement feature data, and deep defect feature data. Finally, a three-dimensional defect model is constructed based on this data, thereby quantifying the surface and deep defect characteristics. In this process, this embodiment integrates multiple sensors, including cameras, lidar, ultrasonic sensors, and ground-penetrating radar, to achieve comprehensive coverage of the detection range. Furthermore, during implementation, this embodiment identifies and classifies different types of defect features from the fused detection data, constructs a three-dimensional defect model based on this, and performs visualization processing. This enables real-time processing and visualization of the detection results, providing maintenance personnel with intuitive analysis results, enhancing the level of intelligence, and providing a solid guarantee for road safety management.

[0105] This embodiment can be widely used in scenarios such as regular maintenance of highway subgrade, health monitoring of high-speed railway subgrade, detection of airport runway defects, and assessment and diagnosis of bridge foundation structures.

[0106] Example 2:

[0107] This embodiment discloses a roadbed defect detection system for implementing the roadbed defect detection method in Embodiment 1; such as Figure 2 As shown, the roadbed defect detection system includes:

[0108] The data acquisition module is used to collect multi-source detection data of a specified roadbed in real time; wherein, the multi-source detection data includes roadbed surface images collected by a camera, laser point cloud data collected by a lidar, ultrasonic detection data collected by an ultrasonic sensor, and electromagnetic wave reflection information collected by a ground-penetrating radar;

[0109] The data fusion module is communicatively connected to the data acquisition module and is used to perform data fusion processing on the multi-source detection data to obtain fused detection data.

[0110] The road damage feature identification module is communicatively connected to the data fusion module. It is used to perform crack detection processing on the roadbed surface image in the fused detection data to obtain crack feature data of the specified roadbed; to perform surface settlement analysis processing on the laser point cloud data in the fused detection data to obtain settlement feature data of the specified roadbed; and to perform deep disease analysis processing on the ultrasonic detection data and electromagnetic wave reflection information in the fused detection data to obtain deep disease feature data of the specified roadbed.

[0111] The model building module is communicatively connected to the road damage feature identification module and is used to construct a three-dimensional model of the roadbed damage based on the crack feature data, the settlement feature data and the deep disease feature data.

[0112] The visualization processing module is communicatively connected to the model building module and is used to perform visualization processing on the three-dimensional model of the disease.

[0113] It should be noted that the working process, working details and technical effects of the roadbed defect detection system provided in this embodiment 2 can be found in embodiment 1, and will not be repeated here.

[0114] Example 3:

[0115] Based on Embodiment 1 or 2, this embodiment discloses an electronic device, which may be a smartphone, tablet computer, laptop computer, or desktop computer, etc. The electronic device may be referred to as a user terminal, portable terminal, desktop terminal, etc. Figure 3 As shown, the electronic device includes:

[0116] Memory, used to store computer program instructions; and,

[0117] A processor is used to execute the computer program instructions to perform the operation of a roadbed defect detection method as described in any of Embodiment 1.

[0118] Specifically, processor 301 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 301 may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). Processor 301 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 301 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen.

[0119] The memory 302 may include one or more computer-readable storage media, which may be non-transitory. The memory 302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 302 are used to store at least one instruction, which is executed by the processor 301 to implement the roadbed defect detection method provided in Embodiment 1 of this application.

[0120] In some embodiments, the terminal may also optionally include a communication interface 303 and at least one peripheral device. The processor 301, memory 302, and communication interface 303 can be connected via a bus or signal line. Each peripheral device can be connected to the communication interface 303 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 304, a display screen 305, and a power supply 306.

[0121] The communication interface 303 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 301 and the memory 302. In some embodiments, the processor 301, the memory 302, and the communication interface 303 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 301, the memory 302, and the communication interface 303 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0122] The radio frequency (RF) circuit 304 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 304 communicates with communication networks and other communication devices via electromagnetic signals.

[0123] Display screen 305 is used to display the UI (User Interface). The UI may include any combination of graphics, text, icons, and video.

[0124] Power supply 306 is used to supply power to various components in electronic devices.

[0125] Example 4:

[0126] Based on any one of Embodiments 1 to 3, this embodiment discloses a computer program product, including a computer program or instructions, which, when executed by a computer, implements a roadbed defect detection method as described in any one of Embodiments 1. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0127] Obviously, those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, the present invention is not limited to any particular hardware and software combination.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting roadbed defects, characterized in that, include: Real-time acquisition of multi-source detection data of a designated roadbed; wherein, the multi-source detection data includes roadbed surface images acquired by a camera, laser point cloud data acquired by a lidar, ultrasonic detection data acquired by an ultrasonic sensor, and electromagnetic wave reflection information acquired by a ground-penetrating radar; The multi-source detection data is fused to obtain fused detection data; Crack detection processing is performed on the roadbed surface image in the fused detection data to obtain crack feature data of the specified roadbed; Surface settlement analysis is performed on the laser point cloud data in the fused detection data to obtain the settlement characteristic data of the specified roadbed; The ultrasonic detection data and electromagnetic wave reflection information in the fused detection data are subjected to in-depth disease analysis and processing to obtain the in-depth disease characteristic data of the specified roadbed. Based on the crack feature data, the settlement feature data, and the deep disease feature data, a three-dimensional disease model of the specified roadbed is constructed. The three-dimensional model of the disease is visualized. The RANSAC algorithm is used to fit the laser point cloud data only to the reference surface, and then the settlement characteristic data of the specified roadbed is calculated based on the reference surface position information obtained by fitting. The deep-seated defect characteristic data includes defect depth data and material anomaly area information; correspondingly, the ultrasonic detection data and electromagnetic wave reflection information in the fused detection data are subjected to deep-seated defect analysis processing to obtain the deep-seated defect characteristic data of the specified roadbed, including: Obtain the ultrasonic echo delay data from the ultrasonic detection data, and calculate the defect depth data of the specified roadbed based on the ultrasonic detection data; wherein, the defect depth data is: d=v·t' / 2； In the formula, v The speed at which the ultrasonic wave emitted by the ultrasonic sensor propagates in the designated roadbed. t’ The echo time of the ultrasonic detection data; The electromagnetic wave reflection information is subjected to Fourier transform analysis to obtain spectral feature information; The material anomaly region information of the specified roadbed is obtained based on the spectral feature information; The multi-source detection data is fused to obtain fused detection data, including: The sampling time of each sensor in the multi-source detection data is obtained, and a reference time is obtained based on the sampling time of each sensor; wherein each sensor includes lidar, camera, ultrasonic sensor and ground-penetrating radar, and the reference time is: ； In the formula, T k For the first k Sampling time of each sensor, K The total number of the sensors; The multi-source detection data is time-synchronized using the reference time to obtain time-synchronized multi-source detection data. Spatial alignment processing is performed on the time-synchronized multi-source detection data to obtain fused detection data; The roadbed surface image in the fused detection data is subjected to crack detection processing to obtain crack feature data of the specified roadbed, including: The roadbed surface image from the fused detection data is input into a pre-trained crack detection model so as to obtain crack image information in the roadbed surface image based on the crack detection model; wherein, the crack detection model adopts a convolutional neural network model; Crack feature data of the specified roadbed is calculated based on the crack image information; wherein, the crack feature data includes crack length and crack width, and the crack length is: ； In the formula, ( x i , y i ) is the center line of the crack image information. i The actual coordinates of each pixel. x i = u i ·R , y i = v i ·R , ( u i , v i ) is the center line of the crack image information. i The pixel coordinates of each pixel R The ratio between the pixel size and the actual size of the pixels in the crack image information is given by the following formula: x i+1 , y i+1 ) is the center line of the crack image information. i +1 pixel's actual coordinates n The total number of pixels in the center line of the crack image information; The width of the crack is: ； In the formula, W i The center line of the crack image information is the first i The actual width of each pixel W i =△ u i ·R , △ u i The center line of the crack image information is the first i The pixel width of each pixel in the normal direction of the center line of the crack image information.

2. The method for detecting roadbed defects according to claim 1, characterized in that, A three-dimensional model of the disease is generated using a point cloud reconstruction algorithm.

3. The method for detecting roadbed defects according to claim 1, characterized in that, After generating the three-dimensional model of the disease, the method further includes: A historical three-dimensional model of the roadbed is obtained, and based on the historical three-dimensional model and the current three-dimensional model of the roadbed, the roadbed is subjected to disease expansion trend prediction processing to obtain disease feature prediction data at a future specified time.

4. A roadbed defect detection system, characterized in that, A method for detecting roadbed defects as described in any one of claims 1 to 3; the roadbed defect detection system comprises: The data acquisition module is used to collect multi-source detection data of a specified roadbed in real time; wherein, the multi-source detection data includes roadbed surface images collected by a camera, laser point cloud data collected by a lidar, ultrasonic detection data collected by an ultrasonic sensor, and electromagnetic wave reflection information collected by a ground-penetrating radar; The data fusion module is communicatively connected to the data acquisition module and is used to perform data fusion processing on the multi-source detection data to obtain fused detection data. The road damage feature identification module is communicatively connected to the data fusion module. It is used to perform crack detection processing on the roadbed surface image in the fused detection data to obtain crack feature data of the specified roadbed; to perform surface settlement analysis processing on the laser point cloud data in the fused detection data to obtain settlement feature data of the specified roadbed; and to perform deep disease analysis processing on the ultrasonic detection data and electromagnetic wave reflection information in the fused detection data to obtain deep disease feature data of the specified roadbed. The model building module is communicatively connected to the road damage feature identification module and is used to construct a three-dimensional model of the roadbed damage based on the crack feature data, the settlement feature data and the deep disease feature data. The visualization processing module is communicatively connected to the model building module and is used to perform visualization processing on the three-dimensional model of the disease.

5. An electronic device, characterized in that, include: Memory is used to store computer program instructions; as well as, A processor is configured to execute the computer program instructions to perform the operation of a roadbed defect detection method as described in any one of claims 1 to 3.

6. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or the instructions are executed by the computer, they implement a method for detecting roadbed defects as described in any one of claims 1 to 3.

Images