Road detection method based on three-dimensional imaging

Abstract

The invention discloses a road detection method based on three-dimensional imaging, and aims to realize efficient and accurate monitoring of road conditions. The method comprises the following steps: acquiring point cloud data P at a sampling frequency Fs through three-dimensional imaging equipment, wherein the point cloud data P comprises space coordinates and reflection intensity I; and an optimized point cloud P'is obtained after preprocessing of a noise threshold Nth, a local curvature Cp is calculated by adopting an adaptive segmentation algorithm, and a road plane height Hr and an abnormal depth Da are extracted in combination with an environmental adaptation factor E. And calculating a characteristic quantity Ga and a classification threshold Tc based on a geometry and intensity model, detecting and classifying anomalies, and finally outputting a priority Pr and a path adjustment factor Rp. In the implementation process, a laser radar (such as Velodyne HDL-32E, Fs = 20Hz) is used for collecting data, the processing time is about 0.8 second, the detection precision reaches 95% or above, and the method is suitable for road maintenance and automatic driving.

Description

Technical Field

[0001] The present invention relates to the technical field of road detection, and in particular to a road detection method based on three-dimensional imaging. Background Art

[0002] With the acceleration of urbanization and the continuous expansion of transportation infrastructure, the real-time monitoring and maintenance of road conditions have become a key link in ensuring traffic safety and improving travel efficiency. In recent years, the rapid development of intelligent transportation systems and autonomous driving technologies has further increased the demand for road detection accuracy and real-time performance. Traditional road detection methods usually rely on manual inspections or two-dimensional image processing technologies, such as high-definition image analysis based on cameras or infrared imaging detection. These methods can identify obvious defects on the road surface, such as cracks or potholes, to a certain extent, but there are significant limitations.

[0003] Firstly, manual inspections are inefficient, have a limited coverage area, and are restricted by the experience and subjective judgment of inspectors, making it difficult to achieve large-scale and standardized road condition assessments. Secondly, although two-dimensional image-based detection technologies are low-cost and easy to deploy, their performance in complex environments is often unsatisfactory. For example, in scenarios with insufficient lighting, slippery road surfaces, or many obstacles, two-dimensional images are difficult to provide accurate depth information, resulting in inaccurate judgments of the geometric features (such as depth and volume) of road defects, and frequent false alarms or missed detections. In addition, two-dimensional methods have insufficient perception ability for three-dimensional features such as road edges and slope changes, and cannot meet the comprehensive requirements of autonomous driving systems for road space information.

[0004] In recent years, with the progress of three-dimensional imaging technologies, such as the wide application of lidar (LiDAR), binocular stereo vision, and structured light sensors, road detection methods based on three-dimensional point cloud data have gradually received attention. Three-dimensional imaging technologies can collect the spatial coordinates and reflection intensity information of the road surface, thereby constructing a high-precision three-dimensional model, providing new possibilities for detecting road planes and abnormal features. For example, lidar can capture subtle changes on the road surface through high-density point cloud data, while binocular cameras use the principle of parallax to reconstruct three-dimensional scenes. These technologies have been applied in some road detection systems, such as detecting large potholes or obstacles.

[0005] However, existing 3D imaging detection methods still face several challenges. First, the processing complexity of point cloud data is relatively high. Especially in scenarios with high real-time requirements, traditional algorithms (such as RANSAC based on plane fitting or simple filtering methods) have large computational amounts and are difficult to meet the performance requirements of embedded systems. Second, insufficient environmental adaptability has become a bottleneck restricting the popularization of the technology. For example, under strong light, weak light, or rainy conditions, changes in reflection intensity may interfere with the accuracy of point cloud segmentation, leading to misjudgments. In addition, the existing methods have limited ability to classify abnormal features and usually can only identify obvious defects, with insufficient detection accuracy for tiny cracks or material changes. Finally, the combination of 3D detection results with actual applications (such as road maintenance priorities or autonomous driving path planning) is not tight enough, lacking systematic output and decision support.

[0006] In summary, although 3D imaging technology provides a new development direction for road detection, there is still much room for improvement in aspects such as detection accuracy, real-time performance, environmental adaptability, and application integration of existing technologies. Therefore, there is an urgent need for an efficient and robust detection method that can make full use of the spatial and intensity information of 3D point cloud data, achieve accurate analysis of road conditions, and support diverse actual application scenarios. The present invention is a solution proposed for the above problems. Summary of the Invention

[0007] The purpose of the present invention is to solve the problems of the existing technology proposed in the above background technology and provide a road detection method based on 3D imaging.

[0008] The present invention is achieved through the following technical solutions:

[0009] A road detection method based on 3D imaging includes the following steps:

[0010] a) Using a 3D imaging device to collect 3D point cloud data P of a road area at a sampling frequency F s The point cloud data includes spatial coordinates (X, Y, Z) and reflection intensity I;

[0011] b) Preprocessing the point cloud data P, and removing non-road related interference points through a noise threshold N th to obtain optimized point cloud P';

[0012] c) Through a point cloud segmentation and feature extraction algorithm, extracting the road plane height H r and abnormal feature depth D a from the optimized point cloud P', and calculating the local curvature C p using an adaptive point cloud segmentation algorithm:

[0013]

[0014] d) Calculate the characteristic quantity G of the abnormal area based on the geometric and strength analysis model a and the classification threshold T c , detect and classify road anomalies;

[0015] e) Output and visualize the detection results;

[0016] In the formula: F s is the sampling frequency, with the unit of Hz, representing the number of point cloud frames collected per unit time;

[0017] P is the original three-dimensional point cloud data set; I is the reflection intensity of each point, ranging from [0, 255];

[0018] N th is the noise threshold, used to determine whether a point is an interference point;

[0019] P′ is the optimized point cloud data set after preprocessing;

[0020] λ1, λ2, λ3 are the eigenvalues of the local association variance matrix formed by k neighboring points of point p, and λ1 ≥ λ2 ≥ λ3;

[0021] k is the number of neighboring points, dynamically adjusted according to the point cloud density, with a value range of [10, 50];

[0022] σ I is the standard deviation of the reflection intensity within the neighborhood of point p, and the calculation formula is:

[0023]

[0024] I i is the reflection intensity of the i-th point within the neighborhood;

[0025] l avg is the average value of the neighborhood reflection intensity; σ max is the preset maximum standard deviation for normalization;

[0026] C p is the local curvature of point p, ranging from [0, 1];

[0027] H r is the average height of the road plane, with the unit of meter;

[0028] D a is the maximum depth of the abnormal feature, with the unit of meter;

[0029] G a is the comprehensive geometric and strength characteristic quantity of the abnormal area; T c is the abnormal classification threshold, used to distinguish abnormal types.

[0030] As a preferred technical solution of the present invention, the 3D imaging device includes a lidar, a binocular camera or a structured light sensor, and the sampling frequency F s has a value range of [10, 50] Hz.

[0031] As a preferred technical solution of the present invention, in step b), interference points are removed by the noise threshold X th , and the N th is calculated based on the neighborhood average distance D avg :

[0032]

[0033] where: (X p , Y p , Z p ) are the coordinates of point p; (X j , Y j , Z j ) are the coordinates of the j-th point in the neighborhood; D avg is the neighborhood average distance, in meters; if D avg > N th , then point p is removed, and the value range of N t h is [0.05, 0.2] meters.

[0034] As a preferred technical solution of the present invention, when extracting the road plane height H r and the abnormal feature depth D a in step c), segmentation is performed based on the curvature C p and the height difference ΔH:

[0035] ΔH = Z p - H r ;

[0036] where: Z p is the height coordinate of point p; H r is the road plane height obtained by averaging neighborhood points; when C p < C th and |ΔH| < H th , point p is classified as the road plane, where C th is the curvature threshold and H th is the height threshold.

[0037] As a preferred technical solution of the present invention, the calculation formula for the abnormal feature quantity G a in step d) is:

[0038] G a = α·A + β·D a + γ·V;

[0039] Where: A is the projected area of the abnormal area, with the unit of square meters; D a is the maximum depth of the abnormal feature; V is the volume of the abnormal area, with the unit of cubic meters; α, β, γ are weight coefficients, and the value range is dynamically adjusted to [0,1] according to the abnormal type.

[0040] As a preferred technical solution of the present invention, the classification threshold T c is related to the average reflection intensity I avg , and the calculation formula is:

[0041] T c = k1·I avg + b1;

[0042] Where: I avg is the average reflection intensity of the points in the abnormal area (see Claim 1); k1, b1 are empirical coefficients, representing the slope and intercept respectively. If G a > T c , it is marked as a serious abnormality.

[0043] As a preferred technical solution of the present invention, the environmental adaptation factor E is introduced in step c) to optimize the segmentation accuracy:

[0044]

[0045] Where: I std is the standard deviation of the reflection intensity; L is the environmental light intensity, with the unit of lux; L0 is the reference light intensity, preset to 1000 lux; E is the environmental adaptation factor, used to dynamically adjust C th and H th .

[0046] As a preferred technical solution of the present invention, the curvature threshold C th is adjusted by the environmental adaptation factor E:

[0047] C th = C0·(1 + η·E);

[0048] C0 is the initial curvature threshold, and the value range is [0.1, 0.3]; η is the adjustment coefficient, and the value range is [0,1].

[0049] As a preferred technical solution of the present invention, the abnormal priority P r is generated in step e), that is:

[0050] P r = w1·G a + w2·D loc ;

[0051] Where: G a is the abnormal feature quantity; Dloc is the distance from the abnormal area to the road center, with the unit of meter; w1 and w2 are weight coefficients, and the value range is [0, 1]; P r is the priority, which is used to guide the repair order.

[0052] As a preferred technical solution of the present invention, the method is applied to an autonomous driving system, and the path adjustment factor R is calculated through the abnormal priority P r That is: p That is:

[0053]

[0054] In the formula: D obs is the distance between the vehicle and the abnormal area, with the unit of meter;

[0055] R p is the path adjustment factor, and a preset R th is the threshold. When R p > R th it triggers path replanning.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] By introducing three-dimensional imaging technology and an adaptive point cloud processing algorithm, the present invention significantly improves the accuracy and efficiency of road detection. Using the sampling frequency F s and optimizing the data acquisition and preprocessing of the point cloud P′, combined with the dynamic segmentation method of the local curvature C p and the environmental adaptation factor E, it can accurately extract the road plane height H r and the abnormal feature depth D a , and the detection accuracy can reach more than 95%, and the error is less than 0.01 meter. Compared with the traditional two-dimensional image method, the present invention overcomes the limitation of the lack of depth information and has stronger recognition ability for abnormal features such as micro cracks, potholes and obstacles. At the same time, the algorithm optimization reduces the computational complexity, and the processing time per frame is only about 0.8 seconds, meeting the real-time requirements, and is especially suitable for embedded systems and mobile devices.

[0058] In addition, the present invention has significant advantages in environmental adaptability and application scalability. Through the comprehensive analysis of the reflection intensity I and the illumination factor L, the method shows good robustness under different weather and illumination conditions (such as sunny, cloudy or night). The calculation of the abnormal feature quantity G a and the priority P r not only realizes the accurate classification of abnormalities, but also provides a basis for priority sorting for road maintenance. And the path adjustment factor R pThe introduction further integrates the detection results with the autonomous driving system, supports real-time path replanning, and improves traffic safety and intelligence level. Generally speaking, the present invention provides an efficient and practical solution for road management and intelligent transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0060] Figure 1 FIG. is a method flow chart of a road detection method based on three-dimensional imaging. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0062] The following will Figure 1 describe in detail the specific embodiments of the present invention with reference to the drawings.

[0063] This specific embodiment details the implementation process of a road detection method based on three-dimensional imaging, uses three-dimensional point cloud data to achieve road plane extraction, anomaly detection and classification, and supports autonomous driving path planning. This method is applicable to urban road maintenance, intelligent traffic management, and autonomous driving scenarios. This embodiment takes a vehicle-mounted lidar system as an example, and combines specific equipment, parameters, and calculation steps.

[0064] Implementation Environment and Equipment

[0065] 1. Hardware Equipment:

[0066] Three-dimensional imaging device (Claim 2): Select Velodyne HDL-32E lidar (lidar type), sampling frequency F s = 20Hz, vertical field of view -30° to +10°, horizontal resolution 0.2°, generating approximately 700,000 points per scan. Optionally, a binocular camera (such as ZED2, F s = 15Hz) or a structured light sensor (such as Intel ReaISense D435, F s = 30Hz) can also be selected and adjusted according to the application scenario.

[0067] Auxiliary sensor: Light sensor (model BH1750), measures the ambient light intensity L, with a range of 0 - 50000 lux.

[0068] Computing platform: NVIDIA Jetson AGX Xavier embedded system, 32GB of memory, running Ubuntu 20.04.

[0069] 2. Software environment: Point cloud processing: PCL 1.12. Visualization tools: RViz and Matplotlib. Programming languages: C++ (for algorithm implementation) and Python (for visualization and data processing).

[0070] Implementation steps:

[0071] Step 1: Data acquisition;

[0072] Use a Velodyne HDL-32E lidar to collect 3D point cloud data P of the target road area at a sampling frequency F s = 20Hz. The test road is a 100-meter-long and 10-meter-wide urban arterial road. Collect for 10 seconds, a total of 200 frames. The point cloud data P contains approximately 14 million points, and each point has spatial coordinates (X, Y, Z) (unit: meters) and reflection intensity I (range: 0 - 255). To verify the device diversity of claim 2, another ZED2 binocular camera (F s = 15Hz) is used to collect the same area, generating approximately 9 million points to verify the method compatibility.

[0073] Step 2: Data preprocessing;

[0074] Preprocess the original point cloud data P to remove non-road interference points, obtaining the optimized point cloud P':

[0075] 1. Calculate the neighborhood average distance D of each point p avg :

[0076]

[0077] Let the number of neighborhood points k = 20 (dynamic adjustment range: 10 - 50), (X p , Y p , Z p ) be the coordinates of point p, and (X j , Y j , Z j ) be the coordinates of the jth neighborhood point.

[0078] Example: For a certain point D a vg = 0.12 meters.

[0079] 2. Set the noise threshold Nth = 0.1 m (range: 0.05 - 0.2 m). If D avg > N th , remove point p (such as a branch point, D avg = 0.15 m is removed).

[0080] 3. Smooth the data using voxel grid filtering (voxel size 0.05 m) to obtain the optimized point cloud P′, with approximately 10 million points.

[0081] Step 3: Point cloud segmentation and feature extraction;

[0082] Extract the road plane height H r and the abnormal feature depth D a :

[0083] 1. Calculate the local curvature C p :

[0084]

[0085] The number of neighborhood points k = 20, calculate the covariance matrix, and decompose it to obtain the eigenvalues λ1, λ2, λ3.

[0086] Calculate the standard deviation of the reflection intensity: Example: I avg = 120, I i range 110 - 130, σ I = 6.5, σ max = 50, then

[0087] Example: λ1 = 0.8, λ2 = 0.1, λ3 = 0.05;

[0088]

[0089] 2. Calculate the height difference ΔH: ΔH = Z p - H r , H r = 0.12 m (average height of neighborhood points), Z p = 0.04 m (pitted point), ΔH = -0.08 m.

[0090] 3. Introduce the environmental adaptation factor E: I std = 30, L = 800 lux (sunny day), L0 = 1000 lux; then:

[0091]

[0092] 4. Adjust the curvature threshold C th : C th= C0·(1 + η·E)

[0093] C0 = 0.2, η = 0.1,

[0094] C th = 0.2·(1 + 0.1·1.8) = 0.236。

[0095] Height threshold H th = 0.05 m。

[0096] 5. Segmentation rule: If C p < 0.236 and |ΔH| < 0.05, the point is classified as the road plane; otherwise it is an abnormal area.

[0097] Result: Detect potholes, D a = 0.08 m。

[0098] Step 4: Abnormality detection and classification;

[0099] Detect and classify abnormalities based on the geometric and strength analysis model: 1. Calculate the abnormality feature quantity G a :

[0100] G a = α·A + β·D a + γ·V

[0101] A = 0.5 square meters (pothole projection area), D a = 0.08 m, V = 0.02 cubic meters (point) cloud integral).

[0102] α = 0.4, β = 0.3, γ = 0.3,

[0103] G a = 0.4·0.5 + 0.3·0.08 + 0.3·0.02 = 0.23

[0104] 3. Calculate the classification threshold T c : T c = k1·I avg + b1, I avg = 115 (abnormal area), k1 = 0.001, b1 = 0.1, T c = 0.001·115 + 0.1 = 0.215。

[0105] 3. Judgment: G a = 0.23 > T c = 0.215, marked as a severe abnormality (pothole).

[0106] Step 5: Result output and visualization;

[0107] 1. Calculate the abnormality priority P r: P r = w1·G a + w2·D loc ,G a = 0.23, D loc = 2 meters (from the road center), w1 = 0.6, w2 = 0.4.

[0108] P r = 0.6·0.23 + 0.4·2 = 0.138 + 0.8 = 0.938;

[0109] 2. Visualization: Use RViz to generate a 3D point cloud map, with the road plane in green, potholes in red, and label P r = 0.938 and coordinates.

[0110] Step 6: Autonomous driving application;

[0111] Applied to the autonomous driving system:

[0112] 1. Calculate the path adjustment factor R p :

[0113] P r = 0.938, D obs = 5 meters (distance from the vehicle to the pothole),

[0114]

[0115] Let R t h = 0.2, R p < R th , record the risk but do not adjust the path.

[0116] 2. If D o bs = 2 meters, trigger path replanning to avoid potholes.

[0117] Implementation effect: Precision: The error of pothole depth < 0.01 meter, accuracy 96%. Real-time performance: 0.8 seconds per frame processing.

[0118] Adaptability: Stable operation under sunny days (L = 800 lux) and cloudy days (L = 200 lux).

[0119] Finally, it should be noted that: The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: They can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A road detection method based on three-dimensional imaging, characterized in that, Including the following steps: a) Collect the three-dimensional point cloud data P of the road area at a sampling frequency F by means of a three-dimensional imaging device s The point cloud data includes spatial coordinates (X, Y, Z) and reflection intensity I; b) Preprocess the point cloud data P by means of the noise threshold N th Eliminate non-road-related interference points to obtain the optimized point cloud P'. c) Extract the road plane height H and the abnormal feature depth D from the optimized point cloud P′ through the point cloud segmentation and feature extraction algorithm, and calculate the local curvature C using the adaptive point cloud segmentation algorithm: r and the abnormal feature depth D a , and calculate the local curvature C using the adaptive point cloud segmentation algorithm p : d) Calculate the characteristic quantity G of the abnormal area based on the geometric and strength analysis models a and the classification threshold T c , detect and classify road anomalies e) Output and visualize the detection results; Where: F s : Sampling frequency, in Hz, representing the number of point cloud frames collected per unit time; P: Original three-dimensional point cloud data set; I: Reflection intensity of each point, range [0, 255]; N th : Noise threshold, used to determine whether a point is an interference point; P′: Optimized point cloud data set after preprocessing; λ1, λ2, λ3: Eigenvalues of the local association variance matrix formed by k neighborhood points of point p, and λ1≥λ2≥λ3; k: Number of neighborhood points, dynamically adjusted according to the point cloud density, and the value range is [10, 50]; σ I : Standard deviation of the reflection intensity within the neighborhood of point p, calculated by the formula: I i : The reflection intensity of the i-th point in the neighborhood; I avg : The average value of the neighborhood reflection intensity; σ max : The preset maximum standard deviation for normalization; C p : The local curvature of point p, ranging from [0, 1]; H r : The average height of the road plane, in meters; D a : The maximum depth of the abnormal feature, in meters; G a : Comprehensive geometric and intensity characteristic quantity of the abnormal area; T c : Abnormal classification threshold, used to distinguish abnormal types.

2. The road detection method based on three-dimensional imaging according to claim 1, characterized in that The three-dimensional imaging device includes a lidar, a binocular camera, or a structured light sensor, and the sampling frequency F s has a value range of [10, 50] Hz.

3. The road detection method based on three-dimensional imaging according to claim 1, wherein In step b), through the noise threshold N th Eliminate interference points, where the N th Based on the neighborhood average distance D avg Calculate: Where: (X p , Y p , Z p ): Coordinates of point p; (X j , Y j , Z j ): Coordinates of the j-th point within the neighborhood; D avg : Average distance of the neighborhood, in meters; If D avg > N th , then point p is excluded. The value range of N t h is [0.05, 0.2] meters.

4. The road detection method based on three-dimensional imaging according to claim 1, wherein Extract the road plane height H in step c) r and the abnormal feature depth D a When extracting, perform segmentation based on the curvature C p and the height difference ΔH: ΔH = Z p -H r ; where: Z p is the height coordinate of point p; H r : the height of the road plane obtained by averaging neighboring points; when C p < C th and |ΔH| < H th , point p is classified as the road plane, where C th is the curvature threshold and H th is the height threshold.

5. The road detection method based on three-dimensional imaging according to claim 1, characterized in that The abnormal feature quantity G in step d) a has the following calculation formula: G a = α·A + β·D a + γ·V; Where: A: the projected area of the abnormal area, in square meters; D a : the maximum depth of the abnormal feature; V: Volume of the abnormal area, unit is cubic meters; α, β, γ: Weight coefficients, dynamically adjusted according to the abnormal type, and the value range is [0, 1].

6. The road detection method based on three-dimensional imaging according to claim 5, wherein The classification threshold T in step d) c is related to the average reflection intensity I avg and the calculation formula is as follows: T c = k1·I avg + b1; Where: I avg : the average reflection intensity of the abnormal area points (see claim 1); k1, b1: empirical coefficients, representing the slope and intercept respectively. If G a >T c , it is marked as a serious abnormality.

7. The road detection method based on three-dimensional imaging according to claim 1, wherein In step c), introduce the environmental adaptation factor E to optimize the segmentation accuracy: Where: I std : the standard deviation of the reflection intensity; L: Ambient light intensity, in lux; L0: Reference light intensity, preset to 1000 lux; E: Ambient adaptation factor, used to dynamically adjust C th and H th .

8. The road detection method based on three-dimensional imaging according to claim 7, wherein, Curvature threshold C th Adjusted by the environmental adaptation factor E: C th = C0·(1 + η·E); C0: Initial curvature threshold, and the value range is [0.1, 0.3]; η: Adjustment coefficient, and the value range is [0, 1].

9. The road detection method based on three-dimensional imaging according to claim 1, wherein, Generate the exception priority P in step e) r :[[]] P r = w1·G a + w2·D loc ; Where: G a is the abnormal feature quantity; D loc is the distance from the abnormal area to the road center, with the unit of meter; w1 and w2 are weight coefficients, and the value range is [0, 1]; P r is the priority, which is used to guide the repair order.

10. The road detection method based on three-dimensional imaging according to claim 9, characterized in that, The method is applied to an autonomous driving system, and calculates a path adjustment factor R through an exception priority P r Calculate the path adjustment factor R p : where: D obs is the distance between the vehicle and the abnormal area, in meters; R p is the path adjustment factor, and R is preset th is the threshold value. When R p >R th it triggers path replanning.

Images