Vision-based path center line detection method

Abstract

The invention discloses a path center line detection method based on vision, and belongs to the technical field of auxiliary driving. The method specifically comprises the following steps: S1, data acquisition and synchronization: acquiring visual data through a visual sensor, acquiring various data in a driving process through auxiliary data, and synchronizing an image frame and sensor data according to a milli-level timestamp; and S2, real-time detection of a path center line: fitting the path center line by extracting lane line feature data, and outputting a result at the same time. Compared with an existing path centerline detection method, the detection effect is evaluated through single data, the performance of the detection effect is evaluated through multiple data in the detection process, the evaluation result is more accurate, the larger the path centerline detection accuracy index is, the higher the accuracy of the detection result is, and meanwhile, the accuracy of the detection result is improved. And when the path midline detection accuracy index is smaller than the set threshold value of the path midline detection accuracy index, sending out a warning prompt that the accuracy of the detection result is too low.

Description

Technical Field

[0001] This invention relates to the field of driver assistance technology, and in particular to a vision-based method for detecting the centerline of a path. Background Technology

[0002] Currently, the most widely used sensors for obstacle detection are lidar and cameras, which can extract the shape features of obstacles. Lidar offers stable signal acquisition and high depth accuracy, but it is bulky and expensive. Camera sensors, on the other hand, can not only acquire geometric feature information of the target but also determine semantic information based on color, and are less expensive, but cannot directly acquire depth information. The Realsense D455 depth camera can capture not only color information but also directly acquire depth information of corresponding points, and is widely used in various industrial scenarios. Besides sensors, environmental characteristics also affect the application of this technology. For example, in agricultural production environments, there is a height difference between tree branches and grass, and the colors of trees and grass are similar, making misjudgment easy due to changes in lighting. Furthermore, tree shapes vary, and the shape of tree branches changes irregularly.

[0003] Current methods for detecting path midlines mainly fall into two categories: traditional image strategies and deep learning. However, existing path midline detection methods lack performance evaluation based on multiple data points during the detection process, making it impossible to obtain accurate evaluations of the detection results in real time. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a vision-based path centerline detection method; it can solve the problem that existing path centerline detection methods based on the above-mentioned methods lack performance evaluation of detection effect through multiple data of the detection process, and cannot obtain accurate evaluation of the detection effect in real time.

[0005] Technical Solution: To solve the above-mentioned technical problems, according to one aspect of the present invention, more specifically, a vision-based path centerline detection method, which specifically includes the following steps:

[0006] S1. Data Acquisition and Synchronization: Visual data is acquired through a visual sensor, and various data during the driving process are acquired through auxiliary data. At the same time, image frames and sensor data are synchronized with millisecond-level timestamps.

[0007] S2. Real-time detection of the path centerline: By extracting lane line feature data, the path centerline is fitted and the result is output simultaneously;

[0008] S3. Error detection judgment: By comparing with the real path, the detection is judged as erroneous. If it is judged as erroneous, the relevant data of the erroneous detection is recorded.

[0009] S4. Data Analysis and Processing: Calculate the path centerline detection accuracy index from the data including the total number of detections, the number of detection errors, the total detection mileage, and the detection error mileage during the path centerline detection process.

[0010] S5. Risk Assessment and Feedback: The accuracy of the detection results is evaluated through the path centerline detection accuracy index. At the same time, a threshold for the path centerline detection accuracy index is set. By comparing the path centerline detection accuracy index with the set threshold, it is determined whether to issue a warning.

[0011] Furthermore, in step S1, when collecting visual data, the vehicle-mounted camera continuously collects road images with a resolution of not less than 1280×720 at 30fps, and transmits the collected road images to the processor in real time. At the same time, the collected road images are preprocessed by grayscale conversion, Gaussian filtering, and perspective transformation.

[0012] Furthermore, in step S1, during data synchronization, the vehicle's real-time position, speed, and heading angle data are acquired in real time, and the image frames and sensor data are synchronized with millisecond-level timestamps.

[0013] Furthermore, in step S2, when extracting vehicle line feature data, a high and low threshold for edge detection is set, the lane line edge contour is extracted, candidate regions for straight line segments are detected, and invalid line segments with a length of less than 15 pixels or an angle deviating from the lane direction are filtered out.

[0014] Furthermore, in step S3, by comparing the actual road conditions with the data from the high-precision map, if the lateral offset distance exceeds 0.3 meters or the confidence level is lower than 0.8, it is determined to be a detection error. At this time, the timestamp of the detection error and the location of the detection error are recorded.

[0015] Furthermore, in step S4, during data analysis preprocessing, an interval duration evaluation index is obtained by comprehensively analyzing the interval between two adjacent error detections, the total detection time, and the time from the first use when each error occurred.

[0016]

[0017] in, As an index for evaluating interval duration, The time interval between each detection error and the previous detection error. To detect the total usage time, The time elapsed since the first use when each error is detected. This represents the number of errors detected.

[0018] Furthermore, in step S4, different error weights are assigned to each detection error based on the offset distance, as follows:

[0019]

[0020] in, This represents the error weight for each detected error.

[0021] Furthermore, in step S4, the path centerline detection accuracy index is obtained by comprehensively analyzing the total detection time, considering the evaluation index based on the interval duration, the error weight of each detection error, and the overall detection time.

[0022]

[0023] in, This is the accuracy index for path centerline detection. This is the error weighting coefficient, with a value of 0.6. This is the interval weighting coefficient, with a value of 0.4.

[0024] Furthermore, in step S5, the accuracy of the detection result is evaluated by the path centerline detection accuracy index. The higher the path centerline detection accuracy index, the higher the accuracy of the detection result, and vice versa. At the same time, when the path centerline detection accuracy index is less than the set threshold of the path centerline detection accuracy index, a warning reminder that the accuracy of the detection result is too low is issued.

[0025] Beneficial Effects: Compared to existing path centerline detection methods that evaluate detection performance using single data points, this invention assesses performance through multiple data points from the detection process, resulting in more accurate evaluations. It comprehensively analyzes the interval between two adjacent error detections, the total detection time, and the time elapsed since the first use of the system when each error occurred to obtain an interval evaluation index. Simultaneously, it assigns different error weights to each error based on its offset distance. Then, by comprehensively analyzing the interval evaluation index, the error weights of each error, and the total detection time, it obtains a path centerline detection accuracy index. A higher path centerline detection accuracy index indicates higher detection accuracy, and vice versa. Furthermore, when the path centerline detection accuracy index is lower than a set threshold, a warning is issued indicating that the detection accuracy is too low. Attached Figure Description

[0026] Figure 1 This is a flowchart illustrating the method. Detailed Implementation

[0027] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Example:

[0029] I. Implementation Scenario Description

[0030] This embodiment is applied to assisted driving scenarios on urban and suburban roads for passenger vehicles. The vehicle needs to achieve real-time centerline detection under typical road conditions such as two-way four-lane, one-way two-lane, and intersection turning, and is adaptable to sunny daytime, cloudy daytime, and low-light evening environments (without extreme weather such as heavy rain or fog). The vehicle configuration is as follows:

[0031] Visual sensors: Intel Realsense D455 depth camera (with both color image and depth data acquisition capabilities), forward-facing vehicle camera (1920×1080 resolution, supports 30fps continuous shooting).

[0032] Auxiliary sensors: GPS positioning module (positioning accuracy ±1m), IMU inertial measurement unit (collects vehicle speed and heading angle, data update frequency 100Hz);

[0033] Processor: NVIDIA Jetson AGX Xavier (supports real-time image preprocessing and data processing);

[0034] High-precision map: A road vector map covering the test area (including precise information such as lane line positions and road boundaries).

[0035] II. Detailed Implementation Steps

[0036] (a) S1: Data Acquisition and Synchronization

[0037] Visual data acquisition and preprocessing:

[0038] The forward-facing vehicle camera is activated to continuously capture road images at a frame rate of 30fps, with the image resolution set to 1920×1080 (meeting the requirement of not less than 1280×720), and transmitted in real time to the NVIDIA Jetson AGX Xavier processor.

[0039] The processor performs a preprocessing procedure on the acquired color images:

[0040] Grayscale conversion: The RGB image is converted to a grayscale image using a weighted average method, with weighting coefficients of R=0.299, G=0.587, and B=0.114;

[0041] Gaussian filtering: Set the Gaussian kernel size to 5×5, standard deviation... Filter image noise;

[0042] Perspective transformation: Based on the camera's intrinsic parameters (focal length f_x=1200px, f_y=1200px, principal point coordinates u_0=960px, v_0=540px) and extrinsic parameters (installation height 1.2m, pitch angle 15°), the image is transformed into a bird's-eye view, focusing on the road area 3-50m in front of the vehicle.

[0043] Auxiliary data collection:

[0044] The vehicle's longitude and latitude coordinates (i.e., real-time location) are obtained in real time through the GPS positioning module, and the vehicle's speed (unit: km / h) and heading angle (unit: °, increasing clockwise from true north as 0°) are collected through the IMU inertial measurement unit, and the auxiliary data acquisition frequency is coordinated with the image frame acquisition frequency.

[0045] Data synchronization:

[0046] A millisecond-precision timestamp (format: YYYY-MM-DD-HH-MM-SS-SSS) is added to each image frame and each set of auxiliary sensor data. The time synchronization module of the processor associates the image frame with the vehicle position, speed, and heading angle data corresponding to the same timestamp to ensure that the data time deviation is ≤1ms.

[0047] (ii) S2: Real-time detection of path centerline

[0048] Lane line feature extraction:

[0049] The Canny edge detection algorithm was used to extract lane line edges. The high threshold for edge detection was set to 200 and the low threshold to 100, preserving the edge contours of lane lines and road boundaries in the image.

[0050] Based on the Hough transform, candidate regions for straight line segments are detected, and the Hough space accumulation threshold is set to 150 to filter out potential lane line segments.

[0051] Perform invalid line segment filtering: remove short line segments with a length of less than 15 pixels, and at the same time calculate the angle between the line segment and the preset direction of the lane (based on the vehicle heading angle and road direction, the preset lane direction fluctuates within ±30°), and remove line segments whose angle deviates from the range.

[0052] Path midline fitting:

[0053] Cluster analysis was performed on the selected valid lane line segments to distinguish between left and right lane lines (based on the horizontal position and slope of the line segments in the image).

[0054] The left and right lane lines were fitted using the least squares method with quadratic polynomials, and the fitting formulas are as follows: (Where x is the horizontal pixel coordinate of the image, and y is the vertical pixel coordinate);

[0055] Calculate the centerline of the left and right lane lines: Under the same longitudinal coordinate y, take the x-coordinate of the left lane line. x-coordinate of the right lane line average The pixel coordinate sequence of the path centerline is obtained, and then converted into the path centerline in the world coordinate system through inverse perspective transformation, and output to the driver assistance control system in real time.

[0056] (iii) S3: Judgment of detection errors

[0057] Real path comparison benchmark: The lane centerline in the high-precision map is used as the real path benchmark. Combined with the real-time vehicle position obtained by the GPS positioning module, the lateral offset distance between the detected output path centerline and the real path is calculated (in world coordinate system, unit: m). At the same time, the confidence of the detection result is calculated through feature matching algorithm (between 0 and 1, the closer to 1, the higher the matching degree).

[0058] Error detection and data logging:

[0059] The judgment rule is set as follows: if the lateral offset distance is greater than 0.3 meters, or the confidence level is less than 0.8, it is judged as a detection error.

[0060] Record error-related data: Each time an error is detected, the current timestamp (millisecond precision), vehicle GPS location coordinates (longitude and latitude), lateral offset distance, and confidence level are automatically recorded and stored in the processor's local database.

[0061] (iv) S4: Data Analysis and Processing

[0062] Assume the duration of this test (total testing time). Seconds (1 hour), during which a total of The following table shows the relevant parameters for each error detected:

[0063]

[0064] Interval Duration Assessment Index calculate:

[0065]

[0066] (in (This is the attenuation coefficient, with a value of 0.001, used to indicate that the closer the error occurs to the end of the detection period, the greater the weight).

[0067] Calculate the summation term:

[0068]

[0069]

[0070]

[0071]

[0072]

[0073] Summation:

[0074]

[0075] calculate

[0076] Error weights distribute:

[0077] The weight of each error is determined based on the offset distance:

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] Path centerline detection accuracy index calculate:

[0084] formula:

[0085] (in , )

[0086] calculate

[0087] Calculate the first term:

[0088] Calculate the second term:

[0089] calculate

[0090] (v) S5: Risk Assessment and Feedback

[0091] Threshold setting: Based on the safety requirements of assisted driving, the threshold for the accuracy index of path centerline detection is set as follows: (This threshold has been calibrated through extensive real-vehicle testing to ensure that the detection accuracy meets driving safety requirements.)

[0092] Accuracy assessment and warning judgment:

[0093] This calculation yielded This indicates that the accuracy of the test results meets the requirements;

[0094] If during the detection process In such cases, the processor will immediately trigger an alert mechanism:

[0095] The instrument panel inside the vehicle displays a red warning icon, and at the same time, a voice announcement says, "Route detection accuracy is insufficient, please drive with caution."

[0096] Send signals to the driver assistance control system to limit the level of intervention of autonomous driving functions (such as reducing the adaptive cruise speed and enhancing driver takeover alerts).

[0097] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A vision-based path centerline detection method, characterized in that, Specifically, the following steps are included: S1. Data Acquisition and Synchronization: Visual data is acquired through a visual sensor, and various data during the driving process are acquired through auxiliary data. At the same time, image frames and sensor data are synchronized with millisecond-level timestamps. S2. Real-time detection of the path centerline: By extracting lane line feature data, the path centerline is fitted and the result is output simultaneously; S3. Error detection judgment: By comparing with the real path, the detection is judged as erroneous. If it is judged as erroneous, the relevant data of the erroneous detection is recorded. S4. Data Analysis and Processing: Calculate the path centerline detection accuracy index from the data including the total number of detections, the number of detection errors, the total detection mileage, and the detection error mileage during the path centerline detection process. S5. Risk Assessment and Feedback: The accuracy of the detection results is evaluated through the path centerline detection accuracy index. At the same time, a threshold for the path centerline detection accuracy index is set. By comparing the path centerline detection accuracy index with the set threshold, it is determined whether to issue a warning.

2. The vision-based path centerline detection method according to claim 1, characterized in that: In step S1, when collecting visual data, the vehicle-mounted camera continuously collects road images with a resolution of not less than 1280×720 at 30fps, and transmits the collected road images to the processor in real time. At the same time, the collected road images are preprocessed by grayscale conversion, Gaussian filtering, and perspective transformation.

3. The vision-based path centerline detection method according to claim 1, characterized in that: In step S1, during data synchronization, the vehicle's real-time position, speed, and heading angle data are acquired in real time, and the image frames and sensor data are synchronized with millisecond-level timestamps.

4. The vision-based path centerline detection method according to claim 1, characterized in that: In step S2, when extracting vehicle line feature data, a high and low threshold for edge detection is set, the lane line edge contour is extracted, candidate regions for straight line segments are detected, and invalid line segments with a length of less than 15 pixels or an angle that deviates from the lane direction are filtered out.

5. The vision-based path centerline detection method according to claim 1, characterized in that: In step S3, the actual road conditions are compared with the data of the high-precision map. If the lateral offset distance exceeds 0.3 meters or the confidence level is lower than 0.8, it is determined to be a detection error. At this time, the timestamp of the detection error and the location of the detection error are recorded.

6. The vision-based path centerline detection method according to claim 1, characterized in that: In step S4, during data analysis preprocessing, an interval duration evaluation index is obtained by comprehensively analyzing the interval between two adjacent error detections, the total detection time, and the time from the first use when each error occurred. ； in, As an index for evaluating interval duration, The time interval between each detection error and the previous detection error. To detect the total usage time, The time elapsed since the first use when each error is detected. This represents the number of errors detected.

7. The vision-based path centerline detection method according to claim 6, characterized in that: In step S4, different error weights are assigned to each detection error based on the offset distance, namely: ； in, This represents the error weight for each detected error.

8. The vision-based path centerline detection method according to claim 7, characterized in that: In step S4, the path centerline detection accuracy index is obtained by evaluating the interval duration index, the error weight of each detection error, and the total detection time. ； in, This is the accuracy index for path centerline detection. This is the error weighting coefficient, with a value of 0.

6. This is the interval weighting coefficient, with a value of 0.

4.

9. The vision-based path centerline detection method according to claim 1, characterized in that: In step S5, the accuracy of the detection result is evaluated by the path centerline detection accuracy index. The higher the path centerline detection accuracy index, the higher the accuracy of the detection result, and vice versa. At the same time, when the path centerline detection accuracy index is less than the set threshold of the path centerline detection accuracy index, a warning reminder that the accuracy of the detection result is too low is issued.

Images