Vehicle-mounted radar external parameter calibration method and device based on natural road characteristics, storage medium and program product

Abstract

The invention discloses a natural road feature-based vehicle-mounted radar external parameter calibration method and device, a storage medium and a program product. The method comprises the steps of obtaining synchronous radar point cloud data and visual image data; determining a visual lane line according to the visual image data, and determining a radar lane line according to the radar point cloud data; lane line transverse deviation is calculated according to the visual lane line and the radar lane line, and deviation confidence is calculated according to the credibility factor and the factor weight; determining a deviation estimated value based on the effective deviation and the deviation confidence by adopting a one-dimensional Kalman filter; the deviation estimation value is evaluated according to preset calibration threshold values, the preset calibration threshold values comprise the minimum calibration threshold value and the maximum calibration threshold value, when the deviation estimation value is larger than the minimum calibration threshold value and the maximum calibration threshold value, the radar coordinates are calibrated based on the deviation estimation value, and when the deviation estimation value is larger than the maximum calibration threshold value, an early warning mode is started. And outputting early warning prompt information in response to the early warning mode, and recording a fault code.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of online real-time external parameter calibration of vehicle-mounted millimeter wave radar, in particular to a vehicle-mounted radar external parameter calibration method based on natural road features, equipment, storage medium and program product. BACKGROUND

[0002] During the entire life cycle of the vehicle, due to bumps, collisions or maintenance, the actual position of the radar may be slightly offset (zero to several degrees). Such offset will cause the target position perceived by the radar to deviate, and in severe cases may cause the ADAS function to be mis-triggered or fail, posing a safety hazard. Therefore, developing a technology that can rely on natural and universal road environment features to realize real-time online monitoring and accurate calibration of radar external parameters during normal driving of the vehicle has become a key problem to be solved in the current vehicle-mounted radar perception field.

[0003] Offline calibration is the standard calibration method before the current mass-produced vehicles are shipped, which requires relying on special sites, equipment and calibration objects to complete external parameter calibration under the condition that the vehicle is stationary or low-speed controlled. Specifically, it can be divided into two core schemes:

[0004] One is the special calibration object method, which sets up reference objects with known size and position such as corner reflectors, checkerboard targets, calibration boards, etc. in a special workshop, and lets the radar detect them. The system solves the external parameters based on the theoretical coordinates of the reference objects and the radar detection coordinates through least squares method, iterative closest point (ICP) algorithm, etc. This scheme is suitable for various vehicle-mounted radars such as millimeter wave radar, laser radar, etc. The advantages are high calibration accuracy (angle deviation can be controlled within 0.1°) and strong stability, and the disadvantages are dependence on special sites and expensive equipment, which cannot adapt to dynamic offset calibration in the whole life cycle of the vehicle. Two is multi-sensor joint offline calibration: combining the detection data of laser radar, high-definition camera and other high-precision sensors to construct the coordinate mapping relationship of multi-source data and realize radar external parameter calibration.

[0005] The disadvantages of offline calibration technology include: 1) unable to monitor online: users cannot know in real time whether the radar external parameters are misaligned, which is easy to form "implicit safety risk"; 2) high maintenance cost: need to return to professional institutions to use expensive equipment for re-calibration, the process is cumbersome and time-consuming; 3) lack of universality: some solutions rely on specific calibration objects (such as checkerboard, special target) or high-precision map support, which cannot adapt to ordinary roads and other conventional driving scenes.

[0006] Online calibration technologies mainly include: 1) Online calibration based on natural environmental features: relying on naturally existing road features (such as lane lines, guardrails, curbs, streetlights, etc.), using radar and cameras for joint detection, extracting features and performing cross-sensor matching to solve for extrinsic parameter deviations. 2) Online calibration based on vehicle motion constraints: utilizing the vehicle's own motion data (such as vehicle speed, yaw rate, steering angle, etc. output from the CAN bus), combined with static environmental point clouds detected by radar, constructing constraints based on kinematic models, and iteratively optimizing extrinsic parameters. 3) Online calibration based on specific scenarios / calibration objects: some solutions still rely on specific auxiliary calibration objects (such as road signs, dedicated roadside targets) or high-precision maps, using radar to detect the coordinates of these known features to complete extrinsic parameter calibration.

[0007] The shortcomings of online calibration technology include: 1) Poor robustness of calibration benchmarks: Mainstream solutions mostly rely on single natural features (such as guardrails, single lane lines) or multi-sensor area matching. Single features are easily affected by occlusion or missing features, leading to calibration failure. Area matching lacks a clear deviation quantification model, and accuracy depends on the matching effect, resulting in weak anti-interference ability; 2) Lack of scene adaptability evaluation mechanism: The effectiveness of calibration scenarios is not judged. In complex scenarios such as curves, acceleration and deceleration, and blurred lane lines, calibration results are still forcibly output, which can easily lead to the accumulation of deviations; 3) Insufficient safety redundancy: It only focuses on solving and compensating external parameters and lacks a hierarchical decision-making mechanism. When external parameters have serious deviations, it cannot issue warnings to the driver in time and record fault codes, which can easily cause ADAS functions to be falsely triggered or fail; 4) Conflict between universality and deployment cost: Some solutions rely on additional sensors such as LiDAR and IMU or high-precision maps, which increases hardware costs and is difficult to adapt to the "forward millimeter-wave radar + forward-looking camera" hardware architecture that is standard in mass-produced vehicles. Summary of the Invention

[0008] To address at least one aspect of the aforementioned problems, this invention provides a method for calibrating extrinsic parameters of vehicle-mounted radar based on natural road features, comprising: acquiring synchronous sensor data, including radar point cloud data and visual image data; determining visual lane lines based on the visual image data, including visual left lane lines and visual right lane lines; determining radar lane lines based on the radar point cloud data, including radar left lane lines and radar right lane lines; calculating the lateral deviation of the lane lines based on the visual lane lines and radar lane lines; and calculating the deviation confidence level based on a confidence factor and factor weights, wherein the confidence factor includes a point cloud quality factor. The system considers visual confidence factors, vehicle state factors, and road state factors. When the deviation confidence level is greater than or equal to a preset confidence threshold, the lateral deviation of the lane line is determined to be a valid deviation. A one-dimensional Kalman filter is used to determine the deviation estimate based on the valid deviation and the deviation confidence level. The deviation estimate is evaluated according to preset calibration thresholds, which include a minimum calibration threshold and a maximum calibration threshold. When the deviation estimate is greater than the minimum calibration threshold and the maximum calibration threshold, the radar coordinates are calibrated based on the deviation estimate. When the deviation estimate is greater than the maximum calibration threshold, a warning mode is activated, and a warning message is output in response to the warning mode, as well as a fault code is recorded.

[0009] Preferably, the step of acquiring synchronous sensor data includes: capturing time-aligned visual image data output by a visual sensor based on the timestamp of the radar point cloud data.

[0010] Preferably, the step of determining the radar lane line based on radar point cloud data includes: filtering moving target points from the radar point cloud data based on Doppler velocity and vehicle speed, and generating a static point cloud; determining the region of interest static point cloud from the static point cloud based on a preset boundary; using a distance-based Euclidean clustering algorithm to determine the left and right point cloud clusters, and determining the left lane line based on the left point cloud cluster and the right lane line based on the right point cloud cluster.

[0011] Preferably, the radar lane line length is determined based on the radar lane line, and the point cloud quality factor is determined based on the radar lane line length and the number of radar point clouds. When the radar lane line length is less than a first length threshold and the number of radar point clouds is less than a first point cloud quantity threshold, the point cloud quality factor is equal to the minimum value. When the radar lane line length is greater than or equal to the first length threshold and less than a second length threshold, and the number of radar point clouds is greater than or equal to the first point cloud quantity threshold and less than a second point cloud quantity threshold, the point cloud quality factor is equal to a first intermediate value. When the radar lane line length is greater than or equal to the second length threshold and less than a third length threshold, and the number of radar point clouds is greater than or equal to the second point cloud quantity threshold and less than a third point cloud quantity threshold, the point cloud quality factor is equal to a second intermediate value, which is greater than the first intermediate value. When the radar lane line length is greater than the third length threshold and the number of radar point clouds is greater than the third point cloud quantity threshold, the point cloud quality factor is equal to the maximum value.

[0012] Preferably, the vehicle sway angular velocity is obtained. When the vehicle sway angular velocity is less than the minimum sway angular threshold, the vehicle state factor is equal to the maximum value. When the vehicle sway angular velocity is greater than the maximum sway angular threshold, the vehicle state factor is equal to the minimum value. When the vehicle sway angular velocity is greater than the minimum sway angular threshold and less than the maximum sway angular threshold, the vehicle state factor is equal to the intermediate value.

[0013] Preferably, the road state factor is determined based on the clarity of the lane lines on both sides, the point cloud cluster of the left lane line, and the point cloud cluster of the right lane line.

[0014] Preferably, the step of calibrating radar coordinates based on the deviation estimate further includes: calculating the angle deviation based on the deviation estimate, and generating a rotation compensation matrix based on the angle deviation.

[0015] In a second aspect, an electronic device is provided, comprising: a processor and a memory, the memory for storing a computer program, the computer program including program instructions, and the processor for calling the computer program to implement the method as described in any of the preceding methods.

[0016] Thirdly, a computer-readable storage medium is provided, characterized in that it is used to store computer program instructions that, when executed by a processor, implement the method described in any of the preceding methods.

[0017] Fourthly, a computer program product is provided, characterized in that it includes a computer program that, when run on a computer, causes the computer to perform the method described in any of the preceding methods.

[0018] The vehicle-mounted millimeter-wave radar extrinsic parameter calibration method based on natural road features of this invention has the following advantages: it uses a full-process online real-time calibration mode, eliminating the need for offline initial calibration, and can dynamically monitor and correct extrinsic parameter deviations during normal vehicle operation, completely eliminating the dependence on dedicated sites and equipment, significantly reducing maintenance costs, and adapting to real road scenarios.

[0019] 1) The calibration benchmark adopts a dual feature fusion of "visual lane lines + radar road boundary lines". The two features verify and complement each other, and the calibration effectiveness can still be guaranteed even in scenarios where a single feature is occluded (such as lane lines being covered by water or radar boundary point clouds being missing). It is suitable for most common driving scenarios such as straight lines and large-radius curves. Combined with RANSAC straight line fitting and density clustering preprocessing, abnormal data interference is further eliminated, and the feature extraction accuracy is improved.

[0020] 2) A comprehensive confidence assessment mechanism is set up with confidence factors (straight line length / point cloud quantity, visual confidence, driving status, scene richness) to achieve accurate judgment on calibration scene and data quality. Only high confidence results participate in subsequent calibration to avoid the accumulation of deviations in complex scenes (such as sharp bends and blurred lane lines). A dual-parameter deviation model of Δd and Δθ is adopted to directly associate with the core error sources of radar extrinsic parameters (lateral translation and heading angle deviation). With the state observer filtering optimization, the calibration angle deviation can be controlled within 0.2°, which is more accurate than the existing single deviation quantification scheme.

[0021] 3) Construct a hierarchical decision-making mechanism of "micro-calibration soft compensation + serious deviation warning + fault code recording" to achieve dual protection of "dynamic calibration + safety warning". The micro-calibration mode can correct minor deviations in real time to avoid the degradation of ADAS function accuracy; the warning mode promptly feeds back to the driver and records fault codes when there is serious inaccuracy, avoiding the risk of false triggering. This solves the shortcomings of existing technologies that only focus on calibration and lack safety warnings, and takes into account both dynamic calibration accuracy and driving safety.

[0022] 4) Relying on standard hardware in mass-produced vehicles (millimeter-wave radar, forward-facing camera, CAN bus), no additional dedicated equipment, calibration objects, or high-precision maps are required. The entire process is completed online in real time, without the need for offline initial calibration. Compared to existing multi-sensor fusion calibration solutions, hardware costs are reduced; compared to offline calibration, it saves on subsequent maintenance site, equipment, and labor costs, significantly improving the mass production applicability of the solution. Attached Figure Description

[0023] To better understand the above and other objects, features, advantages, and functions of the present invention, reference can be made to the embodiments shown in the accompanying drawings. The same reference numerals in the drawings refer to the same parts. Those skilled in the art should understand that the drawings are intended to schematically illustrate preferred embodiments of the invention and do not limit the scope of the invention in any way; the parts in the drawings are not drawn to scale.

[0024] Figure 1 A flowchart illustrating a vehicle-mounted radar extrinsic parameter calibration method based on natural road features according to an embodiment of the present invention is shown.

[0025] Figure 2 A structural block diagram of an electronic device processor for implementing a vehicle-mounted radar extrinsic parameter calibration method based on natural road features, according to an embodiment of the present invention, is shown. Detailed Implementation

[0026] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0027] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0028] To at least partially address one or more of the aforementioned problems and other potential issues, embodiments of this disclosure propose a method for calibrating extrinsic parameters of vehicle-mounted radar based on natural road features, such as... Figure 1 As shown, the process includes: Step S1, acquiring synchronous sensor data, which includes radar point cloud data and visual image data.

[0029] Specifically, the sensor data source is onboard sensors, including onboard millimeter-wave radar and onboard cameras. The synchronous sensor data includes radar point cloud data collected by the forward-facing millimeter-wave radar and visual image data collected by the forward-facing camera.

[0030] In some embodiments, the step of acquiring synchronous sensor data includes: capturing time-aligned visual image data output by a visual sensor based on the timestamp of radar point cloud data.

[0031] Specifically, each time the domain controller receives a frame of radar point cloud data (e.g., with an update rate of 20Hz), it captures a frame of time-aligned structured data processed by the camera, i.e., visual image data, based on its timestamp. The visual image data contains the curve equations of the identified left and right lane lines (with reference to the vehicle coordinate system).

[0032] In some embodiments, the method further includes capturing time-aligned vehicle data based on the timestamps of radar point cloud data. The vehicle data includes vehicle speed and yaw rate. The vehicle steering angle is determined based on the vehicle data. If the vehicle steering angle is less than a preset angle threshold, the process proceeds to step 2. For example, the vehicle speed v is 30 m / s, and the yaw rate ω is 0.05 rad / s. Based on the vehicle speed and yaw rate, if the vehicle is determined to be in a near-straight-line state, the process proceeds to step S2. The vehicle steering angle allows for the determination of the vehicle's driving state, enabling the process to proceed to step S2 under conditions such as straight-line driving or large-radius curves, thereby improving the accuracy of the results.

[0033] Step S2: Determine visual lane lines based on visual image data, including visual left lane lines and visual right lane lines; determine radar lane lines based on radar point cloud data, including radar left lane lines and radar right lane lines.

[0034] Specifically, the camera includes a camera perception module, which directly outputs the visual left lane line and the visual right lane line based on the visual image. The camera perception module performs lane line detection on the visual image, obtains the equations of the left and right lane lines (in the image coordinate system), and transforms them into spatial straight line models Lane_Camera_Left and Lane_Camera_Right in the vehicle coordinate system through camera-vehicle extrinsic parameters.

[0035] For example, the visual equation for the left lane is y_left = 1.85 (meters), and the visual equation for the right lane is y_right = -1.80 (meters) (assuming the Y-axis of the vehicle coordinate system is positive to the left). This means the camera perceives the lane width as approximately 3.65 meters and the vehicle as being roughly centered in the lane.

[0036] In some embodiments, the step of determining radar lane lines based on radar point cloud data includes: step S201, filtering out moving target points from the radar point cloud data based on Doppler velocity and vehicle speed, and generating a static point cloud.

[0037] Specifically, for the current frame of radar point cloud data, all moving target points are first filtered out based on their Doppler velocity (combined with the vehicle's speed v), retaining only the points that are stationary relative to the ground.

[0038] Step S202: Determine the region of interest from the static point cloud based on the preset boundary.

[0039] Specifically, the region of interest is defined by a rectangular bounding box in front of the vehicle. The border of the region of interest is determined by a preset boundary threshold. For example, the preset boundary threshold is set to the static point cloud within a range of 5 to 50 meters in front of the vehicle and ±5 meters laterally from the vehicle's coordinates. This area is typically the main reflection zone of road boundary features such as guardrails and curbs.

[0040] Step S203: Use distance-based Euclidean clustering algorithm to determine the left point cloud cluster and the right point cloud cluster, and determine the left lane line based on the left point cloud cluster and the radar right lane line based on the right point cloud cluster.

[0041] Specifically, a distance-based Euclidean clustering algorithm is used to separate the static point clouds on the left and right sides. For the left point cloud cluster (assumed to be a guardrail), a straight line, Lane_Radar_Left, is fitted using the Random Sample Consensus (RANSAC) algorithm to obtain its equation. Similarly, Lane_Radar_Right is fitted for the right side. For example, the fitting results are: Lane_Radar_Left: y = 1.92 (meters); Lane_Radar_Right: y = -1.75 (meters).

[0042] Step S3: Calculate the lateral deviation of the lane lines based on the visual lane lines and the radar lane lines, and calculate the confidence level of the deviation based on the confidence factor and factor weight. The confidence factor includes the point cloud quality factor, the visual confidence factor, the vehicle state factor, and the road state factor.

[0043] Specifically, the radar fitting line is matched with the visual reference line.

[0044] The left lateral deviation is calculated based on the straight line equations corresponding to Lane_Camera_Left and Lane_Radar_Left: Δd_left = 1.92 - 1.85 = 0.07 m.

[0045] The right lateral deviation is calculated based on the straight line equations corresponding to Lane_Camera_Right and Lane_Radar_Right: Δd_right = -1.75 - (-1.80) = 0.05 m.

[0046] The average lateral deviation is calculated based on the left and right lateral deviations: Δd_raw = (0.07 + 0.05) / 2 = 0.06 m.

[0047] In some embodiments, the calculation of angular deviation Δθ is also included. Angular deviation can be indirectly estimated through geometric relationships such as the difference between deviations on both sides. In this example, it is assumed that the angular deviation is small, and the main focus is on the lateral deviation Δd.

[0048] The confidence factors include point cloud quality factor a, visual confidence factor b, vehicle state factor c, and road state factor d. Factor weights include point cloud quality factor weight α, visual confidence factor weight β, vehicle state factor weight γ, and road state factor weight δ. The bias confidence score is: V = α×a + β×b + γ×c + δ×d. The normalized scores of the four confidence factors (ranging from 0 to 1) are used, and the sum of the weights of each factor is 1. The weights can be calibrated and assigned according to different road scenarios. An example weight assignment (adapted to typical road scenarios) is: α=0.3, β=0.25, γ=0.25, δ=0.2.

[0049] In some embodiments, the radar lane line length is determined based on the radar lane line, and the point cloud quality factor is determined based on the radar lane line length and the number of radar point clouds. When the radar lane line length is less than a first length threshold and the number of radar point clouds is less than a first point cloud quantity threshold, the point cloud quality factor is equal to the minimum value. When the radar lane line length is greater than or equal to the first length threshold and less than a second length threshold, and the number of radar point clouds is greater than or equal to the first point cloud quantity threshold and less than a second point cloud quantity threshold, the point cloud quality factor is equal to a first intermediate value. When the radar lane line length is greater than or equal to the second length threshold and less than a third length threshold, and the number of radar point clouds is greater than or equal to the second point cloud quantity threshold and less than a third point cloud quantity threshold, the point cloud quality factor is equal to a second intermediate value, which is greater than the first intermediate value. When the radar lane line length is greater than the third length threshold and the number of radar point clouds is greater than the third point cloud quantity threshold, the point cloud quality factor is equal to the maximum value.

[0050] Specifically, the point cloud quality factor 'a' is positively correlated with the radar lane line length and the number of radar point clouds; that is, the longer the radar lane line length and the greater the number of radar point clouds, the higher the data reliability.

[0051] For example, the point cloud quality factor can be quantified by setting length thresholds and point cloud quantity thresholds. The length thresholds include a first length threshold (e.g., 10m), a second length threshold (e.g., 20m), and a third length threshold (e.g., 30m); the point cloud quantity thresholds include a first point cloud quantity threshold (e.g., 15), a second point cloud quantity threshold (e.g., 30), and a third point cloud quantity threshold (e.g., 50). In another embodiment, the above length thresholds and point cloud quantity thresholds can be set according to the actual application scenario, and the values ​​of the second intermediate value (e.g., 0.7) and the first intermediate value (e.g., 0.4) can be varied. The point cloud quality factor values ​​determined based on the above thresholds are as follows:

[0052] When the radar lane line length is ≥30 meters and the number of point clouds is ≥50, the point cloud quality factor is 1.0.

[0053] When the radar lane line length is 20 meters or less and the number of point clouds is less than 30 meters, and the number of point clouds is 30 or less and the number of point clouds is less than 50, the point cloud quality factor is taken as the second median value of 0.7.

[0054] When the radar lane line length is 10 meters or less and the number of point clouds is less than 20 meters, and the number of point clouds is 15 or less and the number of point clouds is less than 30, the point cloud quality factor is taken as the first median value of 0.4.

[0055] The radar lane line length is less than 10 meters, or the number of point clouds is less than 15, and the point cloud quality factor is set to the minimum value of 0.

[0056] The visual confidence factor b represents the visual lane line detection confidence level. It directly uses the lane line detection confidence level output by the camera (normalized to 0-1). For example, in one embodiment, the camera reports a confidence level of 95%, the visual confidence factor b = 0.95, and the visual confidence factor weight β = 0.25.

[0057] In some embodiments, the vehicle sway angular velocity is obtained. When the vehicle sway angular velocity is less than the minimum sway angular threshold, the vehicle state factor is equal to the maximum value. When the vehicle sway angular velocity is greater than the maximum sway angular threshold, the vehicle state factor is equal to the minimum value. When the vehicle sway angular velocity is greater than the minimum sway angular threshold and less than the maximum sway angular threshold, the vehicle state factor is equal to the intermediate value.

[0058] Specifically, the vehicle state factor c is used to characterize the vehicle's driving state. During straight-line / large-radius curve driving, the calibration conditions are more stable, and deviation calculations are more accurate. In one embodiment, the minimum yaw angle threshold is set to 0.1 rad / s, the maximum yaw angle threshold is 0.3 rad / s, and the median value is 0.8. Then, based on the yaw rate ω, the vehicle state factor c value is:

[0059] When the yaw rate |ω| ≤ 0.1 rad / s (approximately straight travel), the vehicle state factor c takes the maximum value of 1.0;

[0060] When 0.1 < yaw rate |ω| ≤ 0.3 rad / s (large radius curve), the vehicle state factor c takes the intermediate value of 0.8;

[0061] When the yaw rate |ω|>0.3 rad / s (sharp bend / acceleration / deceleration), the vehicle state factor c takes the minimum value of 0.

[0062] For example, in one embodiment, the yaw rate ω = 0.05 rad / s (approximately straight), c = 1.0; and the vehicle state factor weight γ = 0.25.

[0063] In some embodiments, road state factors are determined based on the clarity of lane lines on both sides, the point cloud cluster of the left lane line, and the point cloud cluster of the right lane line.

[0064] Specifically, the road state factor d depends on the richness of the road scene; clear lane lines, high radar boundary point cloud density, and strong scene adaptability are all desirable. For example:

[0065] When the two lane lines are clear and the point clouds of the radar boundaries on both sides are stable, the road state factor is set to 1.0;

[0066] When the single-lane line is clear and the point cloud of the radar boundary on one side is stable, the road state factor is taken as 0.6;

[0067] When lane lines are blurred or radar boundary point clouds are missing, the road state factor is set to 0.2~0.

[0068] In one embodiment, the two-lane lines are clear, the static point clouds on both sides are stable, d=1.0, and the road state factor weight δ=0.2.

[0069] In some embodiments, the lengths of the fitted lines on both the left and right sides are both >30 meters, the number of point clouds is both >50, and the point cloud quality factor a=1.0; the camera reports a lane line detection confidence level of 95%, and the visual confidence factor b=0.95; the vehicle speed is stable (30 m / s), the yaw rate is extremely small (0.05 rad / s), and it is in an ideal calibration condition, with a vehicle state factor c=1.0; both lane lines are clear, and there are stable static point clouds on both sides, with a road state factor d=1.0. The factor weights for adapting to conventional road scenarios are: α=0.3, β=0.25, γ=0.25, δ=0.2.

[0070] The bias confidence level V = α×a + β×b + γ×c + δ×d = 0.9875, approximately 0.9.

[0071] Step S41: When the deviation confidence level is greater than or equal to the preset confidence threshold, the lateral deviation of the lane line is determined to be a valid deviation; a one-dimensional Kalman filter is used to determine the deviation estimate based on the valid deviation and the deviation confidence level.

[0072] Specifically, the preset confidence threshold is 0.7, and the aforementioned bias confidence score V is calculated to be 0.9 (out of 1.0), which is higher than the preset threshold Th_v = 0.7. Therefore, the calculated Δd_raw = 0.06 m is considered a valid observation.

[0073] A general one-dimensional Kalman filter is employed, and its state dimension, system model, noise parameters, and observation preprocessing are set according to the extrinsic bias estimation scenario of this technical solution to track the estimated value Δd_est. The currently valid Δd_raw = 0.06m and its confidence level V = 0.9 are input into the filter. Through recursive calculation, the filter removes noise that may exist in the previous frame and outputs a more stable and smoother estimated value Δd_est = 0.055m.

[0074] Step S42: Evaluate the deviation estimate based on the preset calibration thresholds. The preset calibration thresholds include a minimum calibration threshold and a maximum calibration threshold. When the deviation estimate is greater than the minimum calibration threshold and the maximum calibration threshold, calibrate the radar coordinates based on the deviation estimate. When the deviation estimate is greater than the maximum calibration threshold, activate the warning mode and output a warning message and record the fault code in response to the warning mode.

[0075] Specifically, the deviation estimates are graded by setting preset calibration thresholds. The minimum calibration threshold Th_min = 0.03m, and the maximum calibration threshold Th_max = 0.15m.

[0076] Decision judgment: Since Th_min (0.03) < |Δd_est| (0.055)≤Th_max (0.15), the system determines that there is a small but significant stable deviation and enters the online micro-calibration mode.

[0077] In some embodiments, the step of calibrating radar coordinates based on the deviation estimate further includes: calculating the angle deviation based on the deviation estimate, and generating a rotation compensation matrix based on the angle deviation.

[0078] Based on the geometric model, and using 25 meters as a reference distance, this lateral deviation mainly corresponds to a small deviation of approximately arctan(0.055 / 25) ≈ 0.126° in the radar's azimuth angle, i.e., Δθ. The calibration module generates a corresponding rotation compensation matrix.

[0079] The rotation compensation matrix is ​​based on the principle of rotation transformation between the vehicle coordinate system and the radar coordinate system. It uses matrix multiplication to transform the point coordinates in the radar coordinate system (…). Convert to the corrected vehicle coordinate system coordinates. ).

[0080] Assume that the radar coordinate system ( () is the coordinate system for the radar's raw data output. The radar defaults to facing forward as positive. The left side of the radar is positive. Upward from the ground is considered positive.

[0081] Vehicle coordinate system ( ) target coordinate system The vehicle is moving in a positive direction. The left side of the vehicle is facing forward. Upward from the ground is considered positive.

[0082] Radar data needs to be converted to this vehicle coordinate system. Let the coordinates of any point in the radar coordinate system be... The corresponding coordinates in the vehicle coordinate system are Points in the radar coordinate system need to be converted into points in the vehicle coordinate system by first passing through Rx(α) (roll correction), then through Ry(β) (pitch correction), and finally through Rz(γ) (heading correction).

[0083] The rotation transformation formula is: .

[0084] In this patent, the roll angle α and pitch angle β during radar installation have been controlled within a very small range (usually ≤0.1°) through factory offline calibration. The dynamic offset during vehicle movement is mainly manifested as heading angle deviation γ (such as the radar turning left or right due to bumps). Therefore, the complete transformation formula can be simplified, retaining only the heading angle rotation matrix.

[0085] This patent assumes that the radar azimuth angle deviation is Δθ (the above result is 0.126°). The compensation matrix needs to rotate the radar coordinate system clockwise by Δθ around the Z-axis, using the lateral deviation caused by Δθ:

[0086] .

[0087] This compensation matrix is ​​sent to the "calibration parameter storage module" for updating. Subsequently, all raw radar point clouds are transformed using this matrix before being output to applications such as ACC and AEB, thus correcting the 0.055-meter system deviation at the software level.

[0088] Throughout the entire process, the driver is unaware of anything. The target location information received by the ADAS system has automatically become more accurate. The system continues to execute steps S1-S4. If the deviation subsequently increases to |Δd_est| > 0.15m due to bumps, the system will exit micro-calibration mode and switch to warning mode, displaying the icon and text "Radar sensor limited, please contact the service center" on the instrument panel, recording the fault code, and reminding the driver to have it professionally inspected.

[0089] The preset scenarios include: a reference distance of 25m; the actual deviation true value is pre-measured using high-precision offline calibration equipment (corner reflector + laser tracker) to ensure accuracy, where the true value is Δd=0.10m and Δθ=0.231°; the test scenario is a suburban road with slightly worn lane lines and a small-radius curve (|ω|=0.2 rad / s), with a confidence level of V=0.75 (medium confidence); data volume: 100 frames of valid data are collected for each scenario, and the average deviation estimate and error (estimated value - true value) are calculated.

[0090] Based on 100 frames of radar point cloud and visual images, the method proposed in this paper estimates the average Δd value (0.102 m) over 100 frames with an estimation error of +0.002° and the average Δθ value (0.233°) with an estimation error of +0.002°. In contrast, the prior art (CN110488234A) calculates an average Δd value (0.115 m) over 100 frames with an estimation error of +0.015°. The prior art (CN110488234A) also calculates an average Δθ value (0.250°) over 100 frames with an estimation error of +0.019°. Therefore, the proposed method demonstrates improved accuracy compared to the prior art.

[0091] Based on the above, an electronic device is provided, comprising: a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to invoke the computer program to implement the method described in any of the preceding methods.

[0092] Specifically, such as Figure 2 As shown, the processor includes a processing layer and an output layer. The processor receives millimeter-wave radar point cloud data, visual image data, and vehicle data via the onboard forward-facing millimeter-wave radar, forward-facing camera, and vehicle bus. The processing layer includes a data synchronization module, a feature extraction and fusion module, an extrinsic parameter calculation and evaluation module, a calibration decision and execution module, and a calibration parameter storage module. The data synchronization module is used to synchronously capture radar point cloud data and visual image data. The feature extraction and fusion module is used to determine visual lane lines based on the visual image data (including visual left and right lane lines) and radar lane lines based on the radar point cloud data (including radar left and right lane lines). The extrinsic parameter calculation and evaluation module is used to calculate the lane line lateral deviation based on the visual and radar lane lines and to calculate the deviation confidence level based on the confidence factor and factor weights. The calibration decision and execution module determines the deviation estimate based on the effective deviation and deviation confidence level, and evaluates the deviation estimate according to preset calibration thresholds, including a minimum calibration threshold and a maximum calibration threshold. When the deviation estimate exceeds both the minimum and maximum calibration thresholds, the radar coordinates are calibrated based on the deviation estimate. When the deviation estimate exceeds the maximum calibration threshold, an early warning mode is activated, and an early warning message is output, along with a fault code, in response. The calibration parameter storage module stores the calibration radar parameters.

[0093] The output layer receives calibrated radar output information through the processing layer and outputs it to ADAS control, or receives early warning information through the processing layer and outputs it through instruments or domain controllers.

[0094] Based on the above, this application also provides a computer-readable storage medium storing instructions that, when executed, cause the method provided in any of the above-described method embodiments to be implemented. The computer-readable storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0095] Based on the above, this application also provides a computer program product, which includes: a computer program (also referred to as code or instructions), which, when run on a computer, causes the computer to execute the method provided in any of the above method embodiments. Optionally, the computer can be an in-vehicle terminal device.

[0096] The methods provided in the embodiments of this application above are described from the perspective of an electronic device as the executing entity. To implement the functions of the methods provided in the embodiments of this application above, the electronic device may include hardware structures and / or software modules, implementing the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is executed in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0097] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0098] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0100] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope and intent of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and variations.

Claims

1. A method for calibrating extrinsic parameters of vehicle-mounted radar based on natural road features, characterized in that, include: Acquire synchronous sensor data, which includes radar point cloud data and visual image data; Visual lane lines are determined based on visual image data, including visual left lane lines and visual right lane lines. Radar lane lines are determined based on radar point cloud data, including radar left lane lines and radar right lane lines. The lateral deviation of the lane lines is calculated based on the visual lane lines and the radar lane lines, and the confidence level of the deviation is calculated based on the confidence factor and factor weight. The confidence factor includes the point cloud quality factor, the visual confidence factor, the vehicle state factor, and the road state factor. When the deviation confidence level is greater than or equal to the preset confidence threshold, the lane line lateral deviation is determined to be a valid deviation; A one-dimensional Kalman filter is used to determine the bias estimate based on the effective bias and the bias confidence level; The deviation estimate is evaluated based on the preset calibration thresholds, which include a minimum calibration threshold and a maximum calibration threshold. When the deviation estimate is greater than the minimum calibration threshold and the maximum calibration threshold, the radar coordinates are calibrated based on the deviation estimate. When the deviation estimate is greater than the maximum calibration threshold, the warning mode is activated, and a warning message is output in response to the warning mode, as well as a fault code is recorded.

2. The method according to claim 1, characterized in that, The steps for acquiring synchronous sensor data include: capturing time-aligned visual image data output by the visual sensor based on the timestamp of the radar point cloud data.

3. The method according to claim 1, characterized in that, The steps for determining radar lane lines based on radar point cloud data include: Moving target points are filtered out from the radar point cloud data based on Doppler velocity and vehicle speed, and a static point cloud is generated. Determine the region of interest from the static point cloud based on preset boundaries; The left and right point cloud clusters are determined by a distance-based Euclidean clustering algorithm. The left lane line is determined based on the left point cloud cluster, and the right lane line is determined based on the right point cloud cluster.

4. The method according to claim 3, characterized in that, The radar lane line length is determined based on the radar lane line, and the point cloud quality factor is determined based on the radar lane line length and the amount of radar point cloud data, including: When the radar lane line length is less than the first length threshold and the radar point cloud quantity is less than the first point cloud quantity threshold, the point cloud quality factor is equal to the minimum value. When the radar lane line length is greater than or equal to the first length threshold and less than the second length threshold, and the radar point cloud quantity is greater than or equal to the first point cloud quantity threshold and less than the second point cloud quantity threshold, the point cloud quality factor is equal to the first intermediate value. When the radar lane line length is greater than or equal to the second length threshold and less than the third length threshold, and the radar point cloud quantity is greater than or equal to the second point cloud quantity threshold and less than the third point cloud quantity threshold, the point cloud quality factor is equal to the second intermediate value, and the second intermediate value is greater than the first intermediate value. When the radar lane line length is greater than the third length threshold and the radar point cloud quantity is greater than the third point cloud quantity threshold, the point cloud quality factor is equal to the maximum value.

5. The method according to claim 4, characterized in that, Obtain the vehicle sway angular velocity. When the vehicle sway angular velocity is less than the minimum sway angular threshold, the vehicle state factor is equal to the maximum value. When the vehicle sway angular velocity is greater than the maximum sway angular threshold, the vehicle state factor is equal to the minimum value. When the vehicle sway angular velocity is greater than the minimum sway angular threshold but less than the maximum sway angular threshold, the vehicle state factor is equal to the intermediate value.

6. The method according to claim 5, characterized in that, Road condition factors are determined based on the clarity of lane lines on both sides, the point cloud cluster of the left lane line, and the point cloud cluster of the right lane line.

7. The method according to claim 1, characterized in that, The step of calibrating radar coordinates based on deviation estimates also includes: calculating the angular deviation based on the deviation estimates, and generating a rotation compensation matrix based on the angular deviation.

8. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being used to invoke the computer program to implement the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, Used to store computer program instructions that, when executed by a processor, implement the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1-7.

Images