Multi-sensor offline calibration system and method for intelligent driving vehicles

Abstract

The present invention discloses a multi-sensor offline calibration system and method for intelligent driving vehicles, the system comprising: a vehicle four-wheel alignment station for locating the vehicle position; a plurality of reflectors, the plurality of reflectors are respectively located around the vehicle four-wheel alignment station, and the reflectors are used to calibrate the radar position; a calibration pattern, the calibration pattern is arranged on the side of the reflector close to the vehicle, and the calibration pattern is used to calibrate the camera position; a reflector moving mechanism, the reflector moving mechanism can make the reflector move in the X / Y / Z axis; a diagnostic instrument, the radar and the camera are both connected to the diagnostic instrument for communication, and are used to determine whether the calibration is successful. The present invention can reduce the parts used for calibration, save costs, and improve the space utilization rate of the production line by compounding the calibration pattern on the reflector; on the other hand, it can improve the calibration efficiency and increase the production capacity.

Description

Technical Field

[0001] The present invention relates to the field of vehicle calibration technology, and in particular to a multi-sensor offline calibration system and method for an intelligent driving vehicle. Background Art

[0002] With the development of the automotive industry, an increasing number of domestic and international OEMs and Tier 1 companies are investing in the R&D, production, and sales of intelligent vehicles and their key components. Cameras and millimeter-wave radar sensors, due to their unique advantages, have become indispensable solutions for all major manufacturers and vehicle models.

[0003] Throughout the history of intelligent driving, especially in recent years, the number of sensors has increased significantly as the level of intelligent driving has continued to improve. From the initial single front radar solution to the fusion of a single radar and a single camera, and then to the current 5R6V12U1L solution, the complexity has also increased. During this process, due to the technical characteristics of cameras and millimeter-wave radars, sensors installed in vehicles with cameras and millimeter-wave radars must be calibrated before they roll off the production line. This corrects and determines the positional relationship between the sensor coordinate system and the world coordinate system. This is why end-of-line calibration has come into being.

[0004] Since domestic research in this area started relatively late compared to top international Tier 1 companies, most domestic OEMs' end-of-line calibration solutions have already been preempted by international Tier 1 companies such as Bosch, Continental, and Veoneer. Furthermore, because early vehicle sensor configurations were mostly 1R1V (one radar, one camera) solutions, the calibration stations provided did not reserve sufficient space for later calibration work after the addition of corner radar, surround-view cameras, and other sensors. Consequently, many OEMs resorted to building additional calibration rooms to handle the increasing number of sensors. This resulted in a series of issues, including low production line space utilization, difficult retrofits, high construction costs, and low calibration efficiency. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: in order to solve the technical problems that the calibration schemes in the existing technology easily lead to low production line space utilization, great difficulty in transformation, high construction cost and low calibration efficiency, the present invention provides a multi-sensor offline calibration system and method for intelligent driving vehicles, which can reduce calibration costs and improve calibration efficiency.

[0006] The technical solution adopted by the present invention to solve its technical problems is: a multi-sensor offline calibration system for intelligent driving vehicles, comprising: a vehicle four-wheel alignment station, used to locate the vehicle position; a plurality of reflectors, the plurality of reflectors are respectively located around the vehicle four-wheel alignment station, and the reflectors are used to calibrate the radar position; a calibration pattern, the calibration pattern is arranged on a side of the reflector close to the vehicle, and the calibration pattern is used to calibrate the camera position; a reflector moving mechanism, the reflector is installed on the reflector moving mechanism, and the reflector moving mechanism can move the reflector on the X / Y / Z axis; a diagnostic instrument, the radar and the camera are both communicatively connected to the diagnostic instrument, and are used to determine whether the calibration is successful.

[0007] Therefore, by compounding the calibration pattern on the reflective plate, on the one hand, the number of parts used for calibration can be reduced, costs can be saved, and the utilization rate of production line space can be improved; on the other hand, the calibration efficiency can be improved and the production capacity can be increased.

[0008] Furthermore, the effective area of ​​the calibration pattern is smaller than the area of ​​the reflective plate.

[0009] Furthermore, the multiple reflective plates include: a forward reflective plate, four corner reflective plates and a backward reflective plate, the forward reflective plate is located in front of the vehicle four-wheel alignment station, the backward reflective plate is located behind the vehicle four-wheel alignment station, and the four corner reflective plates are located at the four corners of the vehicle four-wheel alignment station.

[0010] Furthermore, the forward reflective plate and the backward reflective plate are perpendicular to the driving direction of the vehicle.

[0011] Furthermore, the longitudinal distance Δy between the center of the radar and the center of the reflector is 0.8m to 2m.

[0012] Furthermore, the distance between the lower edge of the reflective plate and the ground is greater than 3 cm.

[0013] Furthermore, assuming that the length of the reflector is L and the width is W, assuming that the coordinates of the center point of the reflector are and the coordinates of the center point of the radar are, the length of the reflector satisfies the relationship L = 0.15m + 0.2×Δy + 2×|x1-x0|, and the width of the reflector satisfies the relationship W = 0.30m + 0.3×Δy + 2×|z1-z0|.

[0014] Furthermore, the reflective plate is perpendicular to the optical axis of the camera.

[0015] The present invention also provides a multi-sensor offline calibration method for an intelligent driving vehicle, using the multi-sensor offline calibration system for an intelligent driving vehicle, the calibration method comprises the following steps:

[0016] S1. The operator drives the vehicle to be calibrated into the vehicle four-wheel alignment station and performs four-wheel alignment to determine the relative position of the vehicle to be calibrated and the reflector;

[0017] S2. Connecting the vehicle to be calibrated to the diagnostic instrument, the diagnostic instrument sends a calibration instruction to the vehicle to be calibrated. At this time, the radar detects the distance between the vehicle and the reflector, and the camera captures an image of the calibration pattern;

[0018] S3. The diagnostic instrument obtains a calibration result based on the radar's detection distance and the image captured by the camera. If the calibration result is within the error range, the calibration is considered successful and the vehicle can be offline normally; if the calibration result is outside the error range, the calibration is considered failed and recalibration is required.

[0019] Furthermore, if the calibration is not completed within the specified time, it is considered a failure.

[0020] The beneficial effect of the present invention is that, by integrating the calibration pattern with the reflective plate, the present invention can reduce the additional purchase of supporting equipment such as camera targets and target moving guide rails, thereby effectively saving costs; and, the integration of the calibration pattern with the reflective plate can also reduce the space occupied by the calibration equipment on the production line, thereby improving the space utilization rate of the production line and increasing production capacity; furthermore, the calibration of the radar and the camera can be completed at the same workstation, thus eliminating the vehicle transfer time. More vehicles can be calibrated per unit time, thereby improving the overall efficiency of the calibration and facilitating the optimization of the production line work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention will be further described below with reference to the accompanying drawings and examples.

[0022] Figure 1 It is a structural diagram of the multi-sensor offline calibration system for intelligent driving vehicles of the present invention.

[0023] Figure 2 Schematic diagram of the reflector and reflector moving mechanism of the present invention.

[0024] Figure 3 Schematic diagram of the calibration pattern of the present invention.

[0025] Figure 4 It is a schematic diagram of the connection between the diagnostic instrument and the sensor of the present invention.

[0026] Figure 5 This is a flow chart of the multi-sensor offline calibration method for an intelligent driving vehicle of the present invention.

[0027] Figure 6 It is a flow chart of camera calibration of the present invention.

[0028] Figure 7It is a flow chart of radar calibration of the present invention.

[0029] In the figure: 1. Vehicle four-wheel alignment station; 2. Reflection plate; 3. Calibration pattern; 4. Reflection plate moving mechanism; 5. Diagnostic instrument; 21. Forward reflection plate; 22. Four-corner reflection plates; 23. Backward reflection plate. DETAILED DESCRIPTION

[0030] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0031] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, features defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0032] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0033] like Figures 1 to 4As shown, the multi-sensor offline calibration system for an intelligent driving vehicle of the present invention includes: a vehicle four-wheel alignment station 1, which is used to locate the vehicle position; a plurality of reflectors 2, which are respectively located around the vehicle four-wheel alignment station 1, and the reflectors 2 are used to calibrate the radar position; a calibration pattern 3, which is arranged on the side of the reflector 2 close to the vehicle, and the calibration pattern 3 is used to calibrate the camera position; a reflector moving mechanism 4, on which the reflector 2 is installed, and the reflector moving mechanism 4 can move the reflector 2 in the X / Y / Z axes; a diagnostic instrument 5, to which the radar and the camera are both communicated and connected, and is used to determine whether the calibration is successful. In other words, the present invention can reduce the additional purchase of supporting equipment such as camera targets and target moving guide rails by integrating the calibration pattern 3 with the reflector 2, which can effectively save costs; and the integration of the calibration pattern 3 with the reflector 2 can also reduce the space occupied by the calibration equipment on the production line, improve the space utilization of the production line, and increase production capacity; furthermore, the calibration of the radar and camera can be completed at the same workstation, which saves the time of vehicle transfer. More vehicles can be calibrated per unit time, which improves the overall efficiency of the calibration and is conducive to the optimization of the work efficiency of the production line.

[0034] For example, the reflector 2 can be made of a metal material (aluminum, iron, steel, etc.). The effective area of ​​the calibration pattern 3 is smaller than the area of ​​the reflector 2. For example, the calibration pattern 3 can be a 2×2 black and white checkerboard, but is not limited to this. It can also be a black and white checkerboard of other formats. The calibration pattern 3 can be flatly affixed to the surface of the reflector 2. When calibration begins, the calibration algorithm (such as the Zhang Zhengyou calibration method) can automatically identify the intersection of the black and white grids and calculate the roll angle, pitch angle, and yaw angle of the camera. The reflector moving mechanism 4 is, for example, a sliding guide rail. The reflector 2 can be mounted on the reflector moving mechanism 4, driving the reflector 2 to move to meet different installation position requirements. The vehicle four-wheel alignment station 1 is used to determine the relative position between the vehicle to be calibrated and the reflector 2, so as to determine the positional relationship between the radar, camera, and the world coordinate system. When the vehicle to be calibrated enters the vehicle four-wheel alignment station 1, the roller rolling switch on the station can be activated to adjust the vehicle to a fixed position to ensure that the vehicle and the calibration facility meet the relative position relationship. For example, a three-dimensional coordinate system is established with a certain point of the vehicle four-wheel alignment station 1 as the origin. When multiple reflectors 2 are moved to the specified position, the coordinates of the center point of the reflector 2 (x0, y0, z0) and the coordinates of the center point of the radar (x1, y1, z1) can be determined.

[0035] Specifically, the multiple reflectors 2 include: a forward reflector 21, four corner reflectors 22, and a backward reflector 23. The forward reflector 21 is located in front of the vehicle's four-wheel alignment station 1, the backward reflector 23 is located behind the vehicle's four-wheel alignment station 1, and the four corner reflectors 22 are located at the four corners of the vehicle's four-wheel alignment station 1. In other words, the multiple reflectors 2 can calibrate radars and cameras at different locations on the vehicle, improving ease of use. It should be noted that the forward reflector 21 and the backward reflector 23 are perpendicular to the vehicle's direction of travel, which improves the accuracy of the calibration results. The longitudinal distance Δy between the center of the radar and the center of the reflector 2 is 0.8m to 2m. The distance between the lower edge of the reflector 2 and the ground is greater than 3cm. Assume that reflector 2 has a length of L and a width of W. Assume that the coordinates of reflector 2's center point are (x0, y0, z0), and the coordinates of the radar's center point are (x1, y1, z1). The length of reflector 2 satisfies the equation L = 0.15m + 0.2×Δy + 2×|x1-x0|, and the width of reflector 2 satisfies the equation W = 0.30m + 0.3×Δy + 2×|z1-z0|. The size and position of reflector 2 are set to obtain accurate calibration parameters to improve the accuracy of the calibration results. The position of reflector 2 can be arranged according to the sensor installation location of different vehicle models, but it must be within the field of view of the radar and sensor, and reflector 2 must be perpendicular to the optical axis of the camera.

[0036] like Figure 5 As shown, the present invention also provides a multi-sensor offline calibration method for an intelligent driving vehicle, using a multi-sensor offline calibration system for an intelligent driving vehicle. The calibration method includes the following steps:

[0037] S1. The operator drives the vehicle to be calibrated into the vehicle four-wheel alignment station 1 and performs four-wheel alignment to determine the relative position of the vehicle to be calibrated and the reflector 2.

[0038] S2. Connect the vehicle to be calibrated to the diagnostic instrument 5. The diagnostic instrument 5 sends a calibration instruction to the vehicle to be calibrated. At this time, the radar detects the distance between the vehicle and the reflector 2, and the camera captures an image of the calibration pattern 3.

[0039] S3. The diagnostic instrument 5 obtains a calibration result based on the radar's detection distance and the image captured by the camera. If the calibration result is within the error range, the calibration is considered successful and the vehicle can be offline normally. If the calibration result exceeds the error range, the calibration is considered failed and needs to be recalibrated.

[0040] It should be noted that if the calibration is not completed within the specified time, it is also considered a calibration failure. When the vehicle to be calibrated enters the vehicle four-wheel alignment station 1, the roller scroll switch is activated to adjust the vehicle to the specified position. At this time, the position of the reflector 2 can be adjusted according to the installation position of the radar and camera on the vehicle. After the position is determined, a three-dimensional coordinate system can be established to obtain the coordinates of the center points of multiple reflectors 2 and the coordinates of the center points of the radar. Then power on the vehicle and connect the vehicle to the diagnostic instrument 5. The diagnostic instrument 5 sends a calibration instruction to the vehicle to be calibrated. The controller inside the vehicle can control the operation of the radar and camera. The radar can detect the distance between it and the reflector 1, and the camera can capture the image of the calibration pattern 3 on the reflector 2. It should be noted that there is a sequence for radar calibration and camera calibration. The radar can be calibrated first, or the camera can be calibrated first, which is set by the manufacturer. After acquiring the radar's detection range and the camera's image, the diagnostic instrument 5 calculates the X, Y, and Z coordinates of the reflector 2 center, the calibration pattern 3 center, the radar center, and the camera center in the world coordinate system, as well as the camera's roll, pitch, and yaw angles. These calibration results are compared with preset standard values. If the calibration results are within the tolerance range, the calibration is considered successful and the vehicle can be taken offline normally. If the calibration results are outside the tolerance range, the calibration is considered a failure and requires recalibration.

[0041] like Figure 6 As shown in the figure, the calibration process of the camera includes: after the vehicle to be calibrated is powered on, the fault code is first cleared, and then the fault code is read to check whether there is an uncalibrated fault code; if not, the reason for the occurrence of the fault code needs to be analyzed; if there is an uncalibrated fault code, the camera calibration is started and the calibration result is obtained; the calibration result is compared with the standard value. If the calibration is successful, the controller is restarted and the fault code is cleared after the restart; if the calibration is unsuccessful, the reason for the calibration failure is displayed.

[0042] like Figure 7 As shown in the figure, the radar calibration process includes: after the vehicle to be calibrated is powered on, the fault code is cleared first, and then the fault code is read to check whether there is an uncalibrated fault code; if not, the cause of the fault code is analyzed; if there is an uncalibrated fault code, the radar calibration is started and the calibration result is obtained; the calibration result is compared with the standard value. If the calibration is successful, the controller is restarted and the fault code is cleared after restart; if the calibration is unsuccessful, the reason for the calibration failure is displayed.

[0043] To sum up, the multi-sensor offline calibration system and method for intelligent driving vehicles of the present invention can reduce the additional purchase of supporting equipment such as camera targets and target moving guide rails by integrating the calibration pattern 3 with the reflector 2, which can effectively save costs; and the integration of the calibration pattern 3 and the reflector 2 can also reduce the space occupied by the calibration equipment on the production line, improve the space utilization of the production line, and increase production capacity; furthermore, the calibration of the radar and camera can be completed at the same workstation, which saves the vehicle transfer time. More vehicles can be calibrated per unit time, which improves the overall efficiency of the calibration and is conducive to the optimization of the production line work efficiency.

[0044] With the above-described preferred embodiments of the present invention as a guide, and with reference to the above description, relevant personnel are fully capable of making various changes and modifications without departing from the technical spirit of this invention. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A multi-sensor offline calibration system for intelligent driving vehicles, characterized in that: include: A vehicle four-wheel alignment station (1) is used to locate the position of the vehicle; A plurality of reflective plates (2), the plurality of reflective plates (2) being respectively located around the vehicle four-wheel alignment station (1), the reflective plates (2) being used to calibrate the radar position; A calibration pattern (3), the calibration pattern (3) being arranged on a side of the reflector (2) close to the vehicle, the calibration pattern (3) being used to calibrate the position of the camera; A reflector moving mechanism (4), the reflector (2) being mounted on the reflector moving mechanism (4), and the reflector moving mechanism (4) being capable of moving the reflector (2) along the X / Y / Z axes; A diagnostic instrument (5), the radar and the camera are both in communication connection with the diagnostic instrument (5) for determining whether the calibration is successful; The effective area of ​​the calibration pattern (3) is smaller than the area of ​​the reflection plate (2); The plurality of reflective plates (2) include: a forward reflective plate (21), four-corner reflective plates (22) and a backward reflective plate (23), wherein the forward reflective plate (21) is located in front of the vehicle four-wheel alignment station (1), the backward reflective plate (23) is located behind the vehicle four-wheel alignment station (1), and the four-corner reflective plates (22) are located at the four corners of the vehicle four-wheel alignment station (1); The forward reflecting plate (21) and the backward reflecting plate (23) are perpendicular to the direction of travel of the vehicle; The longitudinal distance ∆y between the center of the radar and the center of the reflector (2) is 0.8 m to 2 m; The distance between the lower edge of the reflective plate (2) and the ground is greater than 3 cm; Assuming that the length of the reflecting plate (2) is L and the width is W, assuming that the coordinates of the center point of the reflecting plate (2) are (x0, y0, z0), and the coordinates of the center point of the radar are (x1, y1, z1), the length of the reflecting plate (2) satisfies the relationship L=0.15m+0.2×∆y+2×|x1-x0|, and the width of the reflecting plate (2) satisfies the relationship W=0.30m+0.3×∆y+2×|z1-z0|.

2. The multi-sensor offline calibration system for an intelligent driving vehicle according to claim 1, characterized in that: The reflective plate (2) is perpendicular to the optical axis of the camera.

3. A multi-sensor offline calibration method for an intelligent driving vehicle, characterized in that: Using the multi-sensor offline calibration system for an intelligent driving vehicle according to any one of claims 1 to 2, the calibration method comprises the following steps: S1, the operator drives the vehicle to be calibrated into the vehicle four-wheel alignment station (1) and performs four-wheel alignment so that the relative position of the vehicle to be calibrated and the reflective plate (2) is determined; S2, connecting the vehicle to be calibrated to the diagnostic instrument (5), the diagnostic instrument (5) sending a calibration instruction to the vehicle to be calibrated, at which time, the radar detects the distance between the vehicle and the reflector (2), and the camera captures an image of the calibration pattern (3); S3. The diagnostic instrument (5) obtains a calibration result based on the detection distance of the radar and the image captured by the camera. If the calibration result is within the error range, it is considered that the calibration is successful and the vehicle can be offline normally; if the calibration result is beyond the error range, it is considered that the calibration has failed and recalibration is required.

4. The multi-sensor offline calibration method for an intelligent driving vehicle as claimed in claim 3, characterized in that: If the calibration is not completed within the specified time, it is considered a failure.

Images