Method for calibrating a vehicle's environmental sensor taking into account a three-dimensional model of the vehicle, calibration test bench and vehicle
Patent Information
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- AUDI AG
- Filing Date
- 2018-09-21
- Publication Date
- 2026-07-30
AI Technical Summary
Existing calibration methods for vehicle environment sensors are laborious, expensive, and time-consuming, requiring complex mechanical setups and lengthy processes, especially in production lines.
A method using a calibration test stand that determines the pose of a vehicle relative to a measurement sensor by matching the vehicle's outer shell area with a three-dimensional model, allowing calibration without precise vehicle alignment and without the need for special floor structures or vehicle markings, enabling calibration during vehicle movement.
Facilitates quick and reliable calibration of vehicle environment sensors by determining extrinsic parameters without complex setups, reducing time and costs, particularly in production environments.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[0001] The present invention relates to a method for calibrating an environmental sensor of a vehicle. Furthermore, the present invention relates to a calibration test bench and a vehicle.
[0002] Perceiving the vehicle's surroundings in road traffic is an essential component of piloted driving. Environmental sensors, such as cameras, laser scanners, and similar devices, are used today to perceive the environment. Software components, such as those used for interpreting the driving environment, utilize their sensor data to create a current model of the driving environment. To ensure accurate perception by the environmental sensors and to enable subsequent fusion of the sensor data, calibration and measurement of the environmental sensors are necessary. This calibration and measurement process encompasses both intrinsic and extrinsic parameters. Extrinsic parameters can describe, for example, parameters relating to the environmental sensors themselves, parameters relating to the vehicle, and / or parameters relating to other sensors.
[0003] These parameters can be measured or specified in data sheets. However, calibration of the environmental sensors is always necessary to achieve the required parameter accuracy. Calibration is performed by the environmental sensors themselves, by recording a known vehicle environment. The parameters are then determined by calibration software. This known vehicle environment consists of a calibration test bench. This test bench is similar to a chassis dynamometer for vehicles, but it is equipped with appropriate calibration elements, such as calibration targets, and measurement sensors.
[0004] To calibrate the environmental sensors to each other, it is sufficient for the sensors to be able to detect the calibration elements. However, if calibration is required for the installation positions of the environmental sensors on the vehicle, the vehicle's pose relative to the calibration elements must be known. This is usually done using a calibration test bench. The test bench reliably establishes the vehicle's position and orientation, or pose, within the test space. The vehicle is locked into a position and orientation predefined by the test bench. More sophisticated test benches measure the rotation axis of the vehicle's axles and determine the chassis settings to ascertain the vehicle's pose within the test bench.
[0005] Determining the vehicle's pose within the calibration test stand is complex. Determining the vehicle's position by measuring the axis of rotation and adjusting the chassis requires precise positioning of the vehicle on the test stand so that these parameters can be measured. Even a calibration test stand that forces the position by "locking" it into place is complex and expensive. The vehicle's position and / or orientation must be aligned and verified to perform the calibration, which requires a mechanical setup. Another disadvantage of existing calibration test stands is the duration of the calibration process. Even with a so-called "one-shot calibration," the vehicle must be brought into the test stand until it is completely stationary.In production lines where the environmental sensors need to be calibrated before delivery to the customer, this represents a significant time investment.
[0006] In this context, DE 10 2010 004 233 B3 describes a method for determining the position of a camera system relative to an object, wherein the camera system comprises at least one camera. A mirror is arranged such that at least parts of a calibration element are projected into the field of view of the camera system via the mirror and captured by the camera system.
[0007] Furthermore, WO 2015 / 090553 A1 describes a calibration system for calibrating a vehicle camera. This calibration system is characterized by the fact that it includes a calibration medium comprising at least a first calibration reference and a mirror. The vehicle also includes a second calibration reference. This allows the camera to be calibrated by determining the spatial compensation values between the camera and the vehicle without requiring the vehicle to be aligned with the calibration medium.
[0008] Furthermore, WO 2017 / 016541 A1 describes a device for calibrating a vehicle assistance system, comprising a mobile calibration wall and two wheel sensors, each equipped with a laser. The calibration wall, with an applied or printed calibration target image, is positioned parallel to the front of the vehicle at a defined distance. The assistance system then captures actual images of the calibration target image and compares them with a calibration target image stored in the camera. The calibration is complete when this comparison, within a specified tolerance, demonstrates the required agreement between the actual image and the target image.
[0009] Furthermore, EP 2 233 365 A2 discloses a method for determining the position and orientation of a driver assistance system camera of a vehicle relative to the vehicle. In this method, at least one marker of a measuring target attached to the left rear wheel of the rear axle and at least one marker of a measuring target attached to the opposite right wheel of the rear axle are recorded by two measuring cameras in at least two vehicle positions. The geometric driving axis of the vehicle with respect to the two measuring cameras is also determined. Additionally, a driver assistance system calibration target is recorded with the two measuring cameras, and the position of the driver assistance system calibration target relative to the two measuring cameras is determined. Finally, the position and orientation of the driver assistance system calibration target relative to the geometric driving axis of the vehicle are determined.Furthermore, the driver assistance system calibration target is captured by the driver assistance system camera, and the position of the driver assistance system camera relative to the driver assistance system calibration target is determined from this. Finally, the position and orientation of the driver assistance system camera with respect to the vehicle's geometric driving axis are determined.
[0010] The object of the present invention is to provide a solution for how the calibration of a vehicle's environmental sensor can be carried out in a shorter time and yet reliably.
[0011] This problem is solved according to the invention by a method, a calibration test bench, and a vehicle with the features according to the independent claims. Advantageous embodiments of the present invention are specified in the dependent claims.
[0012] A method according to the invention serves to calibrate an environmental sensor of a vehicle. The method comprises performing a measurement with a measuring sensor of a calibration test stand and determining the pose of the vehicle relative to the measuring sensor based on the measurement. Furthermore, the method comprises detecting the calibration element of the calibration test stand using the environmental sensor and determining the pose of the environmental sensor relative to the calibration element based on the calibration element detected by the environmental sensor. It is provided that, during the measurement with the measuring sensor, a region of the vehicle's outer shell is detected, and to determine the vehicle's pose relative to the measuring sensor, the detected region of the outer shell is aligned with a three-dimensional model of the vehicle. The three-dimensional model describes the vehicle's outer shell.
[0013] This procedure is designed to calibrate at least one of the vehicle's environmental sensors. This procedure can be extended to include multiple environmental sensors. The environmental sensor could be, for example, a camera, a laser scanner, a lidar sensor, or similar. The calibration test bench is used to perform the calibration. The vehicle is moved into this test bench, for example, at the end of the production process. The test bench can have one or more measuring sensors with which the corresponding measurements can be taken. In particular, a distance can be determined using the measuring sensor. This measuring sensor could be, for example, a lidar sensor, a laser scanner, or a camera. Based on the measurement taken with the measuring sensor, the vehicle's position relative to the measuring sensor, or rather relative to the calibration test bench, can be determined.In this context, the term "pose" refers to position and orientation. The system aims to determine the position or relative orientation of the vehicle to the measurement sensor, as well as the vehicle's orientation relative to the measurement sensor. Furthermore, the system intends to determine the position of the environmental sensor relative to the calibration element of the calibration test bench based on a measurement taken by the environmental sensor. The calibration element could be, for example, a calibration target or a marker. Such a calibration target could have a pattern, such as a checkerboard pattern. The calibration element could also be an image of an object or the like. It is also possible to determine the distance between the environmental sensor and the calibration element.
[0014] According to a key aspect of the present invention, the measurement with the measuring sensor captures a region of the vehicle's outer shell or outer contour. For example, the spatial position of the outer shell or the region of the outer shell can be determined. In particular, the outer shell can be a car body or a part of the car body that is captured by the measuring sensor. Furthermore, to determine the vehicle's pose relative to the measuring sensor, the captured region of the outer shell is aligned with the three-dimensional model of the vehicle. This three-dimensional model describes the vehicle's outer shell. The determined region of the outer shell is then aligned or matched with the three-dimensional model.In this way, the measurement of the vehicle's outer shell or body can be improved from the perspective of the calibration stand. This means that the vehicle's position and body orientation can be measured by the calibration test stand regardless of the vehicle's position and orientation. Subsequently, the digital three-dimensional model of the vehicle is matched to these measurements. This three-dimensional model can, for example, describe the vehicle's outer shell or body. By matching the three-dimensional model with the measurements from the measuring sensor, the vehicle's pose within the calibration test stand can be determined simply and reliably. This allows conclusions to be drawn about the extrinsic parameters of the environmental sensor and the vehicle's reference coordinate system.Furthermore, it is not necessary to align the vehicle within the calibration test stand or lock it into a specific position. In addition, no special floor structure for the calibration test stand, such as a horizontal roller test stand, is required. Moreover, it is not necessary to attach any markings or markers to the vehicle. This also saves time during calibration. Overall, calibration can thus be carried out simply yet reliably.
[0015] Preferably, measurement points describing the area of the vehicle's outer shell are determined using the measuring sensor. These measurement points are then aligned with the three-dimensional model. The measuring sensor can thus determine multiple measurement points that describe the spatial position of the vehicle's outer shell within the calibration test rig. The measurement points determined by the measuring sensor are used to match the three-dimensional model, or the digital 3D model, to the pose of the actual vehicle. Using the matched digital 3D model of the vehicle to be calibrated, the precise position and orientation within the calibration test rig can be determined. Furthermore, the vehicle's pose relative to the measuring sensor can be established.Methods such as Iterative Closest Point or Physics-Based Registration can be used to match the digital 3D model to the measurement points of the calibration test bench. This enables a simple and reliable determination of the vehicle's pose within the calibration test bench.
[0016] In a further embodiment, the installation position of the environmental sensor on the vehicle is determined based on the aligned three-dimensional model. For example, information about the installation position of the environmental sensor can be stored in the three-dimensional model. Furthermore, an additional recalibration can be performed. This allows for the correction of any potential discrepancy between the actual installation position and the installation position stored in the 3D model. Information describing the installation position of the environmental sensor on the vehicle can also be included. After matching the 3D model with the measurement taken by the measuring sensor, the pose of the environmental sensor relative to the vehicle can then be determined. Based on the pose of the environmental sensor relative to the calibration element, the extrinsic parameters of the environmental sensor can then be determined.This can be done for several or all of the vehicle's environmental sensors.
[0017] Furthermore, it is preferably provided that the installation position of the environmental sensor is also determined based on the pose of the measurement sensor relative to the calibration element. The relative position between the measurement sensor and the calibration element can be known, since the dimensions of the calibration test stand, and thus the position of the calibration element and the measurement sensor, are also known. It can also be provided that the pose of the measurement sensor relative to the calibration element is determined or measured only once. In this case, the calibration element can also be detected using the measurement sensor. This enables simple calibration of the environmental sensor.
[0018] In another embodiment, the installation position of the environmental sensor is determined relative to a reference point on the vehicle. For example, this reference point could be the center point of the vehicle's rear axle. Alternatively, the reference point could be the upper edge of the wheel arch above the rear axis of rotation. This is a fixed point on the vehicle body, excluding the suspension. The three-dimensional model can, for example, contain the relative position between the environmental sensor and the vehicle's reference point. It can also be provided that the relative position between the reference point and the environmental sensor is known. By matching the three-dimensional model, the pose of the measuring sensor relative to the reference point can be determined.Thus, the position of the environmental sensor relative to the reference point can be determined based on the position of the environmental sensor relative to the calibration element, the position of the measurement sensor relative to the reference point, and the position of the measurement sensor relative to the calibration element. Appropriate calculations can be performed for this purpose. For example, the translations and rotations of the positions can be calculated. This allows the position of the environmental sensor relative to the vehicle or the reference point to be determined easily. This can be done for multiple or all environmental sensors of the vehicle.
[0019] This allows the extrinsic parameters for calibration to be determined.
[0020] In a further embodiment, data describing the vehicle's position relative to the measurement sensor and / or the installation position of the environmental sensor are transmitted to a vehicle control unit. For example, the vehicle may have a corresponding driver assistance system which, in addition to at least one environmental sensor, includes a corresponding control unit. When the environmental sensor detects the calibration element of the calibration test bench, the control unit can determine the relative position or pose of the environmental sensor with respect to the calibration element. The data describing the vehicle's position and / or the installation position of the environmental sensor can be transmitted from the calibration test bench or a corresponding computing unit of the calibration test bench to the vehicle's control unit. In particular, it is provided that this data is transmitted wirelessly to the control unit.Thus, the vehicle's control unit contains information describing the position of the environmental sensor relative to the calibration element, and based on this data, the control unit also contains information describing the vehicle's position relative to the calibration test bench or the measurement sensor, and / or the installation position of the environmental sensor. This allows for reliable calibration of the environmental sensor.
[0021] Furthermore, it is advantageous if the vehicle's position relative to the measurement sensor and the position of the environmental sensor relative to the calibration element are determined during vehicle movement relative to the calibration test stand. In other words, calibration can be performed while the vehicle is in motion or driving. By performing the calibration on a moving vehicle, time can be saved. This requires only the three-dimensional model of the vehicle and knowledge of the geometry of the calibration test stand and the position of the measurement sensor relative to the calibration element. The other parameters required for calibration are determined by the calibration test stand during the vehicle's movement.
[0022] The system is specifically designed to use a measurement sensor to detect the area of the vehicle's outer shell along a side. In other words, the measurement sensor can be positioned laterally on the calibration test stand, allowing it to detect the vehicle, or more specifically its side, as the vehicle moves through the test stand. Reducing calibration time can lead to cost savings, particularly in production lines. Multiple measurement sensors can be mounted on the test stand. At least one of these sensors can be positioned anywhere in the vicinity of the vehicle or on the test stand. The measurement sensor can also detect the front and / or rear of the vehicle.In particular, at least one measurement sensor can be positioned at such a height on the calibration test stand that the vehicle can drive underneath it. Alternatively, at least one measurement sensor can be recessed into the ground.
[0023] Furthermore, it is advantageous to determine the intrinsic parameters of the environmental sensor when calibrating it. For example, if the environmental sensor is a camera, the intrinsic parameters could describe its focal length or similar characteristics. This enables reliable calibration of the environmental sensor.
[0024] A calibration test bench according to the invention for a vehicle's environmental sensor comprises a measuring sensor for performing measurements and a computing unit for determining the vehicle's pose relative to the calibration test bench's measuring sensor based on the measurement performed by the measuring sensor. The measuring sensor is configured to detect an area of the vehicle's outer shell during measurement. Furthermore, the computing unit is configured to align the detected area of the outer shell with a three-dimensional model of the vehicle, which describes the vehicle's outer shell, in order to determine the vehicle's pose relative to the measuring sensor or the calibration test bench. As previously described, the measuring sensor can be configured as a lidar sensor, laser scanner, or camera. In particular, the measuring sensor is configured as a distance sensor.The computing unit can also determine data describing the vehicle's position relative to the measurement sensor. Furthermore, the computing unit can determine the installation position of the environmental sensor. Thus, data describing the vehicle's position relative to the calibration test stand and / or the installation position of the environmental sensor can be transmitted by a corresponding transmitter on the calibration test stand.
[0025] A vehicle according to the invention comprises at least one environmental sensor. This environmental sensor is configured to detect a calibration element of the calibration test bench. Furthermore, a control unit of the vehicle is configured to determine the position of the environmental sensor relative to the calibration element based on the calibration element detected by the environmental sensor. The control unit is also configured to receive data describing the position of the vehicle relative to the environmental sensor and / or the calibration test bench. Thus, the control unit contains data describing the position of the vehicle relative to the measurement sensor and / or the installation position of the environmental sensor. Additionally, the control unit contains information based on the measurements of the environmental sensor describing the position of the environmental sensor relative to the calibration element. Furthermore, the control unit may also store data describing the position of the measurement sensor relative to the calibration element.Thus, a corresponding calibration can be carried out using the control unit. A system according to the invention comprises a calibration test bench according to the invention and a vehicle.
[0026] The preferred embodiments and their advantages presented with reference to the method according to the invention apply accordingly to the calibration test bench according to the invention, to the vehicle according to the invention, and to the system according to the invention. Exemplary embodiments of the invention are described below. To this end, the following is shown: Fig. 1 a vehicle with a plurality of environmental sensors, which is located in a calibration test bench for the purpose of calibrating the environmental sensors; Fig. 2 the vehicle according to Fig. 1 in a top view, with the vehicle moving through the calibration test stand; Fig. 3 a three-dimensional model of the vehicle, which is aligned with measuring points determined by measurement sensors of the calibration test stand; and Fig. 4 the vehicle in the calibration test stand, as well as different dimensions to be able to determine a pose of an environment sensor to a calibration element of the calibration test stand.
[0027] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.
[0028] In the figures, identical reference symbols denote functionally equivalent elements.
[0029] Fig. 1 shows a system 1 , which is a vehicle 2 as well as a calibration test bench 3 includes. Regarding the vehicle 2The vehicle shown here in a front view is a passenger car. 2 includes a driver assistance system 4 , which includes a number of environmental sensors 5 as well as a control unit 6 When the environmental sensors 5 The vehicle includes 2 both cameras 7 as well as lidar sensors 8 These environmental sensors 5 They need to be calibrated.
[0030] The calibration test bench 3 includes calibration elements 9 in the form of calibration targets. Furthermore, the calibration test bench includes 3 at least one surveying sensor 10 In the present embodiment, the calibration test bench comprises 3 two surveying sensors 10 , which are designed as lidar sensors. With the respective surveying sensors 10 can be an area of an outer shell 11or the body of the vehicle 2 be recorded. The calibration test bench also includes 3 a computing facility 12 .
[0031] Fig. 2 shows the vehicle 2 according to Fig. 1, which is carried out by the calibration test bench 3 is moved through it. The line describes 13 a trajectory of the vehicle 2 during the calibration process. During the calibration process, the environmental sensors are used. 5 the calibration elements 9 Recorded. Based on the recording of the calibration element. 9 using the environmental sensor 5 can a pose P_CK the environmental sensor 5 to the calibration element 9 to be determined. In addition, the surveying sensors are used. 10 of the calibration test bench 3 each area of the outer shell 11 of the vehicle 2A multiple measurement points are recorded. 16 determines which position of an area of the outer shell 11 of the vehicle 2 describe. The surveying sensors are shown here. 10 side of the calibration test bench 3 arranged so that when the vehicle moves 2 through the calibration test bench 3 each side area 14 of the vehicle 2 is recorded. Basically, the surveying sensors can 10 but arbitrarily on the calibration test bench 3 be arranged.
[0032] Fig. Figure 3 shows a three-dimensional model 15 of the vehicle 2 Furthermore, the majority of measuring points 16 shown, which are equipped with the surveying sensors 10 be provided and which form the outer shell 11 of the vehicle 2 describe. The three-dimensional model describes15 the outer shell 11 of the vehicle 2 The intention here is that the three-dimensional model 15 with the measuring points 16 is brought into agreement. The three-dimensional model can be used for this purpose. 15 - as indicated by the arrows - can be moved translationally and / or rotationally.
[0033] In Fig. Figure 4 is an example of how a pose can be. P_CF an environmental sensor 5 or one of the cameras 7 with respect to a reference point 17 of the vehicle 2 can be determined. The reference point 17 This corresponds to the center point of a rear axle. 18 of the vehicle 2 As previously described, the pose is adjusted during calibration. P_CK the environmental sensor 5 to the calibration element 9 This is determined based on the measurement from the environmental sensor. 5Furthermore, a pose will be P_LF the surveying sensor 10 to the vehicle 2 or the reference point 17 of the vehicle 2 Definitely. This pose P_LF can be based on a measurement from the surveying sensor 10 and the subsequent matching with the digital three-dimensional model 15 The position of the reference point is determined here. 17 in the vehicle 2 known. In addition, a pose can P_KL the surveying sensor 10 to the calibration element 9 be determined. This pose P_KL The pose is known or will be determined or measured at some point. P_CF can be done using the poses P_CK , P_KL and P_LF This can be determined by the rotations and translations of the poses. P_CK , P_KL and P_LF They will be charged. Furthermore, the poses can be... P_CK and P_CF as well as all intrinsic parameters and distortion parameters of the camera lens 7 This determines the overall reliability of the environmental sensors. 5 within a shorter timeframe and in a simple way. QUOTES INCLUDED IN THE DESCRIPTION
[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature
[0000] DE 102010004233 B3
[0006] WO 2015 / 090553 A1
[0007] WO 2017 / 016541 A1
[0008] EP 2233365 A2
[0009]
Claims
[1] Method for calibrating an environmental sensor (5) of a vehicle (2) comprising the steps: - Performing a measurement with a measuring sensor (10) of a calibration test stand (3) and determining a pose (P_LF) of the vehicle (2) relative to the measuring sensor (3) based on the measurement, - Detecting a calibration element (9) of the calibration test stand (3) using the environment sensor (5) and determining a pose (P_CK) of the environment sensor (5) to the calibration element (9) based on the calibration element (9) detected by the environment sensor (5), characterized by , that - during measurement with the measuring sensor (10) an area of an outer shell (11) of the vehicle (2) is recorded and - to determine the pose (P_LF) of the vehicle (2) to the measurement sensor (10) the detected area of the outer shell (11) is aligned with a three-dimensional model (15) of the vehicle (2), wherein the three-dimensional model (15) describes the outer shell (15) of the vehicle (2). [2] Method according to claim 1, characterized by , that measurement points (16) are determined based on the measurement with the surveying sensor (10) which describe the recorded area of the outer shell (11), and the measurement points (16) are aligned with the three-dimensional model (15). [3] Method according to claim 1 or 2, characterized by , that the installation position of the environment sensor (5) on the vehicle (2) is determined on the basis of the aligned, three-dimensional model (15). [4] Method according to claim 3, characterized by, that the installation positions of the environment sensor (5) are also determined based on a pose (P_KL) of the measuring sensor (10) to the calibration element (9). [5] Method according to claim 3 or 4, characterized by , that the installation position of the environment sensor (5) is determined with respect to a reference point (17) of the vehicle (2). [6] Method according to any one of the preceding claims, characterized by , that data describing the pose (P_LF) of the vehicle (2) to the measuring sensor (10) and / or the installation position of the environmental sensor (5) are transmitted to a control unit (6) of the vehicle (2). [7] Method according to any one of the preceding claims, characterized by , that the pose (P_LF) of the vehicle (2) relative to the measurement sensor (10) and the pose (P_CK) of the environment sensor (5) relative to the calibration element (9) are determined during a movement of the vehicle (2) relative to the calibration test stand (3). [8] Method according to any one of the preceding claims, characterized by , that the area of the outer shell (11) on a side area (14) of the vehicle (2) is measured by means of the measuring sensor (10). [9] Method according to any one of the preceding claims, characterized by , that intrinsic parameters of the environmental sensor (5) are also determined to calibrate the environmental sensor (5). [10] Calibration test bench (3) for calibrating an environmental sensor (5) of a vehicle (2) comprising - a surveying sensor (10) for performing measurements and - a computing device (12) for determining a pose (P_LF) of the vehicle (2) to the measuring sensor (3) based on the measurement carried out by means of the measuring sensor (10) characterized by , that - the measuring sensor (10) is designed to detect an area of an outer shell (11) of the vehicle (2) during measurement, and - the computing device (12) is configured to align the measured area of the outer shell (11) with a three-dimensional model (15) of the vehicle (2) in order to determine the pose (P_LF) of the vehicle (2) to the measuring sensor (10), wherein the three-dimensional model (15) describes the outer shell (11) of the vehicle (2). [11] vehicle (2) comprising - an environmental sensor (5) for detecting a calibration element (9) of a calibration test stand (3) and - a control unit (6) which is configured to determine a pose (P_CK) of the environment sensor (5) to the calibration element (9) on the basis of the calibration element (9) detected by the environment sensor (5), characterized by , that - the control unit (6) is designed to receive data describing a pose (P_LF) of the vehicle (2) to a measurement sensor (10) of the calibration test stand (3).