Method for calibrating sensor information from a vehicle and driver assistance system - Patents.com
The method aligns sensor information from different types by generating and comparing three-dimensional representations to achieve accurate online calibration, addressing the accuracy gap in existing sensor calibration methods for autonomous driving.
Patent Information
- Application Number
- JP2024505490
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-05-20
- Filing Date
- 2022-04-07
- Publication Date
- 2026-01-16
- Estimated Expiration
- 2042-04-07
AI Technical Summary
Existing methods for sensor calibration in vehicles fail to achieve the accuracy required for autonomous driving functions as they do not calibrate multiple sensors relative to each other, leading to unreliable environmental perception.
A method that calibrates sensor information from different types (e.g., radar, LIDAR, camera) by generating and comparing three-dimensional representations of the environment, determining differences, and adjusting calibration parameters to align these representations, enabling online sensor calibration during vehicle movement.
Enables highly accurate sensor calibration necessary for reliable autonomous driving by aligning sensor information through iterative comparison and correction of three-dimensional representations, improving accuracy and reliability of environmental detection.
Smart Images

Figure 0007801424000001 
Figure 0007801424000002 
Figure 0007801424000003
Abstract
Description
[Technical Field]
[0001] The present invention relates to a method for online calibration of sensor information from a vehicle and a driver assistance system. [Background technology]
[0002] For autonomous driving, it is essential that the perception of the environment is as reliable as possible. In this context, the environment is detected by sensors of different sensor types, such as at least one radar sensor, one or more cameras, and preferably at least one LIDAR sensor. A global 360° 3D detection of the environment is preferred, so that all static and dynamic objects in the vehicle's environment can be detected.
[0003] In order to ensure reliable detection of the environment, accurate calibration of the sensors, in particular with respect to each other, is necessary. In this respect, constant monitoring of the calibration status of the sensor system and recalibration, if necessary, during the journey is essential for highly automated driving functions, as failure of the autonomous driving functions would otherwise result.
[0004] In known methods, sensors are individually calibrated relative to fixed points on the vehicle, but the entire set of sensors is not calibrated relative to each other, which has the drawback that it is often not possible to achieve the accuracy required for autonomous driving functions. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] German patent application DE 10 2019 208 216 A1 Summary of the Invention [Problem to be solved by the invention]
[0006] Based on this, it is an object of the present invention to provide a method for calibrating sensor information from a vehicle that provides reliable and highly accurate online sensor calibration, i.e., that allows sensor information from different sensor types to be calibrated against each other while the vehicle is moving. [Means for solving the problem]
[0007] This object is achieved by a method with the features of independent claim 1. Preferred embodiments are the subject of the subclaims. A driver assistance system is the subject of alternative independent claim 15.
[0008] According to a first aspect, the present invention relates to a method for calibrating sensor information of a vehicle, the vehicle comprising at least one sensor of a first sensor type and at least one sensor of a second sensor type different from the first sensor type, where "different sensor types" means that the sensors use different methods or technologies to detect the environment, for example based on different types of electromagnetic waves (radar, LIDAR, ultrasound, visible light, etc.).
[0009] The method includes the following steps.
[0010] First, the environment is sensed during movement of the vehicle by at least one sensor of a first sensor type, in which case first sensor information is provided by the sensor of this first sensor type.
[0011] Similarly, the environment is sensed by at least one sensor of a second sensor type during vehicle movement. In this regard, second sensor information is provided by the sensor of the second sensor type. The first and second sensor information can be sensed simultaneously or at least overlapping in time. It is understood that the first and second sensor information relate to at least partly the same environmental region and therefore have at least partly the same coverage.
[0012] A first three-dimensional representation of the environmental information is then generated from the first sensor information, the first three-dimensional representation of the environmental information being, in particular, a 3D point cloud representing the vehicle's surroundings based on a plurality of points in three-dimensional space.
[0013] Furthermore, a second three-dimensional representation of the environmental information is generated from the second sensor information, and the second three-dimensional representation of the environmental information is also, in particular, a 3D point cloud reflecting the vehicle's surrounding environment based on a plurality of points in three-dimensional space.
[0014] The first and second three-dimensional representations of the environmental information or information derived therefrom are then compared. "Comparing" in the sense of the present disclosure is understood to mean, in particular, correlating the first and second three-dimensional representations of the environmental information with each other so that their correspondence can be confirmed. In particular, this can mean determining regions in the first and second three-dimensional representations of the environmental information that correspond to each other. "Information derived therefrom" is understood to mean any information that can be obtained from the first and second three-dimensional representations by any kind of data processing, such as, for example, data reduction, filtering, etc.
[0015] Next, differences between the first and second three-dimensional representations of the environmental information or between information derived therefrom are determined. In particular, it is checked whether there are differences between corresponding regions in the first and second three-dimensional representations as a whole, which differences may be attributed to improper calibration of the sensors. For example, an offset between corresponding regions in the first and second three-dimensional representations (which offset increases with distance from the vehicle) may be due to improper calibration of the roll, pitch, and / or yaw angles of the sensors.
[0016] After determining the difference, correction information is calculated for calibration parameters of the at least one sensor based on the determined difference, the correction information may provide, among other things, an indication of how the calibration of one or more sensors needs to be changed to achieve an improved match between the first and second three-dimensional representations of the environment information.
[0017] Finally, the vehicle's sensors are calibrated to one another based on the calculated correction information, the sensor calibration including in particular software-based calibration, i.e., the sensor information provided by one or more sensors is adapted based on the correction information so as to achieve an improved match between the first and second three-dimensional representations of the environmental information.
[0018] It is understood that it is also possible to calibrate sensors of two or more different sensor types based on the proposed method.
[0019] The technical advantage of the proposed method is that by converting information from multiple different sensors into a 3D representation of the environment, the sensor information can be compared with each other, thereby enabling online calibration of sensors based on the surrounding environment information obtained while the vehicle is moving. This enables highly accurate sensor calibration, which is necessary for reliable and accurate detection of the surrounding environment in autonomous driving functions of vehicles.
[0020] According to an exemplary embodiment, the first and second three-dimensional representations of the environmental information are discrete-time information. In other words, the sensors do not provide continuous-time information, but rather environmental information at discrete points in time, such as at a particular clock rate. Before comparing the first and second three-dimensional representations of the environmental information, or information derived therefrom, the information is synchronized with each other in time. Thus, it is possible to reduce inaccuracies in the match between the first and second three-dimensional representations of the environmental information, which may result from a temporal offset in the environmental information due to different sensors, e.g., different clock rates or different detection times.
[0021] If it is not possible to synchronize the first and second three-dimensional representations of the environmental information in time, an interpolation of information between two time steps of the discrete-time information can be performed before comparing the first and second three-dimensional representations of the environmental information or information derived therefrom. In this way, intermediate values of the sensor information or the three-dimensional representation of the environmental information can be obtained between two successive time steps, thereby achieving an improved matching accuracy.
[0022] According to an exemplary embodiment, first and second three-dimensional representations of environmental information, each reflecting the vehicle environment at the same time, are compared with each other, and differences between the first and second three-dimensional representations of environmental information are used to calculate correction information. In particular, it is checked whether it is possible to determine, among multiple differences existing between corresponding information in the first and second three-dimensional representations of environmental information, differences that are attributable to calibration errors of one or more sensors. If such differences are determined, an attempt can be made to adjust the calibration of one or more sensors so as to reduce the differences, i.e., so as to improve the accuracy of the match between the first and second three-dimensional representations of environmental information.
[0023] According to an exemplary embodiment, correction information for the calibration parameters is calculated iteratively, i.e., in multiple iteration steps, in such a way that first and second three-dimensional representations of at least one environmental information reflecting the vehicle's surrounding environment at the same time are compared with each other to calculate the correction information, and information regarding the agreement between the first and second three-dimensional representations of the environmental information is determined after application of the correction information to the calibration parameters of the at least one sensor, so that the calibration of the sensor can be iteratively improved.
[0024] According to an exemplary embodiment, the correction information is iteratively modified in successive iteration steps so as to reduce a match error between the first and second three-dimensional representations of the environmental information. For example, after determining the correction information in an iteration step, the correction information is applied, thereby modifying the sensor calibration. As a result, the first and / or second three-dimensional representations of the environmental information are modified and checked for consistency. This cycle is performed several times until a termination criterion is met. In this way, the sensor calibration is iteratively improved.
[0025] According to an exemplary embodiment, a minimization or optimization method is used to reduce the matching error. An example of such a method is an iterative nearest neighbor algorithm. When executing this algorithm, an attempt is made to bring the first and second three-dimensional representations of the environmental information as close as possible, for example, by rotation and translation. For example, corresponding points of the first and second three-dimensional representations of the environmental information are determined, and then, for example, a sum of squared distances is formed over all of these pairs of points. This provides a quality metric for the correspondence between the three-dimensional representations of the environmental information and / or the 3D point clouds. The goal of the algorithm is to minimize this quality metric by changing the transformation parameters (i.e., the rotation and translation parameters). As a result, the degree of correspondence between the three-dimensional representations of the environmental information obtained by different sensors can be successively improved.
[0026] According to an exemplary embodiment, the correction information for the calibration parameters is calculated using a plurality of first and second three-dimensional representations of the environmental information determined at different points in time, i.e. in such a way that a plurality of pairs of the first and second three-dimensional representations of the environmental information (each pair of environmental information representing the vehicle's surrounding environment at the same point in time) are compared with each other and the correction information is calculated. By comparing the first and second three-dimensional representations of the environmental information at a plurality of points in time, the accuracy of the sensor calibration can be further improved.
[0027] According to an exemplary embodiment, the sensor of the first sensor type is a camera. In particular, the camera can be designed to generate a two-dimensional image. Multiple sensors of the first sensor type can be provided to detect a wider range around the vehicle. In particular, sensors of the first sensor type can be used to generate a 360° representation of the surroundings, i.e., an all-around view in a horizontal plane.
[0028] According to an exemplary embodiment, the camera is a monocular camera, and from the image information provided by the camera, a 3D representation of the environment information is calculated from a single image or a sequence of temporally consecutive 2D images. For example, structure-from-motion, shape-from-focus, or shape-from-shading methods can be used. Depth estimation can also be performed using neural networks. This provides depth information for the 2D image information from the camera, which can then be used to generate a 3D representation of the environment information. Structure-from-motion methods typically assume a static environment.
[0029] It is also possible to obtain depth information on two-dimensional image information using one or more stereo cameras.
[0030] According to an exemplary embodiment, segmentation of moving objects contained in image information and estimation of the three-dimensional structure and relative movement of the segmented objects and the stationary surroundings are performed based on a time-sequential series of image information, in particular two-dimensional image information, from at least one camera, for example, by the method of German Patent Application DE 10 2019 208 216 A1. This allows for highly accurate determination of segmentation and structure information, even in dynamic environments. Information about the determined relative movement of the moving objects and the surroundings can advantageously be incorporated into the synchronization of the three-dimensional representations of all objects or into the interpolation between two time steps, which leads to higher accuracy in determining correction information for the calibration parameters.
[0031] According to an exemplary embodiment, the sensor of the second sensor type is a radar sensor or a LIDAR sensor.
[0032] According to an exemplary embodiment, moving objects are filtered from the first and second three-dimensional representations of the environmental information such that correction information is calculated based exclusively on stationary objects. Filtering moving objects can improve the accuracy of sensor calibration, since in the case of stationary objects, differences between the first and second three-dimensional representations of the environmental information can be used to directly infer calibration inaccuracies between the sensors.
[0033] According to another exemplary embodiment, the correction information is calculated based on a comparison of first and second three-dimensional representations of environmental information including only stationary objects and a comparison of first and second three-dimensional representations of environmental information including only moving objects. Thus, in addition to stationary objects, moving objects can also be used to calculate the correction information for the sensor calibration. However, it is desirable for movement information, such as trajectory and speed, to be known for moving objects so that the object's movement can be compensated for when calculating the correction information.
[0034] According to a further aspect, the present invention relates to a driver assistance system for a vehicle, the driver assistance system including a sensor of a first sensor type and at least one sensor of a second sensor type different from the first sensor type, the driver assistance system comprising the following steps: detecting an environment in which the vehicle is moving with at least one sensor of a first sensor type and providing first sensor information with the sensor of the first sensor type; detecting an environment in which the vehicle is moving with at least one sensor of a second sensor type and providing second sensor information with the sensor of the second sensor type; creating a first three-dimensional representation of the environmental information from the first sensor information; creating a second three-dimensional representation of the environmental information from the second sensor information; comparing the first and second three-dimensional representations of the environmental information, or information derived therefrom; determining a difference between the first and second three-dimensional representations of the environmental information or information derived therefrom; calculating correction information for calibration parameters of the at least one sensor based on the determined difference; calibrating the vehicle's sensors relative to each other based on the calculated correction information; The device is configured to:
[0035] The term "three-dimensional representation of environmental information" refers to any representation of environmental information in a three-dimensional coordinate system, for example, a discrete spatial representation of an object region in three-dimensional space.
[0036] The term "3D point cloud" as used in this disclosure is understood to mean a collection of points in three-dimensional space, each point indicating that an object part exists at the location in three-dimensional space where the point is found.
[0037] The term "sensor type" as used in this disclosure is understood to mean a sensor type that determines environmental information by a predetermined detection principle. The sensor type may be, for example, a camera, a radar sensor, a LIDAR sensor, an ultrasonic sensor, etc.
[0038] In the sense of the present invention, the expressions "approximately", "substantially" or "about" mean a deviation of ±10%, preferably ±5%, from the respective exact value and / or a deviation in the form of a change that is not important for functionality.
[0039] Further developments, advantages and possible uses of the invention can also be seen from the following description of exemplary embodiments and from the drawings, in which all features described and / or shown are in principle subject matter of the invention, individually or in any combination, regardless of their abstract or subsequent reference in the claims, the contents of which are also made part of this description.
[0040] The invention will now be explained in more detail using exemplary embodiments and with reference to the drawings. [Brief explanation of the drawings]
[0041] [Figure 1] 1 is an exemplary schematic diagram of a vehicle equipped with a driver assistance system including multiple sensors of different sensor types for detecting the vehicle's environment; [Figure 2] 1 is a flowchart illustrating method steps for calibrating camera sensor information and radar and / or LIDAR sensor information. [Figure 3] 1 is a schematic diagram illustrating exemplary method steps for online calibration of sensor information for different sensor types; DETAILED DESCRIPTION OF THE INVENTION
[0042] 1 shows, in an exemplary and schematic way, a vehicle 1 equipped with a driver assistance system that enables detection of the environment by means of a number of sensors 2, 3, 4 of different sensor types, at least some of which enable omnidirectional detection of the environment (360° detection of the environment).
[0043] The vehicle 1 includes at least one sensor 2 of a first sensor type, in particular a radar sensor. The first sensor type is thus based on radar principles. The sensor 2 can be provided, for example, in the front region of the vehicle. It is understood that multiple sensors 2 of the first sensor type can be provided distributed around the vehicle 1, for example in the front, rear, and / or side regions of the vehicle 1. The at least one sensor 2 of the first sensor type generates first sensor information, which can be, for example, raw information provided by a radar sensor. From this first sensor information, a first three-dimensional representation of environmental information is generated. In particular, this is a 3D point cloud. If multiple sensors 2 of the first sensor type are used, the first three-dimensional representation of environmental information can be generated based on sensor information from multiple or all of these sensors 2.
[0044] Furthermore, the vehicle 1 includes at least one sensor 3 of a second sensor type, which is a camera. Thus, the second sensor type is a "camera" type, i.e., an imaging sensor. The sensor 3 can be provided, for example, in the windshield area of the vehicle 1. It is understood that multiple sensors 3 of the second sensor type can be distributed around the vehicle 1, for example, in the front, rear, and / or side areas of the vehicle 1. The at least one sensor 3 of the second sensor type generates second sensor information, which is, for example, image information provided by a camera. The camera can provide two-dimensional image information of the environment, i.e., the image information does not contain depth information. In this case, the second sensor information can be further processed so that depth information of the image information can be obtained from changes in the image information in successive images of the image sequence. For this purpose, methods known to those skilled in the art for generating spatial correlations from two-dimensional image sequences can be used. For example, the structure-from-motion method, the shape-from-focus method, the shape-from-shading method, etc. Depth estimation using neural networks is also possible in principle. If the camera is a stereo camera, the second sensor information can also directly contain three-dimensional information, i.e., depth information for a portion of pixels or for each pixel of the image. From this second sensor information, a second three-dimensional representation of the environment information is generated. In particular, this is a 3D point cloud. If multiple sensors 3 of the second sensor type are used, the second three-dimensional representation of the environment information can be generated based on sensor information from several or all of these sensors 3.
[0045] Preferably, the vehicle 1 also includes at least one sensor 4 of a third sensor type, which is a LIDAR sensor. The third sensor type is therefore based on LIDAR principles. The sensor 4 may be provided, for example, on the roof of the vehicle 1. It is understood that multiple sensors 4 of the third sensor type may be distributed over the vehicle 1. The at least one sensor 4 of the third sensor type generates third sensor information, which may be raw information provided by a LIDAR sensor, for example. From this third sensor information, a third three-dimensional representation of the environmental information is generated, unless already provided by the third sensor information. In particular, this is a 3D point cloud. If multiple sensors 4 of the third sensor type are used, the third three-dimensional representation of the environmental information may be generated based on sensor information from multiple or all of these sensors 4.
[0046] Furthermore, the vehicle further comprises a computing unit 5 configured to further process the data provided by the sensors 2, 3, 4. This computing unit may be a central computing unit as shown in Figure 1, but it is also possible to provide a number of distributed computing units such that the subtasks of the method described below are executed in a distributed manner across several computing units.
[0047] FIG. 2 is a flow chart showing the steps of a method for calibrating the sensor information of different sensors 2, 3, 4 with respect to one another.
[0048] In step S10, sensor information of at least one radar sensor and / or at least one LIDAR sensor is received. If radar and LIDAR sensors are present, the sensor information is first provided separately for each type of sensor.
[0049] If these sensors do not already provide a 3D representation of the environment, specifically a 3D point cloud, a 3D representation is formed from the sensor information. If radar and lidar sensors are present, a 3D representation of the environment, specifically a 3D point cloud, is provided separately for each sensor type. A 3D point cloud can be formed from sensor information from a single sensor or by combining sensor information from multiple sensors of the same sensor type.
[0050] Preferably, in step S11, the 3D point clouds obtained from the sensor information of the radar sensor and, if present, the 3D point clouds obtained from the sensor information of the LIDAR sensor are separated according to static content and dynamic content. Specifically, for each sensor type, a first 3D point cloud including only static objects and a second 3D point cloud including only dynamic objects are created. This makes it possible to generate correction information for calibration parameters separately from static objects and dynamic objects.
[0051] Furthermore, in step S12, second sensor information is received from the camera, and in step S13, a 3D point cloud is generated from the second sensor information.
[0052] For example, by evaluating time-successive images of an image sequence of one or more cameras, a three-dimensional reconstruction of the environment of the vehicle 1 is performed, for example by structure-from-motion methods.
[0053] Preferably, the method disclosed in German Patent Application DE 10 2019 208 216 A1 is used, the disclosure of which is incorporated herein in its entirety. Preferably, this method performs both the output of a 3D reconstruction or 3D point cloud of the environment and the segmentation of moving objects. Thus, it is possible to separate moving and stationary objects in the image information provided by at least one camera (S14). Furthermore, the trajectory of the moving object can be determined by this method, as can the trajectory of the camera system relative to the stationary surroundings. Knowing the object motion also makes it possible to correlate 3D point clouds of different sensor types, including moving objects, with each other and thus derive correction information for calibration. This simplifies, in particular, the synchronization and interpolation steps and provides more accurate results.
[0054] After the separation of static and dynamic content in the 3D point clouds generated from the sensor information of the radar and / or LIDAR sensors and the camera sensor information has been performed, further method steps are performed only on the 3D point cloud containing static objects, or separate 3D point clouds containing static or dynamic objects are generated in each case, and further method steps are performed separately for the static and dynamic objects, i.e., both the 3D point cloud containing static objects and the 3D point cloud containing dynamic objects are compared and used to generate correction information for sensor calibration. Thus, the steps described below can be performed in parallel for the 3D point cloud containing dynamic objects and the 3D point cloud containing static objects.
[0055] Steps S10 / S11 and S12 / S13 / S14, i.e. processing the sensor information provided by the radar or LIDAR sensor and processing the sensor information provided by the camera, can be performed at least partly in parallel.
[0056] In step S15, the 3D point clouds are preferably synchronized with each other so that they can be checked for coincidence. On the one hand, this can be a temporal synchronization. Since the 3D point clouds of each sensor type are generated at different times, the ambient information in the 3D point clouds is locally offset from each other due to the vehicle's movement. This offset can be corrected by synchronizing the 3D point clouds in time. Furthermore, to compensate for the temporal offset between the 3D point clouds of each sensor, intermediate information can be calculated from multiple 3D point clouds that follow each other in time, for example by interpolation.
[0057] Subsequently, in step S16, the 3D point clouds are compared with each other, and differences between the 3D point clouds are determined. For example, corresponding points in the compared point clouds, i.e., points representing the same region of the surrounding scene (view), can be compared with each other to determine the distance between these points or their local offset from each other. Thus, in step S18, it can be determined which calibration inaccuracies exist between the sensors of the vehicle assistance system and which calibration parameters must be changed (e.g., linear offsets or differences due to sensor twist).
[0058] Subsequently, in step S18, the correction information is applied, i.e. the calibration parameters are modified based on the correction information, and then the 3D point clouds are checked again for coincidence and evaluated.
[0059] Then, in step S19, it is determined whether a sufficient match is found. If not, steps S16 to S19 are repeated. A linear gradient descent minimization procedure, for example an iterative closest point (ICP) method, can be performed.
[0060] Once a sufficient match between the 3D point clouds is achieved, correction information regarding the calibration parameters of the sensor is output and / or used to calibrate the sensor in step S20.
[0061] FIG. 3 shows a flow chart highlighting the steps of a method for online calibration of sensor information from a vehicle's sensors.
[0062] First, the environment is sensed while the vehicle is moving by at least one sensor of a first sensor type, and first sensor information is provided by the sensor of the first sensor type (S30).
[0063] Furthermore, the environment in which the vehicle is moving is detected by at least one sensor of a second sensor type, and in this connection second sensor information is provided by the sensor of this second sensor type (S31). Steps S31 and S32 are performed simultaneously or at least temporarily overlapping in time.
[0064] Next, a first three-dimensional representation of the environmental information is created from the first sensor information (S32).
[0065] Simultaneously with or at least overlapping in time with step S32, a second three-dimensional representation of the environmental information is created from the second sensor information (S33).
[0066] The first and second three-dimensional representations of the environment information or information derived therefrom are then compared with each other (S34), where "derived information" means any information that can be obtained from the first or second three-dimensional representations, for example by modification such as filtering, limiting to stationary or non-stationary objects, etc.
[0067] Based on the comparison, differences between the first and second three-dimensional representations of the environmental information or information derived therefrom are determined (S35).
[0068] Based on the determined difference, correction information for the calibration parameters of at least one sensor is calculated (S36). Finally, based on the calculated correction information, the sensors of the vehicle are calibrated relative to each other (S37). This means in particular that the position or orientation of the sensors on the vehicle is not changed, but an indirect calibration is performed by modifying the 3D point cloud based on the correction information.
[0069] The invention has been described above by means of exemplary embodiments, it being understood that many modifications and variations are possible without departing from the scope of protection defined by the claims. [Explanation of symbols]
[0070] 1 vehicle 2 sensors 3 sensors 4 sensors 5 computing units
Claims
1. 1. A method for calibrating sensor information of a driver assistance system of a vehicle, comprising: the driver assistance system of the vehicle comprises at least one sensor of a first sensor type and at least one sensor of a second sensor type different from the first sensor type; The method comprises: detecting an environment in which the vehicle is moving by at least one sensor of the first sensor type and providing first sensor information by at least one sensor of the first sensor type; detecting an environment in which the vehicle is moving with at least one sensor of the second sensor type and providing second sensor information with at least one sensor of the second sensor type; creating a first three-dimensional representation of environmental information from the first sensor information; creating a second three-dimensional representation of environmental information from the second sensor information; comparing the first and second three-dimensional representations of the environment information or information derived therefrom; determining a difference between the first and second three-dimensional representations of the environment information or information derived therefrom; calculating correction information for calibration parameters of at least one sensor based on the determined difference; calibrating the vehicle's sensors relative to each other based on the calculated correction information; Including, the first and second three-dimensional representations of environmental information are 3D point clouds representing an environment surrounding the vehicle based on a plurality of points in three-dimensional space; the step of comparing the first and second three-dimensional representations of the environment information or information derived therefrom comprises confirming a match between the first and second three-dimensional representations of the environment information; A method in which the calculation of correction information for the calibration parameters is performed in multiple iterative steps, in each case comparing at least one pair of first and second three-dimensional representations of environmental information reflecting the vehicle surrounding environment at the same point in time with each other to calculate correction information, and determining information regarding the agreement between the first and second three-dimensional representations of environmental information after application of the correction information to the calibration parameters of the at least one sensor.
2. 2. The method of claim 1, wherein the first and second three-dimensional representations of environmental information are discrete-time information, and the information is synchronized with respect to time before the first and second three-dimensional representations of environmental information or information derived therefrom are compared.
3. 2. The method of claim 1, wherein the first and second three-dimensional representations of environmental information are discrete-time information, and an interpolation of information between two time steps of the discrete-time information is performed before the first and second three-dimensional representations of environmental information or information derived therefrom are compared.
4. 2. The method of claim 1, wherein the sensor of the first sensor type is a camera.
5. The method of claim 1 , wherein the sensor of the second sensor type is a radar sensor or a LIDAR sensor.
6. 2. The method of claim 1, wherein moving objects are filtered from the first and second three-dimensional representations of the environment information, such that calculation of the correction information is performed exclusively based on stationary objects.
7. 1. A driver assistance system for a vehicle having a sensor of a first sensor type and at least one sensor of a second sensor type different from the first sensor type, comprising: The driver assistance system includes: detecting an environment in which the vehicle is moving by at least one sensor of the first sensor type and providing first sensor information by at least one sensor of the first sensor type; detecting an environment in which the vehicle is moving with at least one sensor of the second sensor type and providing second sensor information with at least one sensor of the second sensor type; creating a first three-dimensional representation of environmental information from the first sensor information; creating a second three-dimensional representation of environmental information from the second sensor information; comparing the first and second three-dimensional representations of the environment information or information derived therefrom; determining a difference between the first and second three-dimensional representations of the environment information or information derived therefrom; calculating correction information for calibration parameters of at least one sensor based on the determined difference; calibrating the vehicle's sensors relative to each other based on the calculated correction information; configured to run the first and second three-dimensional representations of environmental information are 3D point clouds representing an environment surrounding the vehicle based on a plurality of points in three-dimensional space; the step of comparing the first and second three-dimensional representations of the environment information or information derived therefrom comprises confirming a match between the first and second three-dimensional representations of the environment information; a driver assistance system, wherein the calculation of correction information for the calibration parameters is performed in a plurality of iterative steps, in each case in which at least one pair of first and second three-dimensional representations of environmental information reflecting the vehicle surrounding environment at the same point in time are compared with each other to calculate correction information, and information regarding the agreement of the first and second three-dimensional representations of environmental information is determined after application of the correction information to the calibration parameters of the at least one sensor.
Citation Information
Patent Citations
Detection, 3D reconstruction and tracking of multiple rigid objects moving relative to each other
DE102019208216A1
Signal processing device and signal processing method
WO2017159382A1