Automated external calibration and calibration verification of various sensor modalities, such as cameras, radars, and lidar sensors

Abstract

The present invention is a method for calibrating first and second sensors of a vehicle, comprising: - obtaining first data from a first sensor and second data from a second sensor; - filtering at least the second data based on the positions of data points of the second data; - determining one or more parameters of calibration between the first and second sensors based on the first data and the filtered second data.

Description

Technical Field

[0001] The present invention relates to a method for calibrating first and second sensors of a vehicle, and a calibration unit for calibrating first and second sensors of a vehicle.

[0002] The present invention also relates to a computer-readable storage medium storing program code, the program code including instructions for performing such a method.

Background Art

[0003] Modern vehicles include an increasing number of sensors of various modalities, both autonomous and non-autonomous. These sensors need to be calibrated. For example, the internal calibration of a camera involves determining a set of parameters for a camera projection model that associates 2D image points with 3D scene points, and the optimal parameters correspond to the minimum reprojection error. When there are sensors of multiple modalities, external calibration is required. External calibration can involve determining parameters that match points from one sensor to corresponding points of another sensor.

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is an object of the present invention to provide a method for calibrating first and second sensors of a vehicle, and a calibration unit for calibrating first and second sensors of a vehicle, which overcome one or more of the above problems of the prior art.

Means for Solving the Problems

[0005] A first aspect of the present invention is a method for calibrating first and second sensors of a vehicle, comprising: - obtaining first data from a first sensor and second data from a second sensor; - filtering at least the second data based on the positions of the data points of the second data; - providing a method including determining one or more parameters of calibration between the first and second sensors based on the first data and the filtered second data.

[0006] The method of the first aspect has the advantage that the calibration is performed based on the filtered second data. The filtering can be performed so that clearly irrelevant points are excluded, and the calibration is performed on the points that have not been excluded. Therefore, the probability of incorrect calibration can be reduced, and the overall reliability of the calibration is improved.

[0007] The first and / or second sensor can include a camera. The step of obtaining the first and / or second data can include the step of obtaining the first and / or second image.

[0008] As will be further outlined hereinafter, it is understood that this method can simultaneously calibrate additional sensors in addition to the first and second sensors.

[0009] Filtering at least the second data means that optionally the first data can also be filtered. Filtering at least the second data can include excluding, i.e., removing, data points of the second data that have positions within certain one or more regions, for example one or more predetermined specific regions. For example, data points having positions outside a specific target region can be removed. The target region can depend on characteristics of the vehicle's steering, for example, the steering direction.

[0010] Calibration can include alignment between the first and second sensors. For example, calibration can include three parameters for translation and three parameters for rotation between the coordinate systems of the first and second sensors.

[0011] In a first embodiment of the method according to the first aspect, the method further includes the step of using pre-calibration to convert the first data and / or the second data into a common domain.

[0012] Using pre-calibration has the advantage that prior knowledge about the relative orientation between the first and second sensors can be used as a starting point for determining the final calibration between the first and second sensors. For example, knowledge about the mounting position of the first sensor relative to the mounting position of the second sensor can be used to determine the pre-calibration.

[0013] In a further embodiment of the method according to the first aspect, the method further includes the steps of detecting a first object in the first data and / or detecting a second object in the filtered second data, and performing a determination of one or more parameters based on the first and / or second objects.

[0014] Performing calibration based on the detected objects has the advantage that an accurate calibration can be performed based on the detected objects that can be fully corresponding in the first and second data, even if the first and second sensors use different modalities (which may result in different types of data points in the first and second data).

[0015] In a further embodiment of the method according to the first aspect, the step of determining one or more parameters is based on the center points of the first and / or second objects.

[0016] For example, a first sensor may acquire a large number of data points regarding a given object, while a second sensor may require fewer data points regarding the same given object. However, it is possible to determine the same center point from the data points on the first sensor and the data points of the second sensor.

[0017] In a further embodiment of the method according to the first aspect, the first sensor and / or the second sensor includes one or more of a normal camera, a stereo camera, a radar, and a lidar.

[0018] In other embodiments, the first and / or second sensor can include any further sensor modality that provides appropriate data for performing this method.

[0019] In a further embodiment of the method according to the first aspect, one or more parameters of the calibration include yaw and pitch parameters, and the method further includes filtering second data based on a filter region having a region width around a center line directed to a point far from the sensor, where the far point has lateral and / or vertical coordinate components corresponding to the lateral and / or vertical coordinate components of the position of the sensor.

[0020] In other words, in this embodiment, the filter region can be a rectangular region in front of the vehicle (when viewed from above). Experiments have shown that this region is particularly useful as a filter region. Here, the filter region refers to a region where data points are not excluded, that is, a region where data points are retained and used in a further calibration process.

[0021] Preferably, the region width is determined based on a predetermined table that assigns a predetermined width to a given pair of the first and second sensors, and / or the region width is determined based on the position and / or orientation of the sensor.

[0022] In a further embodiment of the method according to the first aspect, the position of the remote point in the coordinates of the reference sensor is

Number

Number

Number

Number

[0023] K cal_sensor and K ref_sensor can be the identity matrix. Generally, the matrices used depend on the type of sensor used.

[0024] In a further embodiment of the method according to the first aspect, one or more parameters of the calibration include at least one roll parameter, and the method further includes filtering second data from the sensor based on a filtering region having a fixed width around a center line directed to a remote point that is laterally shifted from the sensor position by a lateral shift in the coordinates of the sensor.

[0025] In a further embodiment of the method according to the first aspect, the step of filtering the second data includes the step of performing filtering based on a filtering region, and the filtering region is adjusted based on the angle in the steering direction of the vehicle.

[0026] In a further embodiment of the method according to the first aspect, the step of determining the lateral location of the filtering region includes the step of evaluating the curvature of the angle in the steering direction, and preferably, the step of evaluating includes the step of multiplying the curvature of the angle in the steering direction by a sensitivity constant, and preferably, further includes the step of multiplying by a constant proportional to the focal length of the camera sensor.

[0027] In a further embodiment of the method according to the first aspect, the method is repeatedly executed using a plurality of candidate pre-calibrations between the first and second sensors, and preferably, the method includes an additional step of selecting a preferred pre-calibration from the plurality of candidate pre-calibrations.

[0028] In a further embodiment of the method according to the first aspect, the step of determining one or more parameters of the calibration between the first sensor and the second sensor is - determining a pairing between a point in the first data and a point in the second data; and - determining one or more parameters such that the error between the paired points is minimized.

[0029] In a further embodiment of the method according to the first aspect, the step of calibrating the first and second sensors further includes the step of calibrating a third sensor using the first and second sensors, and the step of determining one or more parameters of the calibration includes minimizing the error of the point pairs between the first and second sensors, the point pairs between the second and third sensors, and the point pairs between the third and first sensors.

[0030] Preferably, the step of minimizing the point pair error includes the step of minimizing the weighted sum of the point pair errors of the sensor pair.

[0031] In a further embodiment of the method according to the first aspect, the method further includes the step of verifying the pre-calibration, and the step of verifying includes comparing one or more parameters of the determined calibration with one or more corresponding parameters of the pre-calibration, and verifying the calibration if the difference between the determined calibration and the pre-calibration is less than a predetermined threshold.

[0032] In a further embodiment of the method according to the first aspect, the step of verifying, for each of a plurality of sensors including the first and second sensors, - determining a first center point of a detected object projected to the coordinates of another sensor among the plurality of sensors using the parameters of the determined calibration; - determining a second center point of the detected object projected to the coordinates of the other sensor among the plurality of sensors using the parameters of the pre-calibration; - determining the distance between the first and second center points, Preferably, if the distance is less than a predetermined threshold, the calibration between the sensor and the other sensor among the plurality of sensors is verified.

[0033] A second aspect of the present invention is a calibration system for calibrating first and second sensors of a vehicle, the calibration unit having - an acquisition unit for acquiring first data from the first sensor and second data from the second sensor; - a filtering unit for filtering at least the second data based on the positions of the data points of the second data; - Provide a calibration system including a determination unit for determining one or more parameters of calibration between a first sensor and a second sensor based on first data and filtered second data.

[0034] The calibration unit can be a calibration system.

[0035] A further aspect of the present invention refers to a computer-readable storage medium storing program code, where the program code includes instructions that, when executed by a processor, execute one of the methods of the second aspect or an embodiment of the first aspect.

[0036] For the purpose of more clearly showing the technical features of the embodiments of the present invention, the accompanying drawings provided for describing those embodiments are briefly introduced hereinafter. The accompanying drawings in the following description are only some embodiments of the present invention, and modifications to these embodiments are possible without departing from the scope of the present invention defined in the claims.

Brief Description of the Drawings

[0037]

Figure 1

Figure 2A

Figure 2B

Figures 3A-3B

Modes for Carrying Out the Invention

[0038] The foregoing description is only an embodiment of the present invention, and the scope of the present invention is not limited thereto. Any modification or replacement can be easily carried out by those skilled in the art. Therefore, the protection scope of the present invention should follow the protection scope of the appended claims.

[0039] From the perspective of autonomous driving, in order to improve road safety, a plurality of sensors, for example, a plurality of cameras, are important. However, processing the data of a plurality of sensors on a vehicle requires both accurate internal calibration and accurate external calibration for each camera. Accurate internal calibration of a camera can include an optimal set of parameters for a camera projection model that associates 2D image points with 3D scene points, and these optimal parameters correspond to the minimum reprojection error. Hereinafter, reference will be made instead to the task of external calibration. Accurate external calibration can correspond to an accurate camera pose with respect to a reference frame on the vehicle. Accurate calibration enables feature points from one sensor to be reprojected to another sensor with a low reprojection error.

[0040] During an automatic external calibration process (hereinafter simply referred to as calibration), the following data can be used. · Normal camera frames (2D), or depth images (3D) created from stereo cameras. · Radar point clouds (2D / 3D) · LiDAR point clouds (3D) · The sensor to be calibrated can provide additional data, such as speed, intensity, cross-sectional size, and / or detection reliability. These can be used to assist the calibration process. · For sensors that perform tracking, these trajectories can be further utilized to assist the calibration process. (These trajectories can also exhibit additional characteristics such as physical size, reliability, etc.) An important step before calibration is the pairing of the above detections between sensors. If the formats of the detections between any sensor pair are different, they are first converted to the same domain, substantially to the domain that supports their pairing. These common domains and corresponding conversions can be one or more of the following. · Normal camera frame + point cloud: The points of the cloud are projected into the image space of the camera. · Point cloud + point cloud with tracking information: Object detection is performed on a normal point cloud. These objects can then be paired with trajectories. · Point cloud + point cloud: Object detection-based and similar to the previous points. Also, note that since the densities of these point clouds can vary significantly, their one-to-one pairing may not be optional.

[0041] Lidar devices can detect tens of thousands of points in each measurement, while radar devices can only detect 10 to 100 points in each of their measurements, which makes it difficult to handle the one-to-one pairing of such data.

[0042] Examples of processing the input In the first embodiment, each detected object is represented with a single point. To calibrate a single camera and a single radar device, the following can be done.

[0043] To process the camera frame, it is possible to use various (state-of-the-art) image processing and / or object detection methods. For this example, the object to be detected is supposed to be a car visible on the camera frame. It is then possible to use a neural network trained to detect these objects and provide the position of these objects in the image space.

[0044] To process the radar frame, it is possible to use several methods, just as for the camera frame. For example, it is possible to use the relative speed provided by most radars to select which objects are moving and which are stationary. Furthermore, the point cloud can be clustered (for this purpose, several algorithms exist), and then the center point of the objects thus found can be easily set.

[0045] Figure 1 shows how sensor 100, e.g., a camera, views lane 120 with vehicles 122, 124, and 126 in front of the vehicle (not shown) carrying sensor 100.

[0046] When using different sensors, this method can be implemented differently. Using a radar device capable of tracking objects, the clustering step can be eliminated because the required positions are obtained from the device itself. For example, using a lidar instead of a camera, it is possible to cluster the lidar point cloud, just as for the radar point cloud.

[0047] Generally, for each of the detected objects, here the vehicles 122, 124, 126, the bounding boxes 132, 134, 136, and the center points of the bounding boxes 132, 134, 136 can be identified.

[0048] In FIG. 1, the x - marks 142, 144, 146 in the vehicle indicate the centers of the objects detected on the camera frame using a neural network. The o - marks 152, 154, 156 indicate the centers of the clusters detected on the radar frame projected onto the image. As can be understood from FIG. 1, the centers 142, 144, 146 of the objects detected on the camera frame are not perfectly aligned with the centers of the clusters detected on the radar frame.

[0049] Data Pairing An important aspect of the described calibration method is the way the data is paired. This can be a difficult task already, because, for example, the pre - calibration used to convert the data of the sensors to be paired into the same domain can be very corrupted. For this step, a method that can function with all possible combinations of sensor pairings is needed.

[0050] A common way of pairing is to select the closest (using some criterion, spatially) points between sensors. However, this method is very error - prone and requires the use of a more sophisticated method. Beyond this point, the literature does not provide any further guidance.

[0051] Using some filtering methods, sufficient points can be removed, thereby minimizing the possibility of obtaining incorrect pairings. The filtering is as follows. · Simple - Only one point (the center point of the object) per detected object needs to be handled. Filtering these data, for example, by their position, requires little computing power. · Robust - This can be implemented on all sensors.

[0052] For example, it is possible for both data from a first sensor and data from a second sensor to be filtered, and in that case, it is understood that different filtering rules can be applied to data from different sensors. At least one of the filtering methods applied is based on the position of the data, that is, the decision of which data points to retain and which to exclude is based on the position of the data points in the data.

[0053] Pre - calibration may contain significant errors. The goal of the presented method includes removing these errors. If there are no or minimal errors, the distance of the point pairs is minimal, and therefore the calibration result will be very similar to the prior result. Therefore, this method can also be used to verify existing calibration parameters.

[0054] The selection of sensor pairs can be done either by the user or by software. The pairs can be any combination of existing camera, radar, and lidar devices, provided they share a common field of view.

[0055] Generally, to calibrate the yaw and pitch of a sensor, the center of the sensor's field of view should be selected. To calibrate the roll of the sensor, the outer perimeter of the sensor's field of view should be selected. Also, for a second sensor, the selected field of view depends on the field of view and pre-calibration of the first sensor (e.g., one sensor may be facing forward and the other may be facing sideways).

[0056] For yaw and pitch calibration, the following steps can be utilized.

[0057] Select a virtual point (P cal_sensor ) in front of the sensor: X meters vertically from the sensor. (In our case, X = 50.) Horizontally and vertically, the position of the point is not different from the position of the sensor. (The point is defined in the coordinates of the sensor.) Selecting the width of the filtered area can be done by using a predefined list that includes each sensor. (In our case, the width was about 3 - 4 meters, which is equal to the width of a road lane.) · The values of this list should be defined by the user. Those values should be selected considering the position and orientation of the sensor, the possible environment, and the objects that will be detected. · This list should include all possible combinations of sensor pairs.

[0058] Knowing the width of the filtered area, center that area at the previously defined virtual point (P cal_sensor) When set to this, the area is well - defined. This area can be transformed between sensors using their pre - calibration. (For example, let T be the matrix that performs the transformation between the coordinate systems of the sensors. Also, assume that one of these sensors is a camera. In this case, the K camera matrix transformation that projects 3D points into the image space must also be applied. Using these, the following equation can be written: P ref_sensor =(K*)T*P cal_sensor .

[0059] Regarding roll calibration, the steps used are the same as before, except for the position of the virtual point, and the lateral distance is not zero. Rather, it is a value at which objects on the outer peripheral region of the field of view can be optimally selected. For a forward - facing radar, this value can be selected as Y = + / -x meters, where x is between 3 meters and 12 meters, preferably between 4 meters and 8 meters. These values have been shown to be beneficial as they approximately correspond to the right and left road lanes.

[0060] FIG. 2A is a schematic diagram of vehicle 200 and the fields of view of two sensors (radar 210 and camera 220) of the vehicle 200. Specifically, radar 210 has a field of view 212, camera 220 has a field of view 222, and the field of view 222 partially overlaps the field of view 212 of the radar. The left boundary 234a and the right boundary 234b of the basic filter area 240 are defined as lines parallel to a center line (not shown in FIG. 2B) defined by the longitudinal center of the vehicle. Horizontally, these boundaries 234a, 234b have an equal distance to the center line.

[0061] If one of the paired sensors is significantly different from the direction in which the vehicle is moving and, on the other hand, the other sensor is well aligned with this direction of travel, the execution of this method can rely minimally on pre-calibration. Errors in pre-calibration are recognized as errors in the projected filtered area. This dependence on pre-calibration can be mitigated in several ways. · If the first result is not ideal, repeatedly executing this method will likely lead to a better solution. · By performing filtering several times using differently modified pre-calibrations, it is possible to correct the error. (As the number of times of performing the filtering increases, the execution time also increases, but since the filtering algorithm is lightweight (see the above), the overall process will still remain lightweight.) The selected filtering area can also be corrected based on the steering wheel angle. This can further increase the success rate of pairing because on a curved road, if the filtered area is not corrected during steering, there may be no other vehicles in that filtered area. The filtered area can be corrected as follows. · In the case of 3D detection: Y new =Y old +sin(steering angle)*X*c (Here, c is a pre-set constant that corrects the sensitivity of the steering effect.) · In the case of image space detection: Y new =Y old +sin(steering angle)*X*c*f x / res x (Here, f x is the focal length of the camera, and res x is the width of the camera in pixel units.) FIG. 2B is a schematic diagram of vehicle 200, the field of view region 234 of the sensors 230 of that vehicle, the basic filter area 240, and the shifted filter area 242 based on the steering angle of that vehicle. Specifically, it can be understood that the basic filter area 240 corresponding to the filter area when the vehicle is going straight has boundaries 234a, 234b.

[0062] When the vehicle is steering to the left, the filter region is also shifted to the left, and thus a new filter region 242 is obtained with boundaries 236a, 236b that are shifted to the left from the boundaries 234a, 234b of the basic filter region.

[0063] In one embodiment, the shift of the boundary is proportional to the steering angle, or the sine of the steering angle, or another function of the steering angle. The width of the shifted filter region can be the same as the width of the basic filter region, or in other embodiments, it can also vary based on a function of the steering angle.

[0064] As can be understood in FIG. 2B, shifting the filter region to the left has the advantage that the filter region is shifted in the direction in which the vehicle is steering. Therefore, the leading vehicle 202 is more likely to fit within the filter region. Even if the filtering significantly reduces the number of data points that fit within the overlap of the filter region and the field of view region, the leading vehicle is very likely to fit into this overlap.

[0065] In other words, in one embodiment, the vehicle traveling in front of the vehicle equipped with the sensor is outside this region if the central region seen on the radar frame is not offset by the steering angle. Therefore, using the steering angle to correct the filtered area shortens the data collection without changing the quality of the result.

[0066] As can be appreciated in FIG. 2B, the filter region typically only partially overlaps the field of view region 232 of the sensor.

[0067] Calculating the results To find the external calibration of the sensor in question, calculate the transformation that minimizes the error between paired points. A good option for achieving this is to use optimization, which can iteratively minimize the error between paired points to reach a state of transformation that fairly well represents the actual external calibration.

[0068] The point pairing method described above is fast and robust and can be used in any of the sensor types mentioned. Additionally, thanks to the point pairing method, it is possible to calibrate more than two sensors simultaneously to further enhance the stability of this method. This is also called bundle adjustment. A possible formalization of bundle adjustment for our case is as follows. Suppose there are three sensors (A, B, and C). Using normal optimization, only the error between the point pairs of, for example, sensor pair A - B will be minimized. By using bundle adjustment, it is possible to simultaneously minimize the errors between the point pairs of, for example, sensor pairs A - B, A - C, and B - C, which results in a better overall result.

[0069] Similar to that regarding FIG. 1: In the figure, it is possible to see that the detected objects on the camera frame are marked as small x's and the detected objects on the radar frame are marked as circles. Additionally, in reality, there may be additional points (not shown in FIG. 1) representing the detected objects on the lidar frame. Bundle adjustment simultaneously minimizes the errors between the point pairs of all possible sensor combinations and thus efficiently reduces the overall error.

[0070] During bundle adjustment, it is possible for the error metrics between the pairing of any two sensors to be different. Such a decision is to be made by the user.

[0071] Using bundle adjustment is not essential for implementing this method, and normal optimization, or any other method achieving the same goal, may be suitable for solving this problem. In a preferred embodiment, bundle adjustment is used due to its obvious advantages and its popularity.

[0072] Verification The basis of verification is the calibration process described above. If the calibration of the sensor gives the same (or nearly the same) external parameters as the pre-calibration, the pre-calibration is still considered valid.

[0073] Although it may seem simple at this point, the ordinary comparison of the pre- and resulting calibration external parameters is often not the best solution. Rather, the following may be preferred. For each sensor calibrated against another sensor, use both the newly found pre- and resulting external parameters to calculate the center point of that detection projected into the space of the other sensors. Then, the error between these points and their pairs is calculated for both the pre- and resulting cases. This may already be known during the first and last steps of the optimization. The metric used to calculate the error can be freely used and is to be determined by the user. These errors are compared, for example, by obtaining their ratio and setting it against some threshold.

[0074] Repeating this method in a continuous manner makes it possible to achieve a continuous verification process of the external parameters of the sensor.

[0075] Before verification can be enabled, pre-calibration is obtained using the calibration described above.

[0076] Figures 3A and 3B are further detailed flowcharts of a method for performing initialization 300, collecting data 310, calculating results 320, and validating original parameters, also called prior parameters.

[0077] Specifically, initialization 300 includes the following steps.

[0078] In a first step 302, the original external parameters are initialized. This can include determining the pre-calibration based on other sources, such as information regarding the location of sensors.

[0079] Next, in step 304, one or more sensors to be calibrated are selected and sensor pairings are determined. In step 306, a basic filter area is calculated. This basic filter area can be based on yaw-pitch and roll. Thereafter, in step 308, three proposed filter areas are determined and the subsequent steps of collecting data and calculating results are performed individually for these three proposed filter areas. Determining the proposed filter areas can include determining one or more parameters of the proposed filter areas, although it is understood that additional parameters can be adjusted during data acquisition.

[0080] In one embodiment, the width and centerline of the filter area (which may be shifted later along the steering direction of the vehicle) are determined during these steps.

[0081] In a preferred embodiment, the center line is known on the sensor to be calibrated but unknown on the reference sensor. Therefore, the center line must be transformed into the coordinate system of the reference sensor. Since the exact external parameters are not known, this step will (probably) result in an error, so the next optimization can be performed. For example, multiple transformations with slightly different parameters are applied based on various approaches for determining the transformation parameters or based on the resulting (e.g., random) variations to the parameters. Based on the multiple different transformations, it is possible to determine multiple filter areas, and the filter area that produces the best result can be used at the end of the optimization.

[0082] Data collection 310 can be performed to obtain real-world data when a vehicle (or preferably, a fleet of vehicles) is driving on a road. However, it is understood that the presented method can also be applied in a virtual scenario where a virtual vehicle is driving through a virtual world (and obtaining virtual data from a simulated environment).

[0083] Data collection 310 further includes the following steps.

[0084] First, in step 312a, data for yaw-pitch calibration is collected. In parallel, in step 312b, data for roll calibration is collected.

[0085] In steps 314a, 314b, the filter area is moved based on the current steering angle of the vehicle to which one or more sensors are attached. The filter area of one or more or all sensors can be moved based on the steering angle. In addition to the steering angle, additional information such as the speed of the vehicle can be used to move or otherwise adjust the filter area.

[0086] In steps 316a and 316b, corresponding pairs are identified in the filtered data.

[0087] Result calculation 320 includes the following steps.

[0088] First, for all three sets of collected data, cost functions are evaluated and optimized in steps 322a, 322b, and 322c. Even after extensive optimization, small errors remain, these remaining errors in steps 324a, 324b, 324c. Then, in step 326, those errors can be compared and the best result can be selected.

[0089] The original external parameter 302 and the final result of the calibrated external parameter 330 can then be provided to verification 340.

[0090] Verification 340 includes the following steps.

[0091] The paired points are reprojected in step 342 and the error is calculated in step 344. Finally, a verification metric is applied to evaluate whether the prior parameters are correct. For example, if the error calculated in step 344 is less than a predetermined threshold, the prior parameters can be considered correct. Alternatively, this method can be run for several prior parameters and the parameter that produces the smallest calculated error is determined as the correct parameter.

Claims

1. A method for calibrating the first and second sensors of a vehicle (122, 124, 126), The steps include obtaining first data from the first sensor and second data from the second sensor, The steps include filtering the second data based on the location of the data points in the second data, A method comprising the step of determining one or more calibration parameters (302, 330) between the first and second sensors based on the first data and the filtered second data.

2. The method according to claim 1, further comprising the step of converting the first data and / or the second data into a common domain using pre-calibration.

3. The method according to claim 1, further comprising the steps of detecting a first object in the first data and / or detecting a second object in the filtered second data, and performing the determination of one or more parameters (302, 330) based on the first and / or second objects, preferably the step of determining the one or more parameters (302, 330) is based on the center points of the first and / or second objects.

4. The method according to claim 1, wherein the first sensor and / or the second sensor includes one or more of a conventional camera, a stereo camera, a radar (210), and a lidar.

5. The method according to claim 1, wherein the one or more parameters (302, 330) of the calibration include yaw and pitch parameters (302, 330), and the method further includes the step of filtering the second data based on a filtering region having a region width around a center line extending from the sensor (100, 230) to a point far away, wherein the point far away has lateral and / or vertical coordinate components corresponding to the lateral and / or vertical coordinate components of the position of the sensor (100, 230).

6. The region width is determined based on a predetermined table that assigns a predetermined width to a given pair of first and second sensors, and / or The width of the region is determined based on the position and / or orientation of the sensors (100, 230). The method according to claim 5.

7. The position of the distant point in the coordinates of the reference sensor is [Math 1] [Math 2] [Math 3] [Math 4] It is determined as follows, p ｃａｌ＿ｓｅｎｓｏｒ However, this is the position of the distant point in the coordinates of the calibration sensor, p ｒｅｆ＿ｓｅｎｓｏｒ is the position of the distant point in the coordinate system of the reference sensor, T is a matrix that performs a transformation between the coordinate systems of the calibration sensor and the reference sensor, and K ｃａｌ＿ｓｅｎｓｏｒ and K ｒｅｆ＿ｓｅｎｓｏｒ The method according to claim 5, wherein each of the camera matrix transformations is as described above.

8. The method according to claim 1, wherein the one or more parameters (302, 330) of the calibration include at least one roll parameter, and the method further includes the step of filtering the second data from the sensor (100, 230) based on a filtering region having a fixed width around a centerline toward a distant point that is laterally shifted from the sensor position by a lateral shift in the coordinates of the sensor (100, 230).

9. The method according to claim 1, wherein the step of filtering the second data includes a step of filtering based on a filtering region, the filtering region being adjusted based on the steering direction angle of the vehicle (122, 124, 126), and preferably the step of determining the lateral location of the filtering region includes a step of evaluating the curvature of the steering direction angle, preferably the evaluation step includes a step of multiplying the curvature of the steering direction angle by a sensitivity constant, and preferably further multiplying by a constant proportional to the focal length of a camera sensor.

10. The method according to claim 1, wherein the method is repeatedly performed using a plurality of candidate precalibrations between the first and second sensors, and preferably the method includes an additional step of selecting a preferred precalibration from the plurality of candidate precalibrations.

11. The step of determining one or more parameters (302, 330) of the calibration between the first sensor and the second sensor is, The steps include determining the pairing between a point in the first data and a point in the second data, The process includes the step of determining one or more parameters (302, 330) such that errors between paired points are minimized, Preferably, the step of calibrating the first and second sensors further includes calibrating a third sensor using the first and second sensors, wherein the step of determining one or more parameters of the calibration includes minimizing errors in point pairs between the first and second sensors, point pairs between the second and third sensors, and point pairs between the third and first sensors, and preferably, the step of minimizing the errors in the point pairs includes minimizing the weighted sum of the errors in the point pairs of the sensor pair, according to claim 1.

12. The method according to claim 1, further comprising a step (340) of verifying the determined calibration, wherein the verification step (340) includes comparing one or more parameters (302, 330) of the determined calibration with one or more corresponding parameters (302, 330) of a pre-calibration, and verifying the pre-calibration if the difference between the determined calibration and the pre-calibration is less than a predetermined threshold.

13. The verification step (340) is performed with respect to each of the plurality of sensors (100, 230), including the first and second sensors. The steps include determining a first center point of a detected object projected onto the coordinates of another sensor among the plurality of sensors (100, 230) using the parameters (302, 330) of the calibration determined above, The steps include determining a second center point of the detected object projected onto the coordinates of another sensor among the plurality of sensors using the pre-calibration parameters, The process includes the step of determining the distance between the first and second center points, Preferably, the method according to claim 12, wherein the calibration between the sensor (100, 230) and the other sensor among the plurality of sensors (100, 230) is verified when the distance is smaller than a predetermined threshold.

14. A calibration system for calibrating the first and second sensors of a vehicle (122, 124, 126), wherein the calibration system is An acquisition unit for obtaining first data from the first sensor and second data from the second sensor, A filtering unit for filtering the second data based on the location of the data points of the second data, A calibration system comprising a determination unit for determining one or more calibration parameters (302, 330) between the first and second sensors based on the first data and the filtered second data.

15. A computer-readable storage medium storing program code, wherein the program code includes instructions, and when the instructions are executed by a processor, the method according to one of claims 1 to 13 is performed.

Images