A method for lidar sensor correlation

Abstract

The present invention discloses a method for establishing a correlation between LiDAR sensors, so that a perception model created by means of a first LiDAR sensor is used with a second LiDAR sensor having a different interpretation of point intensity than the first LiDAR sensor. Said method comprises the steps of: placing the first LiDAR sensor at a LiDAR sensor position (1) to perform a sensing operation by means of the first LiDAR sensor (101); placing the second LiDAR sensor at the LiDAR sensor position (1) to perform a sensing operation by means of the second LiDAR sensor (102); comparing the point intensity values of a first point cloud and a second point cloud at obstacle points (104), wherein the first point cloud is obtained from a sensing operation performed by the first LiDAR sensor and the second point cloud is obtained from a sensing operation performed by the second LiDAR sensor; and deriving a point intensity correlation function between the first and second LiDAR sensors based on the comparison of the point intensity values (105).

Description

[0001] 8343.1168

[0002] DESCRIPTION

[0003] A METHOD FOR LIDAR SENSOR CORRELATION

[0004] Field of the Invention

[0005] The present invention relates to a method for establishing a correlation between LiDAR sensors, so that a perception model created by means of a LiDAR sensor is used with another LiDAR sensor.

[0006] Background of the Invention

[0007] In autonomous vehicles and vehicles equipped with driver assistance systems, reliable vehicle operation requires detecting obstacles around the vehicle, the distance between such obstacles and the vehicle, and the object classification of those obstacles. In conventional applications, distance sensors are employed for such detection tasks. However, in particular for autonomous vehicles, data obtained from conventional distance sensors is not sufficient. Therefore, in the known art, sensing systems with higher detection speed and higher detection accuracy are used.

[0008] Said sensing systems of the known art utilize LiDAR sensors. A LiDAR (Light Detection and Ranging) sensor is capable of using light to determine a position of an obstacle within a scanning area with high sensitivity and accuracy.

[0009] If a LiDAR sensor performs a sensing operation, a point cloud is obtained, which contains the surface points of obstacles in the environment together with their positional information. In order to use the points in the point cloud by, for example, an autonomous vehicle, the vehicle must analyze the meaning of those points. For this analysis, machine-learning-based perception models are employed. The creation of such perception models requires a relatively long training process.

[0010] In LiDAR sensors, the point intensity (the point intensity value resulting from reflectivity and distance) of the point cloud may vary depending on the characteristics of the LiDAR sensor. With advancing technology, LiDAR sensors are produced that focus on different

[0011] 1

[0012] Confidential8343.1168 purposes, operate at higher speeds, provide distance measurements with greater accuracy, and deliver additional information alongside point positions. Accordingly, a need may arise to switch to different LiDAR sensors. However, a perception model created by means a first LiDAR sensor in an autonomous vehicle cannot be used with a second LiDAR sensor that employs a different approach to point intensity computation. Consequently, when a LiDAR sensor is replaced, a new perception model must also be generated for this new LiDAR sensor. Since generating such a perception model is a timeconsuming process and requires new data labeling, the attempt to enhance detection accuracy through LiDAR sensor replacement becomes costly and difficult for vehicle manufacturers.

[0013] Brief Description of the Invention

[0014] The present invention discloses a method for establishing a correlation between LiDAR sensors, so that a perception model created by means of a first LiDAR sensor is used with a second LiDAR sensor having a different interpretation of point intensity than the first LiDAR sensor. Said method comprises the steps of: placing the first LiDAR sensor at a LiDAR sensor position to perform a sensing operation by means of the first LiDAR sensor; placing the second LiDAR sensor at the LiDAR sensor position to perform a sensing operation by means of the second LiDAR sensor; comparing the point intensity values of a first point cloud and a second point cloud at obstacle points, wherein the first point cloud is obtained from a sensing operation performed by the first LiDAR sensor and the second point cloud is obtained from a sensing operation performed by the second LiDAR sensor; and deriving a point intensity correlation function between the first and second LiDAR sensors based on the comparison of the point intensity values.

[0015] The method according to the present invention compares the point intensity values of a first LiDAR sensor and a second LiDAR sensor, thereby creating a function describing the correlation of point intensity values between said LiDAR sensors. By means of this function, a perception model previously generated for the first LiDAR sensor can be used with the second LiDAR sensor. Preferably, the data of the first LiDAR sensor is modified with the aid of the function, such that a new perception model is trained to derive a model suitable for the second LiDAR sensor.

[0016] 2

[0017] Confidential8343.1168 Object of the Invention

[0018] An object of the present invention is to provide a method for establishing a correlation between LiDAR sensors, so that a perception model created by means of a LiDAR sensor is used with another LiDAR sensor.

[0019] Another object of the present invention is to provide a practical and reliable method.

[0020] Brief Description of the Drawings

[0021] Exemplary embodiments of the method according to the present invention are illustrated in the attached drawings, in which:

[0022] Figure 1 is a flow diagram of the method according to the invention.

[0023] Figure 2 is a top view illustrating the positions of a LiDAR sensor and obstacles, according to an embodiment of the method of the invention.

[0024] All the parts illustrated in figures are individually assigned a reference numeral and the corresponding terms of these numbers are listed below:

[0025] LiDAR sensor position (1) Obstacle position (2) Performing a sensing operation by means of the first LiDAR sensor (101) Performing a sensing operation by means of the second LiDAR sensor (102) Comparing, with a threshold value, the difference in distances

[0026] to obstacles when both LiDAR sensors are placed at the same point (103) Comparing point intensity values (104) Deriving the point intensity correlation function (105)

[0027] Detailed Description of the Invention

[0028] For autonomous vehicles and vehicles equipped with driver assistance systems to move safely, obstacles around the vehicle must be detected. Accordingly, in such vehicles, LiDAR sensors are used to perform the detection process. If a sensing operation is

[0029] 3

[0030] Confidential8343.1168 performed by a LiDAR sensor, a point cloud is obtained, wherein a perception model is employed to analyze the obtained point cloud to detect obstacles. Said perception model is obtained by using the point clouds detected by a LiDAR sensor in a machine learning application. However, when the point intensity values of the point clouds generated by different LiDAR sensors differ from each other, a perception model created for one LiDAR sensor exhibits reduced performance with a different LiDAR sensor. Therefore, the present invention provides a method for establishing a correlation between LiDAR sensors, so that a perception model created by means of a LiDAR sensor is used with another LiDAR sensor.

[0031] The method according to the present invention, as illustrated in figures 1-2, enables a correlation to be established between LiDAR sensors, so that a perception model created by means of a first LiDAR sensor is used with a second LiDAR sensor having a different interpretation of point intensity than the first LiDAR sensor. Said method comprises the steps of: placing the first LiDAR sensor at a LiDAR sensor position (1) to perform a sensing operation by means of the first LiDAR sensor (101); placing the second LiDAR sensor at the LiDAR sensor position (1) to perform a sensing operation by means of the second LiDAR sensor (102); comparing the point intensity values of a first point cloud and a second point cloud at obstacle points (104), wherein the first point cloud is obtained from a sensing operation performed by the first LiDAR sensor and the second point cloud is obtained from a sensing operation performed by the second LiDAR sensor; and deriving a point intensity correlation function between the first and second LiDAR sensors based on the comparison of the point intensity values (105).

[0032] In an embodiment of the invention, a first LiDAR sensor is used in an autonomous vehicle or in a vehicle with a driver assistance system (for example, a motor vehicle) to perform environmental sensing. For this sensing operation, a perception model is created for the first LiDAR sensor using a machine learning application. When it is desired to use in the autonomous vehicle a second LiDAR sensor that has a different interpretation of point intensity (for example, evaluating Lambertian surfaces on a scale of 0-120 instead of 0-100), an already-created perception model performs with reduced accuracy when applied directly to the second LiDAR sensor. Therefore, the present invention provides a method that determines the point intensity correlation between the sensors, so that a perception model created for the first LiDAR sensor is used with high performance for the second

[0033] 4

[0034] Confidential8343.1168 LiDAR sensor. In this method, the first LiDAR sensor and the second LiDAR sensor are placed sequentially (regardless of the order) at a LiDAR sensor position (1), such that sensing operations are performed from this position (1) with both LiDAR sensors, thereby collecting data. As a result of this data collection, a first point cloud and a second point cloud are obtained. By comparing the point intensity values at obstacle positions in the obtained point clouds, a point intensity correlation function defining the relationship between the sensors is derived. The point intensity correlation function thus obtained allows a point cloud acquired by the second LiDAR sensor, in an autonomous vehicle or a vehicle with a driver assistance system, to be utilized in said perception model. In this way, without the need for new data collection and labeling for the second LiDAR sensor having a different interpretation of point intensity, point cloud data collected with the first LiDAR sensor is suitably modified by the derived function before training a new environmental model for the second LiDAR sensor, or, if the environmental sensing model trained with data from the first LiDAR sensor is to be used, the point cloud data acquired by the second LiDAR sensor is suitably modified before being input to the environmental model. Here, the point intensity correlation function is a mathematical function. This mathematical function may be of different orders, have defined coefficients, or be created in a learning-based manner. The function may be applied in a varying manner to points falling within different distance intervals of the point set. Furthermore, the function may vary depending on the material properties of the objects corresponding to the points. The function described herein is used to generate a predicted point cloud corresponding to the first LiDAR sensor, based on the point cloud detected by the second LiDAR sensor. The predicted point cloud corresponding to the first LiDAR sensor carries information such as position, velocity, acceleration, intensity, and reflectivity available in the first LiDAR sensor. Thus, the predicted point cloud created using the function can be directly utilized in the environmental sensing model with high efficiency.

[0035] In a preferred embodiment of the invention, prior to the steps of placing the first LiDAR sensor at a LiDAR sensor position (1) to perform a sensing operation by means of the first LiDAR sensor (101) and placing the second LiDAR sensor at the LiDAR sensor position (1) to perform a sensing operation by means of the second LiDAR sensor (102), the method comprises the step of placing at least one obstacle in at least one obstacle position (2) around the LiDAR sensor position (1). Preferably, obstacles are placed in at least two different obstacle positions (2). Different obstacles may be placed at different

[0036] 5

[0037] Confidential8343.1168 obstacle positions (2), or a single obstacle may be placed at different obstacle positions (2), with the sensing steps (101, 102) repeated for different positions. Preferably, obstacles are placed at different distances and angles around the LiDAR sensor position (1) to collect as much data as possible. In these embodiments, the obstacle preferably has a surface structure with a constant reflectivity percentage (Lambertian surface). In this way, the surface points of the obstacle can be detected more precisely by the LiDAR sensors operating with light, together with a correlatable intensity value. Additionally, in the step of comparing the point intensity values (104), the point intensity values of the point clouds corresponding to an obstacle at a given position are compared between the two LiDAR sensors. Thus, by repeating this operation for different positions, the comparison process can be carried out more effectively.

[0038] In the method according to the present invention, positions of the first and second LiDAR sensors are required to be identical in the steps of sensing (101, 102), in order to enable an accurate comparison of the point intensity values of the LiDAR sensors. If the LiDAR sensors collect (i.e., sense) the point cloud data from different positions, comparison of point intensity values becomes more challenging. It is also preferable to use the same position to determine the point intensity correlation function more accurately. Therefore, in a preferred embodiment of the invention, after the steps of placing the first LiDAR sensor at a LiDAR sensor position (1) to perform a sensing operation by means of the first LiDAR sensor (101) and placing the second LiDAR sensor at the LiDAR sensor position (1) to perform a sensing operation by means of the second LiDAR sensor (102), the method comprises the steps of: determining the sensing positions of the first and second LiDAR sensors; comparing, with a threshold value, the difference in distances to obstacles when both LiDAR sensors are placed at the same point (103); if the difference is less than the threshold value, proceeding to the step of comparing the point intensity values (104); if the difference between said positions is not less than the threshold value, returning to the steps of performing a sensing operation by means of the first LiDAR sensor (101) and performing a sensing operation by means of the second LiDAR sensor (102). During determination of the sensing positions of the first and second LiDAR sensors, the distances to different obstacles are measured by the LiDAR sensors themselves or by external sensors, such as rangefinders. Thus, positions of the LiDAR sensors and the obstacles, together with the obtained point cloud data, are precisely determined and compared with each other. In addition, prior to collecting point cloud data, it is proposed to

[0039] 6

[0040] Confidential8343.1168 measure the ambient light at the obstacle positions using an illuminance meter, in order to reduce noise and enable a more accurate definition of the correlation function.

[0041] The method according to the present invention compares the point intensity values of a first LiDAR sensor and a second LiDAR sensor, thereby creating a function describing the correlation of point intensity values between said LiDAR sensors. By means of this function, a perception model previously generated for the first LiDAR sensor can be used with the second LiDAR sensor. Preferably, the data of the first LiDAR sensor is modified with the aid of the function, such that a new perception model is trained to derive a model suitable for the second LiDAR sensor.

[0042] Confidential

Claims

8343.1168CLAIMS1. A method for establishing a correlation between LiDAR sensors, so that a perception model created by means of a first LiDAR sensor is used with a second LiDAR sensor having a different interpretation of point intensity than the first LiDAR sensor, the method characterized by comprising the steps of:placing the first LiDAR sensor at a LiDAR sensor position (1) to perform a sensing operation by means of the first LiDAR sensor (101);placing the second LiDAR sensor at the LiDAR sensor position (1) to perform a sensing operation by means of the second LiDAR sensor (102); comparing the point intensity values of a first point cloud and a second point cloud at obstacle points (104), wherein the first point cloud is obtained from a sensing operation performed by the first LiDAR sensor and the second point cloud is obtained from a sensing operation performed by the second LiDAR sensor; andderiving a point intensity correlation function between the first and second LiDAR sensors based on the comparison of the point intensity values (105).

2. A method according to claim 1, characterized in that prior to the steps of placing the first LiDAR sensor at a LiDAR sensor position (1) to perform a sensing operation by means of the first LiDAR sensor (101) and placing the second LiDAR sensor at the LiDAR sensor position (1) to perform a sensing operation by means of the second LiDAR sensor (102), the method comprises the step of placing at least one obstacle in at least one obstacle position (2) around the LiDAR sensor position (1).

3. A method according to claim 2, characterized in that the step of placing an obstacle comprises placing obstacles in at least two different obstacle positions (2).

4. A method according to claim 3, characterized in that the step of placing an obstacle comprises placing different obstacles at different obstacle positions (2).8Confidential8343.1168 5. A method according to claim 3, characterized in that the step of placing an obstacle comprises placing a single obstacle at different obstacle positions (2).

6. A method according to any of the claims 2 to 5, characterized in that the obstacle placed at the obstacle position (2) has a surface structure with a constant reflectivity percentage.

7. A method according to any of the preceding claims, characterized in that after the steps of placing the first LiDAR sensor at a LiDAR sensor position (1) to perform a sensing operation by means of the first LiDAR sensor (101) and placing the second LiDAR sensor at the LiDAR sensor position (1) to perform a sensing operation by means of the second LiDAR sensor (102), the method comprises the steps of: determining the sensing positions of the first and second LiDAR sensors; comparing, with a threshold value, the difference in distances to obstacles when both LiDAR sensors are placed at the same point (103); if the difference is less than the threshold value, proceeding to the step of comparing intensity values (104); if the difference between said positions is not less than the threshold value, returning to the steps of performing a sensing operation by means of the first LiDAR sensor (101) and performing a sensing operation by means of the second LiDAR sensor (102).Confidential

Images