Method and system for correcting the position of at least one feature in the environment surrounding an ego vehicle - Patents.com
The method combines camera and radar/lidar data to correct feature positions in ADAS systems, addressing inaccuracies in height and distance estimation, thereby enhancing localization accuracy.
Patent Information
- Application Number
- JP2023563196
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-06-01
- Filing Date
- 2022-05-20
- Publication Date
- 2024-11-27
- Estimated Expiration
- 2042-05-20
AI Technical Summary
Existing methods for feature extraction in ADAS systems suffer from inaccuracies in determining the height and distance of features relative to the vehicle, particularly for distant objects, leading to errors in position correction.
A method involving the fusion of data from a camera and a radar or lidar sensor to generate accurate position corrections by combining camera images with point clouds, using geometric and regression methods to align and correct feature positions based on a flat world assumption.
Enhances the accuracy of feature localization by integrating height information from radar or lidar sensors, reducing errors in position determination and improving the precision of feature detection in the vehicle's surroundings.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a method and system for correcting the position of at least one feature in the environment of an ego-vehicle. [Background technology]
[0002] From the prior art, feature extraction (e.g. other traffic participants, lane geometry, drivable areas, etc.) for use in ADAS systems is known, which is carried out for example in camera images. Furthermore, an internal and external calibration of the camera is carried out in order to transform the extracted features in the image into the vehicle coordinate system. There are currently three main approaches:
[0003] There is a flat world assumption, i.e. everything in the camera's field of view is Sano They are assumed to be located on a plane with no difference (e.g. a road). Others use stereo cameras, which allow them to derive depth information / height information based on the correlation between the two camera images. difference Thirdly, depth and height can be obtained through techniques such as optical flow. difference It is known that the estimation of Summary of the Invention [Problem to be solved by the invention]
[0004] It is an object of the present invention to provide a method and system whereby increased accuracy in locating features is achieved. [Means for solving the problem]
[0005] This problem is advantageously solved by the independent claims 1 and 6. Further advantageous configurations and embodiments are the subject of the dependent claims.
[0006] First, it was considered that a camera in principle has a very high resolution for features located very close to the vehicle, but a very low resolution for features that are further away. Thus, the extraction results are in principle immediately inaccurate. The lateral resolution (i.e. the angle of the feature relative to the ego-vehicle) is indeed good, but the high resolution of the feature relative to the vehicle is very poor. difference Or the distance has a larger error already after a few meters. In all three approaches mentioned above, this problem exists and is manifested in different ways. Especially in high difference It is linked to errors in description and distance description.
[0007] Therefore, according to the invention, there is provided a method for correcting the position of at least one feature in the environment of an ego-vehicle using fusion of data of at least one first and a second environment detection sensor, the method comprising the steps of: - recording the surrounding environment of the host vehicle using a first surrounding environment detection sensor and generating a first surrounding environment representation; - recording the surrounding environment of the host vehicle using a second surrounding environment detection sensor and generating a second surrounding environment representation; - detecting at least one feature in the first surrounding environment representation; - determining a location of at least one feature based on the first surrounding environment representation; - Detecting a high-altitude area in the area in front of the host vehicle based on the second surrounding environment representation The progress of determining - The location of at least one detected feature and the determined height The progress of and determining a corrected position; - correcting the position of the at least one feature based on the determined corrected position.
[0008] Preferably, the first surrounding environment detection sensor is a camera and the second surrounding environment detection sensor is a radar or lidar sensor. Thus, the first surrounding environment representation is a camera image and the second surrounding environment representation is an object list or a point cloud. The at least one feature may for example be an object, e.g. a traffic sign, a road boundary, etc., or another traffic participant, e.g. a further vehicle, a pedestrian, a cyclist, etc. To detect the feature in the first surrounding environment representation, for example a semantic segmentation may be performed. The area in front of the vehicle preferably indicates the roadway or road surface located in front of the vehicle. The detection, localization, high resolution of the feature may be performed. The progress of The determination, fusion and correction of the position are preferably performed in a corresponding computing unit, which is preferably an ECU or an ADCU (Driver Assistance and Autonomous Driving Control Unit). It is also conceivable to use a sensor computing unit, for example a computing unit of a camera, for the method steps. In this case, the generated environment representation is transmitted to the computing unit.
[0009] In a preferred embodiment, the location of the feature in the first surroundings representation is determined by: If the world is flat Assumptions Plane This is output in xy coordinates. If the world is flat Assumptions Plane The reference plane may be the road surface. Based on this assumption, the distance between this feature and the vehicle can also be determined. In this case, the coordinates are based on the correspondingly calibrated coordinate system of the first surrounding environment detection sensor. Also, the first surrounding environment detection sensor is already calibrated. difference It has a model, If the world is flat Assumptions Plane In this case, the high sensitivity of the first surrounding environment detection sensor is not required. difference The model is generally relatively inaccurate. On the other hand, a highly accurate model based on data from the second surrounding environment detection sensor is used. difference The model is substantially more accurate and detailed, and in such a configuration the method can still be carried out as described above.
[0010] In a further preferred configuration, the position and height of at least one feature are The progress of A geometric step is applied to fuse the two. In simple terms, this is done by: The progress of The line of sight can be substantially described as a ray of light reaching the first surrounding environment detection sensor from the object. The progress of Depending on the situation, the line of sight must be extended if necessary. In this way, the two data can be fused and the corrected position can be determined. difference is known, and the position of the first surrounding environment detection sensor is also known.
[0011] In a preferred configuration, The passage of time To this end, a regression method is applied to determine the high difference At least two informative data points are determined.
[0012] Particularly preferably, in one configuration of the method, the least squares method is applied as the regression method. Particularly preferably, the linear least squares method is used. In this case, a linear or polynomial model may be used. The least squares method generally refers to a mathematical method for adjustment. Here, a function is determined for a set of measurement points, which passes through the closest possible vicinity of the measurement points, and thus best sums up the data. For example, measurements of a lidar sensor may be used, and the z-coordinate of the point cloud provided by the lidar is then scaled. difference , x coordinates may be used as positions, which may give the height in front of the vehicle. difference Indicating high The progress of Estimate the model, using only those points whose y deviation from the x-axis is not too large.
[0013] According to the present invention there is further proposed a system for correcting the position of at least one feature in the surrounding environment of a vehicle, comprising at least one first surrounding environment detection sensor and a second surrounding environment detection sensor and a computing unit, wherein the computing unit is configured to perform the method according to any one of claims 1 to 5.
[0014] Here, the computing unit may be configured, for example, as an ECU or ADCU installed in the host vehicle as a separate element, but it is also conceivable that the computing unit performing the method is a component of one of the sensors.
[0015] Further advantageous configurations are the subject of the drawings. [Brief description of the drawings]
[0016] [Figure 1] FIG. 1 is a schematic flow diagram of a method according to one embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram of a scene related to location detection. [Diagram 3] FIG. 3 is a schematic diagram of a system according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0017] FIG. 1 shows a method for correcting the position P of at least one feature in the surrounding environment of the host vehicle using fusion of data from at least one first surrounding environment detection sensor 1 and at least one second surrounding environment detection sensor 2, the method comprising the following steps: In step S1, the surrounding environment of the host vehicle is recorded using the first surrounding environment detection sensor 1 to generate a first surrounding environment representation. In step S2, which may be performed simultaneously with or after step S1, the surrounding environment of the host vehicle is recorded using the second surrounding environment detection sensor 2 to generate a second surrounding environment representation. In step S3, at least one feature in the first surrounding environment representation is detected. In the following step S4, the position P of the at least one feature is determined based on the first surrounding environment representation. In step S5, a high-altitude area in the area in front of the host vehicle is determined based on the second surrounding environment representation. The progress of 3,5 is decided. The progress of 3, 5 may refer to, for example, an ascent 3 or a descent 5. It is also conceivable that there is not only a continuous descent 5 or an ascent 3, but that they alternate with each other at regular intervals, thus resulting in small undulations over a certain section. In step S6, the position P of at least one detected feature and the determined height are calculated. The progress of 3 and 5 are fused to determine the corrected position. Finally, in step S7, the position P of at least one feature is corrected based on the determined corrected position.
[0018] FIG. 2 shows a schematic diagram of a scene for position detection. As shown in FIG. 2, a first surrounding environment detection sensor 1, in this case a camera, is used to record the surrounding environment of the vehicle. In this surrounding environment, in the camera image, a feature is determined, in this figure the position P of a person. In this case, the reference coordinate system 6 of the camera is If the world is flat Assumptions Plane 4. This is If the world is flat Assumptions Plane This means that the reference plane of 4 is the road surface. In this assumption, the camera 1 detects a feature in the detection area 7 of the camera 1 and identifies its position P. However, in the case of a real road Progressmay have, for example, an ascent of 3 or a descent of 5, and therefore a correction of the determined position P is required only if If the world is flat Assumptions Plane 4 is not correctly determined. In this case, reference P1 denotes the corrected position P1 in the case of a potential ascent 3, and reference P2 denotes the corrected position P2 in the case of a potential descent 5. Progress Since 3 and 5 are different, the actual or corrected positions P1 and P2 are If the world is flat Assumptions Plane 4.
[0019] FIG. 3 shows a schematic diagram of a system 10 according to an embodiment of the present invention. Here, the system 10 comprises a first surrounding environment detection sensor 1 and a second surrounding environment detection sensor 2. Furthermore, a calculation unit 8 is provided, by means of which the method according to the present invention can be executed. In this configuration, the calculation unit 8 is provided in the vehicle as a separate element, for example as an ECU / ADCU. In this case, the first surrounding environment detection sensor 1 and the second surrounding environment detection sensor 2 are connected to the calculation unit 8 by means of a data connection D. It is also conceivable to integrate the calculation unit 8 for executing the method in one of the sensors 1, 2. The data connection D may be configured, for example, as a wired or wireless connection. This application relates to the invention described in the claims, but also includes the following as other aspects. 1. A method for correcting a position (P) of at least one feature in a surrounding environment of a vehicle using fusion of data from at least one first surrounding environment detection sensor (1) and a second surrounding environment detection sensor (2), comprising: - a step (S1) of recording the surrounding environment of the host vehicle using the first surrounding environment detection sensor (1) and generating a first surrounding environment representation; - recording the surrounding environment of the host vehicle using the second surrounding environment detection sensor (2) and generating a second surrounding environment representation (S2); - detecting (S3) at least one feature in said first ambient environment representation; and - determining (S4) the position (P) of said at least one feature based on said first ambient environment representation. - determining (S5) a height progression (3,5) in the area in front of the ego-vehicle based on the second representation of the surrounding environment; - a step (S6) of fusing the position (P) of the at least one detected feature with the determined height profile (3,5) to determine a corrected position, - correcting (S7) the position (P) of the at least one feature based on the determined corrected position. 2. 2. The method according to claim 1, characterized in that the position (P) of the feature in the first surrounding environment representation is output in xy coordinates in a plane (4) where the world is assumed to be flat. 3. 2. The method according to claim 1, characterized in that a geometric step is applied to blend the position (P) and the height course (3,5) of the at least one feature. 4. 2. The method according to claim 1, characterized in that a regression method is applied to determine the height course. 5. 5. The method according to claim 4, characterized in that the least squares method is applied as the regression method. 6. A system (10) for correcting a position (P) of at least one feature in the surrounding environment of a vehicle, comprising at least one first surrounding environment detection sensor (1) and a second surrounding environment detection sensor (2) and a calculation unit (8), wherein the calculation unit (8) is configured to execute any one of the methods described in 1 to 4 above. [Explanation of symbols]
[0020] 1. First surrounding environment detection sensor 2 Second surrounding environment detection sensor 3. Ascending roadway Progress 4 If the world is flat Assumptions did plane 5. Descending road Progress 6 Camera Coordinate System 7 Detection Area 8 Computing Units 10. System D Data Connection P is the position of at least one feature P1,P2 corrected position S1-S7 Method Steps
Claims
1. - a step (S1) of recording the surroundings of the host vehicle using the first surroundings detection sensor (1) and generating a first surroundings representation; - recording the surroundings of the vehicle using the second surroundings detection sensor (2) and generating a second surroundings representation (S2); - detecting (S3) at least one feature in said first ambient environment representation; - determining (S4) a position (P) of said at least one feature in x and y coordinates in a plane (4) on the assumption that the world is flat, based on said first environmental representation; - determining (S5) a height progression (3,5) in the area in front of the ego-vehicle on the basis of said second representation of the surrounding environment; - determining (S6) the intersection point between the line of sight of the first surroundings detection sensor (1) and the determined height course (3,5) and determining said intersection point as a corrected position (P1, P2) of the position (P) of the at least one detected feature; - using (S7) said determined corrected position (P1, P2) as the position of said at least one feature.
2. 2. The method according to claim 1, characterized in that for determining the height course a least squares method is applied.
3. A system (10) for correcting a position (P) of at least one feature in the surrounding environment of a vehicle, comprising at least one first surrounding environment detection sensor (1) and a second surrounding environment detection sensor (2) and a calculation unit (8), wherein the calculation unit (8) is configured to perform the method according to claim 1 or 2.
