Method and system for correcting the position of at least one feature in the surrounding environment of a vehicle

Abstract

Method and system for correcting the position of at least one feature in the environment surrounding an ego vehicle - Patents.com The present invention relates to a method for correcting the position (P) of at least one feature in the environment surrounding a vehicle using a fusion of data from at least one first surrounding environment detection sensor (1) and a second surrounding environment detection sensor (2), the method comprising the steps of: - a step (S1) of recording the surrounding environment of the host vehicle using a first surrounding environment detection sensor (1) and generating a first surrounding environment representation; - a step (S2) of recording the surrounding environment of the host vehicle using a second surrounding environment detection sensor (2) and generating a second surrounding environment representation; - detecting (S3) at least one feature in the first surrounding environment representation; - determining (S4) a position (P) of at least one feature based on the first surrounding environment representation; - determining (S5) an altitude profile (3,5) in the area in front of the ego-vehicle based on a second representation of the surrounding environment; - a step (S6) of fusing the position (P) of at least one detected feature with the determined altitude profile (3,5) to determine a corrected position; - correcting (S7) the position (P) of the at least one feature based on the determined corrected position.

Description

[Technical field]

[0001] The present invention relates to a method and system for correcting the position of at least one feature in the environment of an ego-vehicle. [Background technology]

[0002] From the prior art, feature extraction (e.g. other traffic participants, lane geometry, drivable areas, etc.) for use in ADAS systems is known, which is carried out for example in camera images. Furthermore, an internal and external calibration of the camera is carried out in order to transform the extracted features in the image into the vehicle coordinate system. There are currently three main approaches:

[0003] There is a flat world assumption, i.e. everything in the camera's field of view is The following They are assumed to be located on a plane with no difference (e.g. a road). Others use stereo cameras, which allow them to derive depth information / height information based on the correlation between the two camera images. The following Thirdly, depth and height can be obtained through techniques such as optical flow. The following It is known that the estimation of Summary of the Invention [Problem to be solved by the invention]

[0004] It is an object of the present invention to provide a method and system whereby increased accuracy in locating features is achieved. [Means for solving the problem]

[0005] This problem is advantageously solved by the independent claims 1 and 6. Further advantageous configurations and embodiments are the subject of the dependent claims.

[0006] First, it was considered that the camera, in principle, has a very high resolution for features located very close to the vehicle, but a very low resolution for more distant features. Therefore, the extraction results are, in principle, immediately inaccurate. The lateral resolution (i.e., the angle of the feature relative to the host vehicle) is indeed good, but the height The following or distance of the feature relative to the vehicle already has a larger error after just a few meters. This problem exists in all three approaches described above and appears prominently in various forms. In particular, it is linked to the errors in the height The following description and the distance description.

[0007] Therefore, according to the present invention, in a method for correcting the position of at least one feature in the surrounding environment of a host vehicle by using the fusion of data from at least one first surrounding environment detection sensor and a second surrounding environment detection sensor, - recording the surrounding environment of the host vehicle using the first surrounding environment detection sensor and generating a first surrounding environment representation; - recording the surrounding environment of the host vehicle using the second surrounding environment detection sensor and generating a second surrounding environment representation; - detecting at least one feature in the first surrounding environment representation; - specifying the position of at least one feature based on the first surrounding environment representation; - determining the height The following process in the region in front of the host vehicle based on the second surrounding environment representation; - fusing the position of at least one detected feature and the determined height The following process to determine a corrected position; - and correcting the position of at least one feature based on the determined corrected position.

[0008] Preferably, the first peripheral environment detection sensor is a camera, and the second peripheral environment detection sensor is a radar sensor or a lidar sensor. Accordingly, the first peripheral environment representation is a camera image, and the second peripheral environment representation is an object list or a point cloud. At least one feature may be, for example, an object, such as a traffic sign, a road boundary line, etc., or another traffic participant, such as a further vehicle, a pedestrian, a cyclist, etc. To detect features in the first peripheral environment representation, for example, semantic segmentation may be performed. The area in front of the vehicle preferably indicates the lane or road surface located in front of the vehicle. Detection of features, determination of position, determination of height, fusion and correction of positions are preferably performed in a corresponding calculation unit. This calculation unit is preferably an ECU or an ADCU (driving assistance and autonomous driving control unit). It is also conceivable to use a sensor calculation unit, for example a camera calculation unit, for the method steps. In this case, the generated peripheral environment representation is transmitted to the calculation unit. The following process The determination of height, fusion and correction of positions are preferably performed in a corresponding calculation unit. This calculation unit is preferably an ECU or an ADCU (driving assistance and autonomous driving control unit). It is also conceivable to use a sensor calculation unit, for example a camera calculation unit, for the method steps. In this case, the generated peripheral environment representation is transmitted to the calculation unit.

[0009] In a preferred embodiment, the position of the feature in the first peripheral environment representation is If the world is flat assumed The flat plane thus obtained to be output in xy coordinates. This If the world is flat assumed The flat plane thus obtained reference plane may be the road surface. Based on this assumption, the distance between this feature and the host vehicle can also be determined. In this case, the coordinates are based on the coordinate system of the first peripheral environment detection sensor that is correspondingly calibrated. Also, the first peripheral environment detection sensor already has a height The following model, and If the world is flat it is also conceivable not to use the The flat plane thus obtained assumption. In this case, the height The following model of the first peripheral environment detection sensor is generally relatively inaccurate. On the other hand, the height The following model based on the data of the second peripheral environment detection sensor is substantially more accurate and more detailed. Also, in such a configuration, the method can be implemented as described above.

[0010] In a further preferred configuration, a geometric step is applied to fuse the position of at least one feature with the height The following process which, in simplified terms, means that the line of sight of the first ambient detection sensor, preferably the camera, intersects the determined height The following process The line of sight can be substantially described as the light ray reaching the first ambient detection sensor from the object. Depending on the height The following process the line of sight may need to be extended if necessary. In this way, two data can be fused and the corrected position can be determined because the height The following is known at this point and the position of the first ambient detection sensor is also known.

[0011] In a preferred configuration, a regression method is applied to determine the height The following process For this purpose, based on the data of the second ambient detection sensor, at least two data points with height The following information are determined.

[0012] Particularly preferably, in one configuration of the method, the least squares method is applied as the regression method. Particularly preferably, it is the linear least squares method. In this case, a linear model or a polynomial model may be used. The least squares method generally represents a mathematical method for adjustment. Here, a function for a set of measurement points is determined, and this function passes through the vicinity of the measurement points as close as possible, thus best summarizing the data. For example, the measurement values of a lidar sensor can be used well, the z coordinate of the point cloud supplied by the lidar can be used as the height The following and the x coordinate can be used as the position, thereby estimating the height The following model showing the height The following process in front of the vehicle. In this case, only points with a not too large y deviation from the x-axis are used.

[0013] According to the present invention, there is further proposed a system for correcting the position of at least one feature in the surrounding environment of a host vehicle, comprising at least one first surrounding environment detection sensor, at least one second surrounding environment detection sensor, and a calculation unit, wherein the calculation unit is configured to execute the method according to any one of claims 1 to 5.

[0014] Here, the calculation unit may be configured as, for example, an ECU or an ADCU mounted on the host vehicle as a separate element. Also, the calculation unit that executes this method may be considered as one component of the sensors.

[0015] A further advantageous configuration is the subject of the drawings.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Modes for Carrying Out the Invention

[0017] Figure 1 shows a method for correcting the position P of at least one feature in the surrounding environment of a host vehicle by using the fusion of data from at least one first surrounding environment detection sensor 1 and a second surrounding environment detection sensor 2. This method comprises the following steps. In step S1, the surrounding environment of the host vehicle is recorded using the first surrounding environment detection sensor 1, and a first surrounding environment representation is generated. In step S2, which may be performed simultaneously with or after step S1, the surrounding environment of the host vehicle is recorded using the second surrounding environment detection sensor 2, and a second surrounding environment representation is generated. In step S3, at least one feature in the first surrounding environment representation is detected. In the next step S4, the position P of at least one feature is identified based on the first surrounding environment representation. In step S5, a height The following process 3,5 in the area in front of the host vehicle is determined based on the second surrounding environment representation. The height The following process 3,5 may relate to, for example, an increase of 3 or a decrease of 5. It is also conceivable that not only continuous decreases of 5 or increases of 3 exist, but that both alternate with each other at regular intervals, thus having small undulations over a certain section. In step S6, the position P of at least one detected feature and the determined height The following process 3,5 are fused to determine a corrected position. Finally, in step S7, the position P of at least one feature is corrected based on the determined corrected position.

[0018] Figure 2 shows a schematic diagram of a scene related to position detection. As shown in Figure 2, the first surrounding environment detection sensor 1, in this case a camera, is used to record the surrounding environment of the host vehicle. In this surrounding environment, in the camera image, a feature, in this figure the position P of a person, is determined. In this case, the reference coordinate system 6 of the camera is If the world is flat assumed The flat plane thus obtained to be calibrated based on 4. This means that If the world is flat the reference plane of the assumed The flat plane thus obtained 4 is the road surface. In this assumption, the camera 1 detects the feature in the detection area 7 of the camera 1 and identifies its position P. However, on a real road The processFor example, it may have an upward movement of 3 or a downward movement of 5. Therefore, when the determined position P needs to be corrected, If the world is flat hypothesis The flat plane thus obtained it is because the position based on 4 is not correctly identified. In this case, the reference sign P1 indicates the corrected position P1 in the case of a potential upward movement of 3, and the reference sign P2 indicates the corrected position P2 in the case of a potential downward movement of 5. Since the The process lanes 3 and 5 are different, the actual or corrected positions P1 and P2 are If the world is flat hypothesis The flat plane thus obtained significantly different in part from the position P identified based on 4.

[0019] FIG. 3 shows a schematic diagram of a system 10 according to an embodiment of the present invention. Here, the system 10 includes a first peripheral environment detection sensor 1 and a second peripheral environment detection sensor 2. Further, a calculation unit 8 is provided, and the method according to the present invention can be executed using the calculation unit 8. In this configuration, the calculation unit 8 is provided in the host vehicle as a separate element, for example, as an ECU / ADCU. In this case, the first peripheral environment detection sensor 1 and the second peripheral environment detection sensor 2 are connected to the calculation unit 8 using a data connection D. It is also conceivable to integrate the calculation unit 8 for executing the present method into one of the sensors 1 and 2. The data connection D may be configured, for example, by wire or wirelessly. Note that this application relates to the invention described in the claims, but also includes the following from other viewpoints. 1. In a method for correcting the position (P) of at least one feature in the surrounding environment of a host vehicle by using the fusion of data from at least one first surrounding environment detection sensor (1) and a second surrounding environment detection sensor (2), - recording the surrounding environment of the host vehicle using the first surrounding environment detection sensor (1) and generating a first surrounding environment representation (step S1); - recording the surrounding environment of the host vehicle using the second surrounding environment detection sensor (2) and generating a second surrounding environment representation (step S2); - detecting at least one feature in the first surrounding environment representation (step S3); - specifying the position (P) of the at least one feature based on the first surrounding environment representation (step S4); - determining the height profile (3, 5) in the region in front of the host vehicle based on the second surrounding environment representation (step S5); - fusing the position (P) of the at least one detected feature and the determined height profile (3, 5) to determine a corrected position (step S6); - correcting the position (P) of the at least one feature based on the determined corrected position (step S7). 2. The method according to 1 above, characterized in that the position (P) of the feature in the first surrounding environment representation is output in xy coordinates on a plane (4) assuming that the world is flat. 3. The method according to 1 above, characterized in that a geometric step is applied to fuse the position (P) of the at least one feature and the height profile (3, 5). 4. The method according to 1 above, characterized in that a regression method is applied to determine the height profile. 5. The method according to 4 above, characterized in that the least squares method is applied as the regression method. 6. In a system (10) for correcting the position (P) of at least one feature in the surrounding environment of a host vehicle, comprising at least one first surrounding environment detection sensor (1) and a second surrounding environment detection sensor (2) and a calculation unit (8), the system (10), wherein the calculation unit (8) is configured to execute the method according to any one of 1 to 4 above.

Explanation of Symbols

[0020] 1 First peripheral environment detection sensor 2 Second peripheral environment detection sensor 3 Upward lane of the course 4 assuming the world is flat Hypothesis performed Plane 5 Downward lane of the course 6 Camera coordinate system 7 Detection area 8 Calculation unit 10 System D Data connection P Position of at least one feature P1, P2 Corrected positions S1 - S7 Method steps

Claims

1. - A step (S1) of recording the surrounding environment of the host vehicle using a first surrounding environment detection sensor (1) and generating a first surrounding environment representation; - A step (S2) of recording the surrounding environment of the host vehicle using a second surrounding environment detection sensor (2) and generating a second surrounding environment representation; - A step (S3) of detecting at least one feature in the first surrounding environment representation; - A step (S4) of specifying the position (P) of the at least one feature in the xy coordinates of a plane (4) assuming that the world is a plane based on the first surrounding environment representation; - A step (S5) of determining the height profile (3, 5) in the region in front of the host vehicle based on the second surrounding environment representation; - A step (S6) of obtaining an intersection point between the line of sight of the first surrounding environment detection sensor (1) and the determined height profile (3, 5), and determining the intersection point as a corrected position (P1, P2) of the position (P) of the at least one detected feature; - A method for correcting the position (P) of at least one feature in the surrounding environment of a host vehicle, comprising a step (S7) of using the determined corrected position (P1, P2) as the position of the at least one feature.

2. The method according to claim 1, characterized in that the least squares method is applied to determine the height profile.

3. In a system (10) for correcting the position (P) of at least one feature in the surrounding environment of a host vehicle, comprising at least one first surrounding environment detection sensor (1) and a second surrounding environment detection sensor (2) and a calculation unit (8), the calculation unit (8) is configured to execute the method according to claim 1 or 2. System (10).

Images