Method for operating an assistance system of a vehicle and assistance system of a vehicle
By classifying static and movable objects in the vehicle's surrounding environment map and using static objects for map registration, the problem of errors in the surrounding environment map is solved, achieving high-precision vehicle positioning and map registration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ROBERT BOSCH GMBH
- Filing Date
- 2021-08-11
- Publication Date
- 2026-06-05
AI Technical Summary
Existing vehicle surrounding environment maps suffer from errors in environmental data and motion information, leading to discrepancies between the illustrations and reality, making it difficult to provide accurate and reliable information.
By sensing the vehicle's surrounding environment, at least two surrounding environment maps are generated, divided into cells and assigned values. Objects are classified as static and moving objects. Static objects are used for map registration, while moving objects are ignored, and static objects are given priority for localization.
It achieves highly accurate and reliable surrounding environment maps, improves the accuracy and reliability of vehicle positioning, simplifies the map registration process, and reduces resource consumption.
Smart Images
Figure CN114074665B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for operating an auxiliary system for a vehicle, and to an auxiliary system for a vehicle. Background Technology
[0002] A known system assists in generating a vehicle's surrounding environment map, known as an "occupancy grid map." This type of map maps objects within the vehicle's surrounding environment. Typically, the surrounding environment map is generated using environmental data sensed by the vehicle's environmental sensors. However, due to errors in the environmental data and motion information, discrepancies can arise between the depiction in the surrounding environment map and reality. Summary of the Invention
[0003] In contrast, the objective of the method of the present invention is to provide a map of the surrounding environment with particularly accurate and reliable information. This is achieved through a method for operating an assistance system for a vehicle, the method comprising the following steps:
[0004] - Sensing, especially using the vehicle's environmental sensors, to sense the vehicle's surroundings.
[0005] - Generate at least two surrounding environment maps at different time points, wherein each of the at least two surrounding environment maps has an image of the surrounding environment, wherein each surrounding environment map is divided into multiple cells, and wherein a predefined value is assigned to each cell based on the presence of an object at the corresponding cell location in the surrounding environment.
[0006] - Classify objects sensed in the vehicle's surrounding environment.
[0007] - Identifying static objects using the aforementioned classification, and
[0008] - Register at least two surrounding environment maps with each other.
[0009] Here, the registration is performed based on the corresponding cells representing static objects in the surrounding environment map.
[0010] In other words, this method generates two surrounding environment maps, which can also be referred to as "occupancy grid maps," based on the sensed surrounding environment. Each surrounding environment map is divided into multiple cells, each of which is assigned a predetermined value. Here, the cell value is assigned based on whether an object exists at a location corresponding to the location of a specific cell in the vehicle's surrounding environment. In particular, this enables the vehicle to be located, preferably relative to an object. For example, a numerical value can be assigned as a cell value, which preferably either indicates the presence or absence of an object.
[0011] Objects sensed in the vehicle's surrounding environment are classified. Here, identifying and categorizing objects into different classes is considered classification. Specifically, objects are classified as static and movable (or non-static) objects. Next, objects classified as static are identified from the classified objects. Then, areas of the two surrounding environment maps are used for registration between the two maps where static objects have been identified at their respective environmental locations. Preferably, all other areas of the surrounding environment maps are ignored. Objects whose locations in the surrounding environment do not change over time are considered static objects, such as buildings, walls, trees, or other structural objects. This means that registering the surrounding environment maps based on immovable objects that always have the same location in the surrounding environment makes it possible, for example, to determine the relative positions of these objects with particular reliability.
[0012] Therefore, this method offers the advantage of using immovable areas of the surrounding environment for registration of the surrounding environment map. For example, even when the time interval between driving through the surrounding environment in a specific area is large, it is possible to reliably and easily register and verify a previously generated first surrounding environment map of the same area. This can be achieved particularly simply and reliably because, for example, static objects do not change their location in the surrounding environment. Therefore, it is possible to provide a surrounding environment map with particularly high accuracy, thereby allowing, for example, the highest possible positioning accuracy for vehicles in the surrounding environment map.
[0013] The preferred embodiments described herein are preferred extensions of the present invention.
[0014] Preferably, the method further includes the step of locating the vehicle in at least one of the surrounding environment maps based on the mutual registration of at least two surrounding environment maps. Preferably, for this purpose, a first sensing of the first surrounding environment map is performed as a so-called "training" process, for example, by means of the vehicle's manual driving controlled by the driver. Here, determining the vehicle's position and orientation within the surrounding environment map is considered localization. For example, objects are plotted into the first surrounding environment map using vehicle ranging and environmental sensing devices. Later, a second surrounding environment map can be generated by means of a second sensing through a second driving in the same area of the surrounding environment and registered with the first surrounding environment map. Therefore, based on this registration, the position of the vehicle, especially relative to objects or similar objects plotted in the surrounding environment map, can be estimated. Since the two surrounding environment maps are mutually registered based on static objects, particularly accurate and reliable vehicle localization can be achieved.
[0015] Preferably, the method further includes the step of prioritizing cells representing static objects in the surrounding environment. Here, the prioritized cells are used for cross-registration of at least two surrounding environment maps. For example, prioritizing cells during the analysis and evaluation of surrounding environment maps can be considered as prioritization. Particularly preferably, a weight factor can be assigned to all cells of the surrounding environment map, which is, for example, multiplied by the cell value. Here, prioritization can include increasing the weight factor, making the values of cells representing static objects, for example, more important in vehicle localization, thereby achieving higher accuracy in the obstacle map.
[0016] Particularly preferably, the method further includes the step of identifying movable objects in the vehicle's surrounding environment by means of classification. Here, those cells representing movable objects in the surrounding environment map are excluded from the registration of at least two surrounding environment maps. Preferably, those cells representing movable objects are additionally excluded from the vehicle localization step. This means that cells are not used for the registration of the two surrounding environment maps when a movable object has been identified at the corresponding location in the surrounding environment of the cell. For example, these cells can be ignored during registration. For example, predetermined, especially additional, values can be assigned to these cells, which limit the consideration of the corresponding cells during registration. Thus, particularly high accuracy can be achieved when registering the surrounding environment maps and when locating the vehicle in at least one of these surrounding environment maps, because information that may cause errors due to the different positions of movable objects at different points in time when recording the surrounding environment maps is not taken into account. Furthermore, the method can be implemented particularly simply and resource-efficiently, because, for example, for the registration step and especially for the localization step, a relatively small number of objects in the surrounding environment need to be accurately sensed and analyzed.
[0017] Preferably, the method further includes the step of neutralizing the cells representing movable objects in the surrounding environment map. Here, neutralization is defined as "resetting the cell value to a standard value, especially a standard value that existed before assignment." In particular, neutralization is defined as "resetting the cell so that it represents the state 'no object'." Alternatively, this neutralization can also set the cell to the state "occupied," i.e., it can mark the cell as having an object. By neutralizing cells for identified movable objects, areas representing static objects in the surrounding environment map can be highlighted in a particularly simple way, enabling accurate registration and precise positioning using simple and cost-effective means.
[0018] Preferably, the method further includes the step of identifying traffic participants in the vehicle's surrounding environment by means of classification. Here, the identified traffic participants are classified as movable objects. Here, people, animals, and vehicles of any type, especially those located within the surrounding environment area that the vehicle can traverse, are considered traffic participants. Each of these objects is considered a traffic participant, regardless of whether it is moving or stationary at the sensing time. This means, for example, a parked motor vehicle, whether or not it has passengers, is identified as a traffic participant and accordingly classified as a movable object.
[0019] Further preferably, sensing the vehicle's surrounding environment includes generating camera images using a camera. Here, the objects sensed in the vehicle's surrounding environment are classified by analyzing and evaluating the camera images. The camera is preferably part of the vehicle's environmental sensing device. This classification can be implemented in a simple and reliable manner through the analysis and evaluation of the camera images generated by the camera.
[0020] Preferably, the classification is performed automatically using an analysis and evaluation unit. For example, the analysis and evaluation unit may be part of the vehicle's control equipment, or alternatively, it may be provided separately from the control equipment. In particular, the analysis and evaluation unit can include an algorithm configured to analyze generated camera images for classification and to identify static and moving objects based on the camera images. For example, the analysis and evaluation unit can possess artificial intelligence, which is used to perform the classification.
[0021] Preferably, the classification can be implemented and / or confirmed manually by means of user input from the vehicle user, particularly by means of a human-machine interface, such as a display and / or input device. For example, the generated ambient camera images can be displayed to the user via a display and / or input device, whereby the user can select regions of the image and mark these regions as static or movable. Alternatively or additionally, the display and / or input device can display which regions have been classified as static or movable by the analysis and evaluation unit. These regions can be confirmed or adapted through user input. This allows the method to be implemented in a verifiable and particularly reliable manner for the driver, as the correct classification function can be verified and adapted if necessary through interaction with the driver.
[0022] Preferably, the method further includes the step of identifying L-shaped objects in the environment surrounding the vehicle. Here, the identified L-shaped objects are identified as static objects. An object having two at least substantially straight faces—which intersect at right angles or preferably between 20° and 160°—is considered an L-shaped object, especially where each of these faces is arranged vertically. For example, such an L-shaped object refers to a structural structure such as a wall or a building. Therefore, by identifying such L-shaped structures, it is possible to identify in a particularly simple way which areas of the surrounding environment contain static objects, i.e., immovable objects, preferably without the need for laborious analysis, such as imaging of these objects.
[0023] Furthermore, the present invention proposes a vehicle assistance system. This assistance system includes an environmental sensing device configured to sense the vehicle's surrounding environment and a control device configured to implement the described method. In particular, the environmental sensing device is also configured to sense the vehicle's ranging data. Here, the assistance system can be provided with particularly low and cost-effective hardware, enabling the vehicle to operate with high accuracy and high comfort for the driver.
[0024] Preferably, the environmental sensing device of the auxiliary system includes radar sensors and / or lidar sensors for sensing the surrounding environment. It is particularly advantageous to use radar sensors with a medium effective range, for example, within a range of up to a maximum of 50m, preferably up to a maximum of 25m, and especially at least 5m. Here, a map of the surrounding environment is generated based on the environmental data sensed by means of the environmental sensing device, and preferably the vehicle is also located in the map of the surrounding environment.
[0025] Particularly preferred is that the environmental sensing device of the auxiliary system also includes a camera for sensing the surrounding environment and for generating camera images of the surrounding environment. Preferably, the camera is a close-range camera, particularly configured to sense the surrounding environment in an area between 1m and 20m away from the vehicle.
[0026] Preferably, the auxiliary system further includes an analysis and evaluation unit and / or an input device. Here, the analysis and evaluation unit is configured to automatically classify objects sensed in the vehicle's surrounding environment. For example, the analysis and evaluation unit may be part of the vehicle's control equipment, or alternatively, it may be provided in addition to the control equipment. Alternatively or additionally, this classification can be performed and / or confirmed by means of an input device, particularly by means of manual input by the vehicle user. Attached Figure Description
[0027] The present invention will now be described with reference to embodiments and accompanying drawings. In the drawings, components with the same function are designated by the same reference numerals. Here are examples:
[0028] Figure 1 is a schematic simplified diagram of the operation of a vehicle with an auxiliary system according to a preferred embodiment of the present invention, and
[0029] Figure 2 shows another schematic diagram of the operation of the vehicle in Figure 1. Detailed Implementation
[0030] Figures 1 and 2 show schematic simplified diagrams of a vehicle 10 with an assistance system 50. Here, several different simplified diagrams of the method of operating the assistance system 50 for the vehicle 10 are shown.
[0031] The auxiliary system 50 includes an environmental sensing device 52 configured to sense the surrounding environment U of the vehicle 10. The environmental sensing device 52 includes a radar sensor, a lidar sensor, and a camera. With the aid of the environmental sensing device 52, objects 4 in the surrounding environment U of the vehicle 10 can be sensed, such as objects in the form of obstacles, as schematically shown, for example, in Figure 1(a).
[0032] Additionally, the auxiliary system 50 includes a control device 51 configured to generate an ambient map 2 based on the sensed ambient environment U, which is an image of the ambient environment U, as shown in FIG1(b).
[0033] The surrounding environment map 2 is divided into multiple cells 3. Here, the surrounding environment map 2 is two-dimensional and constructed on a plane of the lane surface, on which the vehicle 10 can move within the surrounding environment U. Each cell 3 is square and has a side length of 1m. The entire surrounding environment map 2 is also square and has ten cells 3 arranged side by side in both length and width, that is, the surrounding environment map 2 reflects an area with a side length of 10m.
[0034] Based on the sensed object 4, a predefined value is assigned to each cell 3 of the surrounding environment map 2 as follows: the cell is located at a location in the surrounding environment map 2 corresponding to the location in the surrounding environment U where object 4 is located. For example, a value representing the status "occupied" is assigned to the corresponding cell 3a in the surrounding environment map 2 that represents the location where object 4 is located in the surrounding environment U (see Figure 1(b)).
[0035] The vehicle 10 in the surrounding environment U can be located using the surrounding environment map 2. To this end, a second surrounding environment map is generated at another point in time. Figure 2b And compare it with at least one other first surrounding environment Figure 2a Phase registration. This is shown simplified in Figure 2(c).
[0036] Here, firstly, during the initial training drive, vehicle 10 is used to navigate, for example, through an area of surrounding environment U under the manual control of the driver. During this training drive, a first ambient environment is generated. Figure 2a This is shown in Figures 2(a) and (b).
[0037] When vehicle 10, for example, relocates to the same area of the surrounding environment U at a later point in time, a second surrounding environment can be generated in the second step. Figure 2b Here, in two surrounding environments Figure 2a , 2b In each surrounding environment map, assign corresponding values to all cells 3 to represent objects 4 or free areas in the surrounding environment U.
[0038] Next, in another step, the two surrounding environments will be... Figure 2a , 2b Mutual registration. This is schematically shown in Figure 2(c). Here, based on those cells 3 representing the determined object 4b, the two surrounding environments can be registered. Figure 2a , 2b Overlapping, and thus enabling adjustments based on the two surrounding environments during the second drive. Figure 2a , 2b The vehicle 10 in the surrounding environment U is located with exceptionally high accuracy.
[0039] In the surrounding environment of the moving vehicle 10, the surrounding environment U may frequently change over time. Within the surrounding environment U, there may be movable objects 4b, such as other traffic participants, such as pedestrians and moving or parked vehicles (see Figure 2(a)). These movable objects 4b may, for example, be present in the environment that creates the surrounding environment. Figure 2a , 2b The two points in time are located in different places, or can be, for example, in a second surrounding environment. Figure 2b The second sensing is no longer located in the same area of the surrounding environment U. In order to achieve simple and reliable functioning of the auxiliary system 50 even in such variable surrounding environment conditions, the method for operating the auxiliary system 50 described below is implemented.
[0040] Here, objects 4 sensed in the surrounding environment U of the vehicle 10 are classified using an environmental sensing device 52. This classification is performed using camera images sensed by the camera of the environmental sensing device 52. Alternatively or additionally, the classification can also be performed based on environmental data recorded using radar sensors and / or lidar sensors. For this purpose, the camera images are analyzed using the analysis and evaluation unit 53 of the auxiliary system 50. Here, the sensed objects 4 are classified into static objects 4a and movable objects 4b. For example, pedestrians and parked vehicles, exemplarily shown in FIG. 2(a), are identified and classified accordingly as movable objects 4b. Additionally, the walls of the restricted parking space in FIG. 2(a) are identified and classified accordingly as static objects 4a.
[0041] Here, the classification can be implemented in a particularly simple way, such that L-shaped objects are searched in the surrounding environment U, and the identified L-shaped objects are identified as static objects 4b. For example, based on the L-shaped intersection of the sides 41 in this case at right angles, the wall in Figure 2(a) can be easily identified as a static object 4b.
[0042] After classifying object 4, the two surrounding environments are then analyzed. Figure 2a , 2b The registration process ensures that only the cells representing the static object 4a are considered. To simplify this process, the surrounding environment is... Figure 2a , 2b The region 40b representing the movable object 4b is neutralized (see Figure 2(b)). This means that the cells 3b representing the movable object 4b are assigned values indicating the state "unoccupied" or "no object". Thus, the corresponding cells 3b are excluded from the surrounding environment. Figure 2a , 2b It is excluded from the registration and therefore from the subsequent positioning of vehicle 10.
[0043] Here, neutralization of a specific cell 3b representing the movable object 4b offers the following advantages: it avoids issues during registration and / or positioning, such as those arising from the corresponding surrounding environment. Figure 2a , 2b The error is caused by the slight movement of object 4 between the two time points. In particular, only the surrounding environment remains. Figure 2a , 2b The area within the surrounding environment U is always located at the same point. Therefore, it is possible to provide the surrounding environment location in a simple way and with particularly high accuracy. Figure 2a , 2b In order to make use of the surrounding environment Figure 2a , 2b It can achieve the highest possible positioning accuracy for vehicle 10.
Claims
1. A method for operating an auxiliary system (50) of a vehicle (10), the method comprising the steps of: - Sensing the surrounding environment (U) of the vehicle (10). - Generate at least two surrounding environment maps (2a, 2b) at different time points, where, Each surrounding environment map (2a, 2b) is divided into multiple cells (3), wherein, based on the presence of an object (4) at a location in the surrounding environment (U) corresponding to the cell (3), a predefined value is assigned to each cell (3). - Classify the objects (4) sensed in the surrounding environment (U) of the vehicle (10). - Identifying static objects using the aforementioned classification (4a), and - Register the at least two surrounding environment maps (2a, 2b) together. The registration is performed based on the corresponding cell (3) of the static object (4a) in the surrounding environment map (2a, 2b).
2. The method according to claim 1, further comprising the step of locating the vehicle (10) in at least one of the surrounding environment maps (2a, 2b) based on the mutual registration of the at least two surrounding environment maps (2a, 2b).
3. The method according to any one of the preceding claims, further comprising the step of: preferentially considering a cell (3) representing a static object (4a) in the surrounding environment (U), wherein, The mutual registration of the at least two surrounding environment maps (2a, 2b) is performed using the preferred cell (3).
4. The method according to claim 1 or 2, further comprising the following steps: - Using the classification, movable objects (4b) in the surrounding environment (U) of the vehicle (10) are identified, wherein, Those cells (3b) representing movable objects (4b) in the surrounding environment maps (2a, 2b) are excluded from the registration of the at least two surrounding environment maps (2a, 2b).
5. The method according to claim 4, further comprising the following steps: - Neutralize those cells (3) representing movable objects (4b) in the surrounding environment map (2a, 2b), wherein, Reset the cell value to the standard value.
6. The method according to claim 5, wherein, The standard value is a standard value that existed before the allocation.
7. The method according to claim 4, further comprising the following steps: - By means of the classification, traffic participants in the surrounding environment (U) of the vehicle (10) are identified, wherein, The traffic participants are identified as movable objects (4b).
8. The method according to claim 1 or 2, wherein, Sensing of the surrounding environment (U) of the vehicle (10) includes generating camera images by means of a camera, and wherein the classification is performed by means of analysis and evaluation of the camera images.
9. The method according to claim 1 or 2, wherein, The classification is carried out automatically by means of the analysis and evaluation unit (53).
10. The method according to claim 1 or 2, wherein, The classification can be implemented and / or confirmed manually by the user input of the user of the vehicle (10).
11. The method according to claim 1 or 2, further comprising the following steps: - Identify L-shaped objects in the surrounding environment (U) of the vehicle (10), wherein, The identified L-shaped objects are classified as static objects (4a).
12. An assistance system for a vehicle (10), said assistance system comprising: - An environmental sensing device (52) for sensing the surrounding environment (U) of the vehicle (10), and - A control device (51) configured to implement the method according to any one of the preceding claims.
13. The auxiliary system according to claim 12, wherein, The environmental sensing device (52) has a radar sensor and / or a lidar sensor for sensing the surrounding environment (U).
14. The auxiliary system according to claim 12, wherein, The environmental sensing device (52) has a camera for sensing the surrounding environment (U) and for generating camera images.
15. The auxiliary system according to claim 13, wherein, The environmental sensing device (52) additionally includes a camera for sensing the surrounding environment (U) and for generating camera images.
16. The auxiliary system according to claim 12 or 13, the auxiliary system further comprising an analysis and evaluation unit (53) and / or an input device (54) for classifying objects (4) sensed in the surrounding environment (U) of the vehicle (10).