Method for creating an environment model of a vehicle

Abstract

Method for creating an environment model of a vehicle, characterized in that sensor data from a sensor system are received (101), at least one object is determined (102) based on the sensor data, the object is displayed to an observer, in particular the driver, after the vehicle has been driven (103) and the object is assigned an object class from a predefined list of object classes by the observer (104).

Description

[0001] The present invention relates to a method for creating an environment model of a vehicle, an environment model unit, a driver assistance system and a vehicle.

[0002] Drivers increasingly desire to be relieved of some of the burden of driving by driver assistance systems. The following levels of automation are distinguished, each with its own set of requirements for these systems.

[0003] "Driver Only" refers to a level of automation where the driver is permanently responsible for longitudinal control (acceleration and deceleration) and lateral control (steering) throughout the entire journey. If the vehicle is equipped with a driver assistance system, this system does not interfere with the vehicle's longitudinal or lateral control. Examples of driver assistance systems that do not interfere with the vehicle's longitudinal or lateral control include, among others, light assistance systems that control headlights depending on the situation, weather, and / or brightness; distance warning systems that warn of hidden obstacles, particularly when parking; rain assistance systems that activate the windshield wipers depending on the amount of water or dirt on the windshield; and an attention assist system that, for example, recommends taking a break based on the driver's pupil movements.A lane change assistant that warns the driver of a lane change without prior activation of the turn signals (indicators), as described, for example, in EP 1 557 784 A1; a traffic sign assistant that alerts the driver to traffic signs, especially speed limits; a blind spot assistant that warns the driver of road users in the vehicle's blind spot; or a reversing camera system that provides the driver with information about the area behind the vehicle, as described, for example, in EP 1 400 409 A2. Further assistance systems are described, among other places, in WO 2007 / 104 625 A1.

[0004] "Assisted" refers to a level of automation where the driver permanently assumes either lateral or longitudinal control of the vehicle. The other driving task is handled within certain limits by a driver assistance system. The driver must continuously monitor the driver assistance system and be ready to fully take over vehicle control at any time. Examples of such driver assistance systems are known as "Adaptive Cruise Control" and "Parking Assistant." Adaptive Cruise Control can, within limits, take over longitudinal control of the vehicle and regulate the vehicle's speed, taking into account the distance to a vehicle ahead. A corresponding radar system is known, for example, from WO 2008 / 040 341 A1. The Parking Assistant assists with parking by taking over steering, although the driver remains in control of the vehicle's forward and reverse movements.A corresponding parking assistant is described, for example, in EP 2 043 044 B1.

[0005] "Partially automated" refers to a level of automation where a driver assistance system takes over both the lateral and longitudinal control of the vehicle for a certain period and / or in specific situations. As with fully automated driving, the driver must continuously monitor the driver assistance system and be ready to fully take over control of the vehicle at any time. An example of a driver assistance system that enables partially automated driving is known as the highway assistant. The highway assistant can take over the longitudinal and lateral control of the vehicle in the specific situation of highway driving up to a certain speed. However, the driver must constantly check whether the highway assistant is functioning reliably and be ready to immediately take over control of the vehicle, for example, upon receiving a prompt from the driver assistance system.

[0006] Even in a level of automation described as "highly automated," a driver assistance system takes over both the lateral and longitudinal control of the vehicle for a certain period and / or in specific situations. Unlike partially automated driving, the driver no longer needs to constantly monitor the driver assistance system. If the driver assistance system independently detects a system limit and, consequently, safe vehicle control is no longer guaranteed, the system prompts the driver to take over. A highway chauffeur can serve as an example of a highly automated driving system. Such a chauffeur could automatically control the vehicle's longitudinal and lateral movements on highways up to a certain speed limit, without requiring the driver to monitor the chauffeur at all times.As soon as the motorway driver detects a system limitation, such as an uncontrolled toll plaza or an unforeseen construction site, it would request the driver to take over control of the vehicle within a certain timeframe. A procedure for highly automated driving is described, for example, in DE 10 2012 101686.

[0007] Even in a level of automation described as "fully automated," the driver assistance system takes over the lateral and longitudinal control of the vehicle, but only within a defined use case. The driver does not need to monitor the driver assistance system. Before leaving this use case, the driver assistance system prompts the driver to take over the driving task with sufficient time. If the driver does not comply with this prompt, the vehicle is put into a low-risk system state. The driver assistance system recognizes all system boundaries and is capable of assuming a low-risk system state in all situations. An example of a driver assistance system that enables "fully automated" driving could be a highway pilot.This system could take over both longitudinal and lateral control of the vehicle on highways up to a certain speed limit. The driver would not need to monitor the highway pilot and could attend to other tasks, such as preparing for a meeting, thus making the best use of travel time. As soon as it is necessary to exit the highway, the highway pilot would prompt the driver to take over. If the driver does not respond to this prompt, the highway pilot would brake the vehicle and preferably steer it onto a parking area or shoulder, where it would be brought to a complete stop. A method for fully automated operation has been proposed, for example, in US 8,457,827 B1.

[0008] As soon as continuous monitoring of the driver assistance system is no longer planned, i.e. in the case of partially or highly automated driving, relevant objects, e.g. other road users or traffic signs, must be recognized with high reliability, which cannot always be guaranteed with previous methods, for example from WO 2013 / 087 067 A1, and by previous driver assistance systems.

[0009] DE 60 2004 011 164 T2 discloses that a detection unit recognizes targets located in front of the vehicle based on a detection result obtained from a preview sensor and then classifies the recognized targets according to the types to which they belong. A control unit determines the information to be displayed based on the targets recognized by the detection unit and the navigation information. A display device is controlled by the control unit to show the determined information. The control unit controls the display device so that symbols indicating the recognized targets are displayed to be superimposed on the navigation information and also controls the display device so that the symbols are displayed using a variety of different display colors corresponding to the types to which the respective targets belong.

[0010] DE 10 2013 102 087 A1 discloses that a driver assistance system comprises a sensor that detects objects and / or features of objects in the vicinity of a motor vehicle, and an object recognition unit that evaluates data from the sensor to generate object information and determines a confidence value indicating the probability of the accuracy of this information. If the confidence value is greater than a minimum threshold but less than a predefined threshold, the system displays the object information to the driver, who confirms or rejects it, e.g., by actuating or not actuating a control. If confirmed by the driver, the system begins or continues the execution of a driver assistance function based on the object information. If rejected by the driver, the system does not execute or discontinues the driver assistance function based on the object information.If the confidence value exceeds the predefined threshold, the system executes the driver assistance function independently.

[0011] Driver assistance systems typically rely on an environment model for control, which provides them with the necessary information about objects, infrastructure, road layout, etc.

[0012] The detected objects can be of different types and each triggers a different rule. For example, it's possible to drive very close to infrastructure, such as a wall. In contrast, overtaking a cyclist requires maintaining a safety distance of at least 1.5 meters.

[0013] Based on this, the present invention was based on the objective of providing a method for creating an environment model which can more precisely indicate to a driver assistance system the objects located in the environment of the vehicle.

[0014] According to the invention, this problem was solved by a method for creating an environment model according to claim 1, an environment model unit according to claim 8, a driver assistance system according to claim 9, and a vehicle according to claim 10. Advantageous embodiments of the method are described in claims 1 to 7, which refer back to claim 1.

[0015] The method for creating an environment model of a vehicle, in which sensor data from a sensor system is received, at least one object is determined based on the sensor data, the object is displayed to an observer, in particular the driver, after the vehicle has traveled, and the observer assigns an object class to the object from a predefined list of object classes, can enable a more precise description of the vehicle's surroundings. The object class can be used by a driver assistance system that relies on the environment model, for example, to determine the risk of an object suddenly changing lanes. Furthermore, the object class can influence the decision of whether to perform a risky evasive maneuver around an obstacle or to drive into it in a controlled manner.For example, if the object is assigned to the object class "passenger car," it may be more sensible to accept the material damage of a collision and not expose the driver to the risk of an evasive maneuver with an uncertain outcome. On the other hand, the risk of an evasive maneuver may be acceptable if the object is assigned to the object class "pedestrian," whose health would be substantially endangered in a collision. The list of object classes may include, in particular, the object classes "truck," "motorcycle," "infrastructure," "cyclist," "pedestrian," "passenger car," "tram," and "bus." Displaying the object after the vehicle has passed can prevent additional strain on the viewer while the vehicle is in motion. The viewer can, for example, concentrate fully on driving the vehicle. Based on the subsequent classification of the objects, i.e.,However, by assigning an object class to the respective object, the environment model can still be further improved.

[0016] Sensor data can be acquired using various sensor systems. Examples of suitable sensor systems include radar, laser, ultrasound, and camera systems. Radar systems can detect objects even in precipitation or fog. A laser system is described, for example, in WO 2012 / 139 796 A1. Laser systems are characterized by a particularly long detection range. The detection accuracy of ultrasound sensors, such as those described in WO 2013 / 072 167 A1, can be especially high at close range. Camera systems can offer higher resolution compared to other sensor systems. Infrared camera systems can distinguish between living and non-living objects. Furthermore, camera systems can be combined to create stereo camera systems to obtain distance information.

[0017] The process primarily utilizes sensor data from the vehicle's sensor systems whose environment is to be modeled. However, sensor data from other vehicles can also be used. Furthermore, it is conceivable to access sensor data from infrastructure sensor systems, such as traffic lights, traffic monitoring cameras, fog sensors, or road contact loops. The data can be transmitted wirelessly from vehicle to vehicle or initially to a background system, i.e., a back-end.

[0018] The described method for creating a vehicle environment model can improve the reliability of assigning objects to specific object classes, i.e., the classification accuracy of the sensors. Ultimately, improved object assignment to object classes can help reduce the number of unnecessary driver takeovers, which can lead to improved driving comfort and a lower stress level.

[0019] In an initial implementation of the method for creating an environment model of a vehicle, the predefined list of object classes is determined based on sensor data. The sensor data can, in particular, make it possible to shorten the list of object classes. For example, a radar system can determine whether an object consists primarily of metal. If this is not the case, object classes such as passenger cars, trucks, and / or motorcycles can be removed from the predefined list. The shorter predefined list of object classes makes it easier for the observer to identify the object.

[0020] According to another further development of the procedure for creating an environment model of a vehicle, a preliminary object class is assigned to the object based on the sensor data and the preliminary object class is displayed to the viewer.

[0021] Based on the sensor data, an assumption about the object class can often be made. For example, if the detected object is over 10 meters long, it is highly likely to be a truck. Therefore, the object can be provisionally assigned the object class "truck." The observer then only needs to confirm that the object should be assigned the object class "truck." However, it is also conceivable that the object was incorrectly assigned the provisional object class "truck." For example, the sensor data for an ISO container located at the roadside might differ only slightly from that of a truck. A human observer, on the other hand, would recognize that the ISO container is an immobile obstacle.The viewer can thus correct the incorrect assignment to the provisional object class "truck" and correctly assign the object to the object class "infrastructure".

[0022] Furthermore, a further development of the procedure for creating an environment model of a vehicle is characterized by the fact that a classification certainty is specified with which the object is assigned to the preliminary object class.

[0023] Displaying a classification certainty can increase the viewer's confidence in the automatic assignment of an object to a preliminary object class.

[0024] Furthermore, one embodiment of the method for creating an environment model of a vehicle provides that the object is displayed to an observer while the vehicle is driving.

[0025] The viewer can thus directly assign the object to an object class. This assignment of object to object class can be provided, for example, via a background system or directly to another vehicle.

[0026] A method for contact-analog display is described, for example, in DE 10 2012 215 216 144 A1. With contact-analog display, objects can be shown to the viewer directly at the location where they would normally see them. With contact-analog display, the driver does not have to refocus between displays at close range in the head-up display, instrument cluster, or central information display and the actual driving situation at a greater distance. The merging of the displayed objects with the immediately perceived environment can enable a viewer to assign an object to a specific object class much more easily. If, for example, data glasses as described in DE 196 25 435 A are used, the viewer's field of vision can be less restricted.For example, the smart glasses can display objects located to the side of the vehicle as soon as the viewer looks out of the vehicle's side window. Another application of smart glasses, in particular augmented reality glasses, is described in DE 10 2013 005342 A1.

[0027] With regard to the environment model unit, the task described above was solved by an environment model unit, wherein the environment unit has a receiving device for receiving sensor data from at least one sensor system, and wherein the environment model unit is configured to perform one of the procedures described above.

[0028] With regard to the driver assistance system, the solution to the problem derived above consists in a driver assistance system which is configured to receive an environment model from an environment model unit, in particular as described above, and which is configured to control at least one operating parameter of a vehicle, in particular its speed and / or its distance to a road user ahead, based on the environment model.

[0029] Finally, a solution to the above-mentioned problem consists of a vehicle which has a sensor system for detecting the vehicle's surroundings as well as an environment modeling unit and / or a driver assistance system as described above.

[0030] The vehicle in question can be, in particular, a motorized individual means of transport. Primarily, this would be a passenger car. However, it is also conceivable that the vehicle could be a motorcycle.

[0031] The following are examples of implementation explained in more detail with reference to the figures. This shows Fig. 1. An exemplary illustration of the procedure for creating an environmental model of a vehicle; and Fig. 2. An exemplary view from the cockpit of a vehicle before the classification of an object. Fig. 3. An exemplary view from the cockpit of the vehicle after the classification of the object.

[0032] According to the Fig. In the flowchart shown in 1, according to an exemplary embodiment, sensor data from a sensor system are received in a first step 101. The sensor system can be, for example, a radar system, a laser system, an ultrasound system, or a camera system.

[0033] In a second step, 102, an object is determined based on the sensor data. The object can, for example, be represented as a cuboid that moves relative to the vehicle and relative to the road.

[0034] The detected object, the cuboid, is then displayed to a viewer in a third step. In this example, the cuboid is displayed to the viewer using smart glasses. Alternatively, the object can also be displayed to the viewer in a contact-analog head-up display.

[0035] The observer can then assign the object to an object class in step 104, e.g., passenger car, truck, infrastructure, or critical / non-critical. If the vehicle is in a highly automated driving mode, the driver is the observer, and recognizes that a critical object has not been detected based on the sensor data, the observer (i.e., the driver) can also take over control of the vehicle.

[0036] In a further step, the viewer's assignment to the object class—that is, the viewer's feedback—is stored to enable automated assignment of a comparable object to the same object class the next time. In terms of machine learning, the vehicle's environment model can thus be continuously refined and improved.

[0037] In the Fig. Figure 2 shows a view from the cockpit of a vehicle. The vehicle is traveling in the middle lane 202 of the three lanes 201, 202, and 203. In the right lane 203, a truck 204 is located, for example, at a distance of more than 200 meters. The truck 204 is detected by the vehicle's sensor system as a moving object relative to the vehicle and the roadway and is marked with a frame 205 on a contact-analog head-up display. A question mark 206 is displayed within the frame 205, indicating that the object marked by the frame 205 has not yet been assigned to an object class.

[0038] By means of an input device 307 located on the steering wheel 306 of the vehicle, the observer and driver of the vehicle can, as in the Fig.3. An object class can be assigned to the object. The assignment of the object to the object class is confirmed to the viewer by an "OK" displayed in frame 305.

Claims

[1] Method for creating an environment model of a vehicle, characterized by , that sensor data from a sensor system is received (101), based on the sensor data at least one object is determined (102), the object is displayed to an observer, in particular the driver, after the vehicle has traveled (103) and the observer assigns an object class from a predefined list of object classes to the object (104). [2] Method for creating an environment model of a vehicle according to claim 1, characterized by , that the predefined list of object classes is determined based on the sensor data. [3] Method for creating an environment model of a vehicle according to one of claims 1 or 2, characterized by , that a preliminary object class is assigned to the object based on the sensor data, and that the preliminary object class is displayed to the viewer. [4] Method for creating an environment model of a vehicle according to claim 3, characterized by , which specifies its classification certainty, with which the object is assigned to the preliminary object class. [5] Method for creating an environment model of a vehicle according to any one of the preceding claims 1 to 4, characterized by , that the object is displayed to a viewer while the vehicle is in motion. [6] Method for creating an environment model of a vehicle according to any one of the preceding claims 1 to 5, characterized by that the object is displayed to the viewer by means of a contact-analog head-up display or data glasses. [7] Environment model unit, characterized bythat the environment model unit has a receiving device for receiving sensor data from at least one sensor system, and that the environment model unit is configured to perform a method according to any one of the preceding claims 1 to 6. [8] Driver assistance system, characterized by that the driver assistance system is configured to receive an object with the object class assigned to it from an environment model unit according to the preceding claim 7, and that the driver assistance system is configured to control at least one operating parameter of a vehicle, in particular its speed and / or its distance to a road user ahead, taking into account the object and the object class assigned to the object. [9] Vehicle with at least one sensor system for detecting the vehicle's surroundings and an environment modeling unit according to claim 7 and / or a driver assistance system according to claim 8.

Images