Validation of road lane models based on swarm data

Abstract

Procedures (900), comprising: - Determining (901) a property of a roadway lane (300) in the environment of a vehicle from a fusion of swarm data (201) and sensor data (101) acquired by the vehicle, - Validating (902) a model of the roadway lane (300) taking into account the property of the roadway lane (300), and - Use (903) the model of the road lane (300) in an environment model of the vehicle after the road lane (300) has been validated.

Description

TECHNICAL AREA

[0001] The present invention relates to techniques for robust control in lateral driver assistance systems. These techniques relate in particular to techniques for stabilizing an environment model in driver assistance systems. Swarm data is used to stabilize the environment model. BACKGROUND OF THE INVENTION

[0002] Driver assistance systems for vehicles, especially for cars or trucks, make it easier for the driver to orient themselves while driving. In a vehicle equipped with a driver assistance system, a model of the surroundings can be displayed to the driver on a screen. This model can show objects such as traffic signs, lanes, etc. It can also show objects detected by sensors, such as cars, motorcycles, trucks, construction sites, etc. Sensors used to detect these objects can be, for example, cameras or radar sensors integrated into the vehicle.

[0003] Cameras can, for example, detect markings on the road and determine their distance from one another. This allows the width of a road lane (hereinafter simply referred to as lane) to be determined. As soon as the lane width exceeds a predefined value, it is recognized and displayed as a driving lane or shoulder. If the lane width falls below the predefined value, the lane is hidden, for example, in the surrounding environment model.

[0004] In practice, however, a changing or non-constant lane width poses a problem. For example, it has been observed that, based on camera data, the lane width can sometimes only be detected inaccurately or with fluctuations. This can lead to the lane, e.g., the hard shoulder of a motorway, being switched on and off. The environmental model displayed to the driver in driver assistance systems is then described as unstable.

[0005] German patent application DE 10 2019 216 914 A1 describes a system and method for the automated generation of semantic maps, comprising at least one processing system with at least one processing unit. The processing system is designed to generate a probabilistic road model for road elements. The processing system is designed to generate a probabilistic traffic model for the road model. The processing system is designed to perform statistical inference on the road model and the traffic model. The processing system is designed to generate an estimate of a semantic road model based on the statistical inference. The processing system is designed to generate a semantic map that includes the estimate of the semantic road model.Such a system has the disadvantage that it is not always easy to reliably generate the probabilistic traffic model.

[0006] The publication DE 10 2017 215 708 A1 describes a method for determining a quality map describing the quality of road markings along drivable road sections, for use in the fully automated guidance of a motor vehicle, wherein in each motor vehicle of a fleet comprising multiple motor vehicles, road markings in the vicinity of the motor vehicle are detected by evaluating sensor data from at least one environmental sensor, at least one quality information describing the quality of the road markings and / or the quality of the detection of the road markings is determined, and the quality information and an associated position information describing its recording location are transmitted to a stationary backend device external to the motor vehicle, wherein the transmitted,The backend system uses location-specific quality information of the vehicle fleet to create and / or update the quality map. However, this method has the disadvantage that the quality of road markings often cannot be reliably determined.

[0007] The document DE 10 2017 212 361 A1 describes a method for operating at least one motor vehicle, wherein surface information relating to a road surface of at least one road section is sent to a receiving motor vehicle by a central device or a sending motor vehicle, wherein a control device of the receiving motor vehicle automatically intervenes in the driving operation depending on the surface information, while the motor vehicle is in the road section.

[0008] The publication DE 10 2011 084 264 A1 describes a communication system for a motor vehicle, wherein the communication system has a telemetry endpoint, in particular with a multitude of interfaces, as well as a motor vehicle control unit endpoint, wherein the communication system further comprises a bus via which the endpoints communicate with each other, and wherein the communication system has a firewall which monitors the communication between the endpoints.

[0009] The document DE 10 2019 217 428 A1 describes a method for operating a driver assistance system for the automated execution of a driving maneuver, wherein the release of the driving maneuver is checked before the execution of the driving maneuver and / or the necessity of aborting the driving maneuver is checked during the execution of the driving maneuver, wherein the release of the driving maneuver and / or the necessity of aborting is checked depending on swarm data.

[0010] The publication DE 10 2017 206 343 A1 describes a method for determining data from a traffic scenario, comprising the steps: - capturing the environment of a vehicle using a sensor device; - capturing the behavior of road users using the sensor device; - combining and evaluating the captured data of the environment and the behavior of the road users; and - storing the combined and evaluated data. SUMMARY OF THE INVENTION

[0011] Therefore, there is a need for techniques to stabilize the environmental model in driver assistance systems. In particular, there is a need for techniques that avoid the lane being repeatedly faded in and out.

[0012] The object of the present invention is to stabilize the use of a lane in an environment model of driver assistance systems and thus improve the support provided to the driver by these systems. Particularly in the case of lateral driver assistance systems, i.e., systems that assist, for example, when the vehicle changes lanes, stable detection and thus the use of lanes is of crucial importance.

[0013] According to the present invention, this problem is solved by a method according to claim 1, a driver assistance system according to claim 14, and a vehicle according to claim 15. The dependent claims define preferred and advantageous embodiments of the present invention.

[0014] According to the present invention, a method for stabilizing an environment model in driver assistance systems is provided. In this method, a property of a lane is determined from a fusion of swarm data and local sensor data – that is, sensor data acquired by the respective vehicle.

[0015] Swarm data is the collection of data originating from various objects, such as vehicles. Swarm data can, for example, refer to an aggregation of data collected from different objects. For instance, swarm data for specific landmarks can include information from various objects. Swarm data can be distinguished from map data. Map data contains only one set of information for specific landmarks, not multiple sets from different objects. Swarm data can, for example, include image data of landmarks from multiple sources, i.e., from the various objects. Alternatively or additionally, swarm data could also include positional information for landmarks, such as information about the arrangement of traffic signs, traffic lights, lane layouts, etc.Position information can be determined, for example, based on the navigation systems of the various vehicles. Within the swarm data, other sensors could also be considered as information sources, such as radar, ultrasonic, or LiDAR sensors. The swarm data can, for example, include references between image data from different sources, such as transformation instructions that describe the relationship between the different perspectives.

[0016] Taking the track's properties into account, the track is validated. More precisely, a model of the track can be validated. This track model can then be used in the environment model. The track model thus describes the physical track within the context of the environment model. For example, the track model can describe its path, width, and appearance. For the sake of simplicity, the following discussion will focus on the use of the track within the environment model.

[0017] The lane is used – for example, in a vehicle environment model – after it has been validated. Such use of the lane could include, for instance, displaying the lane in a view of the vehicle's surroundings. It could also be used in connection with an emergency braking assistant that identifies a stationary area for braking the vehicle.

[0018] Using swarm data allows for a more precise determination of a lane's characteristics. Swarm data has the advantage of a greater range (> 50m) than conventional sensor data, such as camera data (approx. 50-60m). In other words, swarm data allows us to "see" further ahead than, for example, a camera mounted in the vehicle. Furthermore, by incorporating swarm data, information from other road users can also be integrated. This allows for a more unambiguous identification of a lane, leading to a more stable environmental model.

[0019] According to one embodiment, the lane characteristic can include the lane type. The lane type can be, for example, a hard shoulder or a driving lane. This can be distinguished, for instance, by lane width and / or lane markings. In this way, the driver can obtain a more precise representation of the surroundings, particularly of the lanes under consideration. Especially if a lane change is desired or necessary by the driver, the driver can obtain detailed information from the driver assistance system.

[0020] Furthermore, the track property can include a minimum, maximum, and / or average width. Determining the track width allows conclusions to be drawn about the type of track. The track width property enables track validation. By considering the track width, a more stable parameterization of the track and consequently an improved environmental model can be achieved.

[0021] According to another embodiment, swarm data can be loaded from a cloud while the vehicle is driving. Loading swarm data while driving allows the environmental model to be constantly updated and adapt to changing traffic conditions. For example, only swarm data that is no older than a predefined threshold could be considered. This would allow, for instance, the inclusion of mobile construction sites and other obstacles.

[0022] The process can include locating the vehicle using swarm and sensor data before determining the characteristics of a track. This improves the fusion of swarm and sensor data, thus avoiding the need to compare sensor data with swarm data relating to different landmarks.

[0023] Furthermore, fusion can involve overlaying swarm data and sensor data. This means, for example, that camera images from the swarm data and sensor data are superimposed or combined into a single image. This allows for the precise determination of a track's characteristics. For instance, image registration could be performed. Objects in the camera images could be detected and registered against each other.

[0024] For example, the fusion can be based on detecting objects near the track. This represents a way to efficiently and accurately implement the fusion of swarm data and sensor data.

[0025] According to another embodiment, the validation of the track can be based on a comparison of the track's properties with a limit value. For example, the track can be validated by comparing the coil's properties with a quantity relevant to the track. This allows, for instance, verification that the track has a certain minimum width. It could also be verified whether the track type meets specific requirements for a given road type.

[0026] It would be possible to compare the track's property with a limit value relative to a threshold. This would allow for the consideration of a tolerance for deviations between the property and a quantity relevant to the track.

[0027] The method can also include determining another property of the track from swarm data or sensor data. This can improve the validation of the track.

[0028] Further characteristics of the lane can include the environment, its use by road users, or changes in the lane's characteristics. Various factors can be considered to support and improve lane validation.

[0029] According to another embodiment, the limit value and the threshold value can be applied depending on the other properties of the track. The validation of the track can thus be adapted.

[0030] The procedure can further include checking the validation of the trace using an additional property. This can improve the validation of the trace and thus stabilize the environment model.

[0031] The present invention further provides a driver assistance system located in a vehicle and comprising a camera configured to perform the method as previously described.

[0032] Finally, according to the present invention, a vehicle is provided which includes a driver assistance system as previously described.

[0033] The examples described above and those described below can be combined with each other and with further examples. BRIEF DESCRIPTION OF THE FIGURES

[0034] The properties, features and advantages of this invention described above, as well as the manner in which they are achieved, will become clearer and more easily understood in connection with the following description of the exemplary embodiments, which are explained in more detail in conjunction with the figures. Fig. Figure 1 shows several vehicles with a camera and a cloud. Fig. Figure 2 shows a vehicle according to an embodiment of the present invention with a driver assistance system comprising a camera, driving on a road. Fig. Figure 3 shows process steps of a method according to an embodiment of the present invention. Fig. Figure 4 schematically shows a validation of the trace taking into account the property of the trace. Fig. Figure 5 shows process steps of a method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0035] The drawings are to be regarded as schematic representations, and the elements depicted in the drawings are not necessarily shown to scale. Rather, the various elements are represented in such a way that their function and general significance become apparent to a person skilled in the art.

[0036] The following describes techniques for stabilizing an environmental model in driver assistance systems. Stabilizing the environmental model prevents sudden fluctuations and reduces measurement artifacts. These methods can assist the driver in orienting themselves in traffic, provide decision support, and ultimately enable autonomous driving.

[0037] The following describes techniques for stabilizing an environment model in driver assistance systems, particularly with regard to determining a lane property. However, these techniques can also be more generally based on determining a property of an environment object, i.e., an object located in the vicinity of the vehicle. Other types of environment objects include, for example: guardrails; road surface; traffic signs; traffic management systems such as traffic lights; etc.

[0038] Fig. Figure 1 shows several vehicles 100, each with a camera 103, and a cloud 200. Each vehicle records its surroundings with its respective camera. The camera data is sent to a cloud 300, where it is collected and, if necessary, further processed. The amount of data originating from various objects, such as vehicles, is called swarm data. Swarm data can contain, among other things, historical information, for example, information about how other vehicles drove in the respective lanes, e.g., on a highway. Using this swarm data, one can, for example, determine that two lanes on a highway were occupied by other road users, but no one drove on the shoulder. Swarm data, stored, for example, in a cloud 300, can be retrieved or sent from there. Swarm data can be understood as an additional sensor source.

[0039] Fig. Figure 2 shows a vehicle 100 according to an embodiment of the present invention with a driver assistance system 102 coupled to a camera 103. The vehicle 100 is driving in lane 300 of a road. The driver assistance system 102 can be implemented in hardware and / or software. The vehicle 100, or rather the driver assistance system 102 of the vehicle 100, can retrieve and receive swarm data 201 from a cloud 200. The cloud 200 can send swarm data 201 to the vehicle 100, or rather the driver assistance system 102 of the vehicle 100. The camera 103 of the vehicle 100 can detect the vehicle's surroundings up to a certain range. The camera 103, or rather the driver assistance system 102, can provide the camera data, i.e., sensor data 101. The driver assistance system 102 can include a processor, memory, an interface, and a display.The processor can be configured to load and execute program code from memory. Executing this program code can cause the processor to receive and analyze data from sensors and swarm data. Furthermore, executing the program code can cause the processor to display a track on the screen if the data analysis detects a track near the vehicle.

[0040] Swarm data 201 can be loaded from a cloud 200 while driving 906. Since the vehicle 100 is moving on a road, the surroundings are constantly changing. The surroundings can be continuously determined or updated from sensor data 101, e.g., image data from the camera 103. By repeatedly determining the current surroundings at sufficiently short intervals, changes in the surroundings can be identified, e.g., the addition or removal of lanes or shoulders. Furthermore, swarm data 201 can be continuously loaded from a cloud 200 or an external server while driving. Only map sections suitable for the driver or the vehicle can be loaded as swarm data 201 from the cloud 200. The swarm data 201 comprising the map sections can have a small data volume.This allows not only changes detected by the vehicle's own sensors, but also changes detected by other road users to be transmitted to and recorded by the vehicle and driver via the Cloud 200. The driver thus benefits from the sensors of other road users, for example, those who can already detect a construction site that is not yet visible to the driver. With fast data transmission and processing, the driver can also learn about the construction site almost simultaneously with the other road user who detects it. This example illustrates how loading swarm data while driving allows the environmental model to be constantly updated and continuously adapted to changing traffic conditions.

[0041] Fig. Figure 3 shows process steps of a method 900 according to an embodiment of the present invention. The method 900 serves to stabilize an environment model in driver assistance systems 102 and can comprise determining 901 a property of a track 300 from a fusion of swarm data 201 and sensor data 101, validating 902 the track 300 taking into account the property of the track 300, and using 903 the track 300 after the track 300 has been validated.

[0042] The fusion can involve an overlay of swarm data 201 and sensor data 101. The fusion can be based on the detection of objects 500 near track 300. As in Fig. As shown in Figure 2, a 100 km / h speed limit sign 500 can be detected by camera 103 at the edge of the road. Using this object 500, the corresponding swarm data 201 can be loaded from cloud 200 and used for fusion. The position of vehicle 100 can then be determined using this object 500.

[0043] For example, the property of lane 300 can include the type of lane 300. The type of lane 300 can be, for example, a hard shoulder 301 or a driving lane 302. As in Fig. As shown in Figure 2, vehicle 100 can be located on a hard shoulder 301 or in a driving lane 302. The lanes adjacent to the vehicle's lane can also be driving lanes 302 or hard shoulders 301. Especially on a motorway, it can be advantageous for the driver to be shown whether the adjacent lane is a hard shoulder. A characteristic of a hard shoulder might be, for example, its almost non-existent lane occupancy.

[0044] Fig. Figure 4 schematically shows a validation of the track taking into account the track's property. According to one embodiment, in method 900, the property of the track 300 can include a minimum, maximum, or average width 401. The minimum, maximum, or average width 401 can be determined based on the swarm data 201 and camera data 101. The validation of the track 300 can be based on a comparison of the track 300's property with a limit value 402. For example, a difference between the minimum, maximum, or average width 401 and the limit value 402 can be calculated. For example, an integral of the difference between the minimum, maximum, or average width 401 and the limit value 402 can be calculated over time. The comparison of the track 300's property, for example, the minimum, maximum, or average width 401, with a limit value 402 can be related to a threshold value 403.For example, a relation of the integral of the difference between the minimum, maximum, or average width 401 and the limit value 402 over time to the threshold value 403 can be determined. For example, it can be determined that the integral of the difference between the minimum, maximum, or average width 401 and the limit value 402 over time should be greater than the threshold value 403 for a side strip to be validated.

[0045] Fig. Figure 5 shows process steps of a process 900 according to an embodiment of the present invention. Apart from process steps 901 to 903, which are already described in Figure 5, the process 900 can be carried out using the following methods: Fig. As explained in section 3, further procedural steps may be included. Procedure 900 may also include locating 907 the vehicle 100 using swarm data 201 and sensor data 101 before determining the property of a lane 300. Procedure 900 may also include determining 904 a further property of lane 300 from swarm data 201 or sensor data 101. The further property of lane 300 may include the environment, the use of lane 300 by road users, or changes in the property of lane 300. The limit value 402 and the threshold value 403, which are described in Fig.The parameters described in section 4 can be applied or adjusted depending on the additional properties of lane 300. For example, the threshold can be reduced if the journey takes place at night, as the camera's detection capability increases at night due to reduced ambient light, thus resulting in a lower detection error. For instance, after detecting a rural road, a narrower hard shoulder, and therefore a lower threshold, can be assumed. Furthermore, procedure 900 can include a check 905 of the validation of lane 300 using the additional properties. Thus, the procedure can display a hard shoulder only if it appears plausible based on further indicators, such as the presence of a motorway or low traffic volume.

[0046] The order of the described procedural steps may vary. Reference symbol list 100 vehicles 101 sensor data 102 Driver assistance systems 103 Camera 200 Cloud 201 swarm data 300 lanes 301 Emergency lane 302 lanes 401 Width of a track 402 Limit value 403 Threshold 500 objects 900 procedures 901 Determining a property of a trace 902 Validating a track 903 Using a track 904 Determining another property of a trace 905 Checking the validation of a track 906 Loading swarm data 907 Locating a vehicle

Claims

Method (900), comprising: - Determining (901) a property of a road lane (300) in a vehicle environment from a fusion of swarm data (201) and sensor data (101) acquired by the vehicle, - Validating (902) a model of the road lane (300) taking into account the property of the road lane (300), and - Using (903) the model of the road lane (300) in an environment model of the vehicle after the road lane (300) has been validated. Method (900) according to claim 1, wherein the property of the roadway lane (300) comprises the type of roadway lane (300), wherein the type of roadway lane (300) is, for example, a hard shoulder (301) or a driving lane (302). Method (900) according to one of the preceding claims, wherein the property of the roadway lane (300) comprises a minimum, maximum and / or average width (401). Method (900) according to one of the preceding claims, wherein the swarm data (201) are loaded from a cloud (200) during a journey of the vehicle (906). Method (900) according to one of the preceding claims, further comprising: - Locating (907) the vehicle (100) using the swarm data (201) and sensor data (101) prior to determining the property of a roadway lane (300). Method (900) according to one of the preceding claims, wherein the fusion comprises a superposition of the swarm data (201) and sensor data (101). Method (900) according to one of the preceding claims, wherein the fusion is based on the detection of objects (500) in the vicinity of the roadway lane (300). Method (900) according to one of the preceding claims, wherein the validation of the model of the roadway track (300) is based on a comparison of the property of the roadway track (300) with a limit value (402). Method (900) according to claim 8, wherein the comparison of the property of the roadway lane (300) with a limit value (402) is set in relation to a threshold value (403). Method (900) according to one of the preceding claims, further comprising: - Determining (904) a further property of the roadway track (300) from swarm data (201) or sensor data (101). Method (900) according to claim 10, wherein the further property of the roadway lane (300) includes the surroundings, the use of the roadway lane (300) by road users or the change of the property of the roadway lane (300). Method (900) according to claim 9, as well as claim 10 or 11, wherein the limit value (402) and the threshold value (403) are applied depending on the further property of the roadway lane (300). Method (900) according to one of claims 10 to 12, further comprising: - Checking (905) the validation of the roadway track (300) using the further property. Driver assistance system (102) implemented in a vehicle (100) and coupled with a camera (103), wherein the driver assistance system (102) is configured to perform the method (900) according to one of the preceding claims. Vehicle (100) with driver assistance system (102) according to claim 14 .

Images