Method and device for detecting objects and / or structures in the vicinity of a vehicle

Abstract

Methods for detecting objects and / or structures in the vicinity of a vehicle, wherein - the position of the vehicle (F) is determined (1); - Localization data (LD) for the determined position are transmitted to a central server (S) (2); - Swarm data (SD) for a future position following a journey route are received from the central server (S) (3), wherein the swarm data are based on a statistical evaluation of sensor data that were recorded at an earlier time at the future position by a large number of other vehicles and include information about an object and / or structure located ahead on the vehicle's route with a negative result from plausibility checks by a large number of vehicles; and - using the received swarm data (SD) (4) it is checked whether sensor detection is affected by the preceding object and / or structure by using the information contained in the swarm data about the preceding object and / or structure to validate the detection.

Description

[0001] The present invention relates to a method for detecting objects and / or structures in the environment of a vehicle, which in particular ensures robust detection of such objects or structures. The present invention further relates to a corresponding device.

[0002] A variety of assistance systems are now being used in motor vehicles, particularly to improve driver comfort and safety when driving on the road. For this purpose, sensors for monitoring the vehicle's surroundings, such as one or more video cameras or ultrasonic, radar, or laser-based sensors, are installed in the vehicle.

[0003] Simple driver assistance systems inform and warn the driver about particularly dangerous situations, while more advanced systems also intervene automatically in the vehicle's control. For example, adaptive cruise control, often referred to as ACC (an abbreviation of the English term “Adaptive Cruise Control”), can provide adaptive cruise control. This uses a sensor, often a radar sensor located in the front, to determine the position and speed of a vehicle in front, and then the speed and distance of the following vehicle equipped with adaptive cruise control is automatically regulated accordingly using engine and braking intervention. This automated longitudinal control relieves the driver, making long motorway journeys, for example, more comfortable. It can also be used, if necessary.In conjunction with an emergency braking assistant, they contribute to driving safety and enable more energy-efficient driving. Lane departure warning systems, which use cameras to detect side markings, can ensure that the vehicle stays within its lane and, if necessary, automatically countersteers. Furthermore, a lane departure warning system and adaptive cruise control can be used together for an emergency system that automatically initiates an emergency stop. For example, sensors can detect a driver's unconsciousness based on the driver's failure to steer, brake, or accelerate for an extended period.

[0004] While optical sensor systems such as infrared, lidar or camera sensors depend on good visibility, ultrasonic and radar sensors can be operated without any problems even in the dark, in fog or under rapidly changing light conditions, such as when entering or exiting a tunnel. Even under such conditions, radar sensors in particular can precisely measure the distances between objects detected by the sensors and the vehicle. They can also separate multiple objects such as other vehicles, guardrails or people and track their movement, whereby their relative speeds can also be precisely determined. For this reason, radar sensors are often used for driver assistance systems, either in addition to other sensors, in which case a joint evaluation of the generated sensor data takes place, or completely based on radar sensors.

[0005] But even if objects in the vehicle's surroundings and structural limitations of the roadway, such as crash barriers, can be reliably detected with radar or ultrasonic sensors under many environmental conditions, there are also situations in which the detection is distorted or disturbed and the detected objects are implausible.

[0006] Against this background, DE 10 2015 219 933 A1 describes methods for verifying the plausibility of measured values ​​generated, for example, by a vehicle's sensors. To increase the safety and reliability of the sensors, an ego vehicle equipped with V2X technology uses the V2X information from V2X vehicles in the local area, among other things, to verify its own sensor data.

[0007] DE 10 2018 205 322 A1 relates to a method for detecting a malfunction of at least one environmental sensor of a motor vehicle. The malfunction can be detected while the motor vehicle is traveling along a road network. The invention also includes a control unit for implementing the method.

[0008] It is an object of the invention to provide an improved method for detecting objects and / or structures in the environment of a vehicle and a corresponding device.

[0009] This object is achieved by a method having the features of claim 1 and by a device according to claim 11. Preferred embodiments of the invention are the subject of the dependent claims.

[0010] In a method according to the invention for detecting objects and / or structures in the vicinity of a vehicle, the position of the vehicle is determined. Localization data for the determined position is transmitted to a central server. Swarm data for a future position following travel along a route is received by the central server, wherein the swarm data is based on a statistical evaluation of sensor data that was recorded at an earlier point in time at the future position by a plurality of other vehicles and includes information about an object and / or a structure located ahead on the vehicle's route with a negative plausibility check result for a plurality of vehicles.Using the received swarm data, it is checked whether a sensory detection is impaired by the object and / or structure ahead by using the information contained in the swarm data about the object and / or structure ahead to verify the plausibility of the detection.

[0011] The swarm data can be based on sensor data that was previously recorded by a large number of other vehicles traveling the same route at exactly the same positions. In contrast to the method described in DE 10 2015 219 933 A1, this is not limited to sensor data from vehicles that are close to the position of the ego vehicle at the same time, i.e. in the current local environment of the ego vehicle. Rather, the swarm data can be generated based on sensor data from other vehicles that have been recorded over a longer period of time, which can extend over several hours, days, weeks or months. This ensures that the swarm data can be used to determine whether objects orStructures in the current or upcoming vehicle environment can be reliably detected by the sensors of the ego vehicle and can be used, for example, by a driver assistance system or there is an impairment of the functionality of the sensors.

[0012] According to one embodiment of the invention, the surroundings of the vehicle are detected with one or more sensors of the vehicle, wherein sensor data for objects and / or structures in the surroundings of the vehicle are generated and a plausibility check of the detection of the surrounding object is carried out by comparing the sensor data generated by the vehicle with the swarm data.

[0013] In particular, in the event of a negative plausibility check result, the sensor data generated by the vehicle can be discarded or corrected.

[0014] In this case, it is advantageous if, in the event of a negative result of the plausibility check, the sensor data generated by the vehicle and / or information about the negative result of the plausibility check are transmitted to the central server.

[0015] Advantageously, highly accurate localization data for the position of the vehicle when driving along the route are determined by means of a digital environmental map and sensory detection of landmarks recorded in the digital environmental map.

[0016] According to one embodiment, the averaged lane of the swarm vehicles is evaluated during the plausibility check and an environmental object detected by the vehicle's sensor is declared invalid if, according to the digital map, it is located next to the lane, but the sensor detection shows that the environmental object is located on this lane.

[0017] Likewise, according to a further embodiment, the digital map is advantageously evaluated to determine whether the vehicle will shortly enter an environment for which an impairment of the sensors is known and, in this case, a warning about a possible impairment of the sensors is issued.

[0018] Furthermore, according to a further embodiment, the digital map is evaluated to determine whether it has an environmental object in the current surroundings of the vehicle which should be detected unhindered by a sensor of the vehicle and, in the event that this environmental object is nevertheless not detected, a warning is issued about a possible defect, a misalignment or contamination of the sensors.

[0019] Likewise, according to a further embodiment, for an environmental object detected by a sensor of the vehicle, which according to the digital map and / or further sensor data runs substantially parallel to the direction of travel over an extended area, but for which the sensor data has a jump, the detected environmental object is declared invalid.

[0020] The device according to the invention for detecting objects and / or structures in the environment of a vehicle comprises - at least one sensor that detects the surroundings of the vehicle; - a position determination unit for determining the position of the vehicle; - a communication unit for transmitting localization data for the determined position of the vehicle to a central server and for receiving swarm data for a future position following a route from the central server, wherein the swarm data has been generated by statistically evaluating sensor data that were recorded at an earlier point in time at the future position by a plurality of other vehicles and information about an object lying ahead on the route of the vehicle and / or a structure lying ahead on the route of the vehicle with a negative plausibility check result for a plurality of vehicles; and - an evaluation and control unit which uses the received swarm data to check whether a detection of the object and / or structure in front is impaired by the at least one sensor, by using the information contained in the swarm data about the object and / or structure in front to check the plausibility of the detection.

[0021] Preferably, at least one sensor is designed as a radar sensor.

[0022] Further features of the present invention will become apparent from the following description and claims in conjunction with the figures. Fig. 1 shows a flow diagram of a method according to the invention; and Fig. 2 schematically shows a vehicle with a device according to the invention and a central server which transmits swarm data to the vehicle based on location data of the vehicle.

[0023] To better understand the principles of the present invention, embodiments of the invention are explained in more detail below with reference to the figures. It is understood that the invention is not limited to these embodiments and that the described features may also be combined or modified without departing from the scope of the invention as defined in the claims.

[0024] Fig. Figure 1 shows a flowchart of a method according to the invention. In a first method step 1, the current position data of an ego vehicle are determined while traveling along a route, for example, using a satellite-based navigation system such as a GPS system in combination with digital map data. Particularly in conjunction with highly accurate map data that records the positions of various landmarks such as traffic lights or traffic signs, the current position of the ego vehicle can be determined very precisely by comparing it with the relative distances and positions of these landmarks measured by vehicle sensors.

[0025] The determined position is transmitted to a central server in process step 2. This server can be provided as a backend server for the ego vehicle and a multitude of other vehicles on the Internet and can be part of an IT infrastructure not further described here. Communication between the respective vehicle and the server takes place via a wireless data connection, for example, using mobile radio units provided in the vehicles.

[0026] The server then determines swarm data for the position transmitted by the ego vehicle or a position passed by the ego vehicle in the future during the course of the journey. For this purpose, the central server can access a database containing a comprehensive collection of swarm data for the road network in a larger geographical area.

[0027] This swarm data is based on parameters recorded by a large number of vehicles, the so-called vehicle swarm, at an earlier point in time while driving along the road network. For this purpose, the recorded parameters were transmitted from the sponge vehicles to the central server, which statistically evaluated them using suitable algorithms and used them to determine the swarm data.

[0028] The swarm data can contain various metadata, such as a swarm trajectory for a road being traveled, which results from the trajectories traveled by the various swarm vehicles on this road, or the road markings, guardrails, or road edges detected by the swarm vehicles when traveling on this road. Furthermore, the swarm data can also contain detected landmarks such as road signs. If necessary, time-related parameters such as the time, date, and / or day of the week when the sensor data was recorded can be taken into account during the statistical analysis of the sensor data, thus providing time-dependent swarm data. It is also possible to provide the recorded sensor data with a timestamp and to include this time stamp in the statistical analysis.For example, older sensor data can be given less weight or given an expiration date based on the timestamp in order to be able to respond better to changes in the road network when generating the swarm data.

[0029] The swarm data requested based on the position information of the ego vehicle are then sent back from the server to the ego vehicle and received by it in a process step 3. This transmission can also be carried out, in particular, via a mobile network.

[0030] Using the swarm data thus received, a process step 4 is then used to check whether the sensory detection of objects and / or structures in the vicinity of the ego vehicle is impaired at the current or future position. Such impairment can be caused, in particular, by faulty sensor signals, which can be detected by an automatic plausibility check of the sensor data received by the ego vehicle's sensors in a process step 5. Thus, using suitable algorithms, objects in the vehicle's surroundings detected by the vehicle's sensors can be compared with the swarm data for the current vehicle position.

[0031] For example, if the guardrails next to the roadway are detected with radar and / or ultrasonic sensors of the ego vehicle, the position of the guardrails determined by the ego vehicle based on the sensor data can be compared with information about the current position of the outer lane markings contained in the swarm data. This allows the relative position of the guardrail and the lane marking, as well as the distance between the guardrail and the lane marking, to be determined.

[0032] Although the detection of guardrails, especially with radar sensors, can usually be achieved without problems over large sections of a busy road, unwanted reflections of the emitted radar waves can lead to incorrect detection or assignment of the guardrails detected by the radar sensor, for example, when driving through a tunnel or over a metal bridge. Based on the detected reflections, a guardrail position can be incorrectly determined that is not adjacent to the busy roadway, but rather in the middle of that roadway.

[0033] This can be detected by comparing the sensor data with the swarm data. In this process, a check can first be carried out in step 6 to determine whether one or more objects and / or structures in the vicinity of the ego vehicle have been detected by the ego vehicle's sensors. If this is the case, a check can then be carried out in step 7 to determine whether these are plausible when compared with the swarm data. If the sensor data is plausible, it can then be used by a driver assistance system in step 8.

[0034] If, on the other hand, the sensor data is not plausible, for example because the distance between the guardrail and the lane marking changes suddenly due to incorrect object detection by the radar / ultrasound sensors in a tunnel or on a metal bridge, further action can then be taken. For example, in process step 9 it can first be checked whether the falsified data can be corrected, e.g. by shifting the position of a detected object determined on the basis of the sensor data by an offset distance known from the history. After such a correction in process step 10, the corrected data can also be used by the driver assistance system in process step 8. If, on the other hand, a correction is not possible, the data for the affected radar or ultrasonic objects is set to invalid in process step 11 and discarded.

[0035] In addition to information about the road markings, other information can also be used to verify the plausibility of the sensor data. For example, the swarm trajectory in this area can also be considered to check whether the distance between the radar object and the swarm trajectory is plausible. Likewise, information about whether the vehicle is currently in a tunnel or on a metal bridge can be obtained from the swarm data or map data and taken into account in the plausibility check.

[0036] This plausibility check significantly increases the robustness and reliability of the detected radar objects. This is particularly advantageous when an outdoor camera is also intended for detecting such objects, but this camera does not provide clear data due to unfavorable lighting conditions.

[0037] Furthermore, comparing the radar objects with the swarm data enables a significantly longer forecast compared to a comparison with information from an external camera, as the sponge data is not limited to the current field of view of the camera. This means that this forecast can also be carried out for areas that lie behind a bend that has yet to be negotiated or within a tunnel that has not yet been entered. This is particularly advantageous for automated vehicle functions that require early knowledge of the further road or lane conditions. For example, in the case of an automatic emergency stop due to a driver's health impairment, it must be ensured that the hard shoulder is wide enough to ensure that the vehicle automatically switches to the beach and then automatically stops.

[0038] Likewise, the ability to preview swarm data over a longer distance can provide early warning of potential system limitations of a driver assistance system. For example, a warning can be issued before entering a tunnel that an automatic emergency stop function is only partially available.

[0039] Furthermore, in the event of significant discrepancies between the swarm data and the sensor-based object detections, information about this can be transmitted to the central server, stored there, and then used for further functional improvements. For example, information about the guardrail in a particular tunnel being incorrectly detected by radar sensors, because the radar signals indicate it is located within the lane markings known from the swarm data, can be stored and transmitted to vehicles before entering that tunnel.

[0040] As described above, the swarm data can be transmitted using a pull model when the swarm data is required by a vehicle and is therefore requested from the server. However, the transmission can also be carried out in advance using a push model, with the swarm data subsequently being stored in the vehicle's memory as soon as new swarm data for a route section becomes available. In this case, the transmission can also be carried out to multiple vehicles simultaneously.

[0041] Instead of improving the robustness or reliability of detected radar objects through the plausibility check described above, the functionality of one or more sensors can also be checked by comparing the sensor data with the swarm data. For this purpose, the swarm data, if necessary in combination with a high-precision digital map available in the vehicle, can be evaluated to determine whether there is an environmental object in the current vicinity of the vehicle that should be detected by the vehicle's sensors. If such an environmental object is present, but no corresponding sensor data is output by a sensor in the vehicle, this may be due to a defect, misalignment, or contamination of this sensor. The driver of the vehicle can then be informed of this in a method step 12 by means of a warning message.The warning can be issued, for example, by a corresponding visual display in the instrument cluster or head-up display or as a voice output via the vehicle's loudspeakers.

[0042] In Fig. Figure 2 schematically shows a vehicle F with a device according to the invention and a central server S. The vehicle F can be, in particular, a passenger car or a truck.

[0043] The vehicle has various units for the inventive detection of objects and / or structures in the vehicle's surroundings. In particular, an evaluation and control unit AS is provided for implementing the inventive method. This unit can be configured, for example, as a control unit of the vehicle F and can have one or more microcontrollers or microprocessors.

[0044] A rough localization of the vehicle can initially be achieved using a navigation unit N based on received GPS signals. In conjunction with highly accurate map data and sensor-supported detection of landmarks in the vehicle's surroundings, the vehicle can then be located very precisely. For this purpose, the vehicle can be equipped with a vehicle camera K, which records the vehicle's surroundings in front of and, if necessary, also to the side of the vehicle. The image or video data generated by the vehicle camera can be fed via a digital data bus DB in the vehicle to the evaluation and control unit AS, which evaluates the image data in order to identify the landmarks in the vehicle's surroundings using suitable image processing methods and thus generate localization data for the vehicle's current position.

[0045] Furthermore, the vehicle has at least one radar sensor RS, which detects the vehicle's surroundings and generates sensor data RSD for objects and / or structures in the vehicle's surroundings. The sensor data RSD from the radar sensor is also fed to the evaluation and control unit AS.

[0046] The vehicle also has a mobile radio transmission unit C with which the localization data can be transmitted to the server and swarm data SD can be received.

[0047] The evaluation and control unit AS can then perform the plausibility check of the sensor data RSD described above based on the received swarm data SD. The data can be compared continuously, allowing for rapid response to erroneous sensor data at any time.

[0048] The vehicle F is also provided with a driver assistance system FA, which provides assistance functions such as an automatic emergency stop of the vehicle based on the sensor data, provided that it is recognized as valid during the plausibility check.

[0049] Additionally, the vehicle may include other components not shown. For example, a memory for storing the swarm data may be provided.

[0050] The invention can be applied in any vehicle that requires high-precision localization for integrated assistance systems or automated driving, but is not limited to this, but can also be used, for example, in mobile robots. List of reference symbols 1 - 12 process steps F vehicle RS radar sensor K Camera AS evaluation and control unit N Navigation unit FA driver assistance system C Communication unit DB data bus LD localization data SD swarm data S Server

Claims

[1] Method for detecting objects and / or structures in the environment of a vehicle, wherein - the position of the vehicle (F) is determined (1); - localization data (LD) for the determined position are transmitted to a central server (S) (2); - swarm data (SD) for a future position following a route are received (3) by the central server (S), wherein the swarm data are based on a statistical evaluation of sensor data that were recorded at an earlier point in time at the future position by a plurality of other vehicles and comprise information about an object lying ahead on the route of the vehicle and / or a structure lying ahead on the route of the vehicle with a negative plausibility check result for a plurality of vehicles; and - using the received swarm data (SD) it is checked (4) whether a sensory detection is impaired by the object and / or structure in front, by using the information contained in the swarm data about the object and / or structure in front to check the plausibility of the detection. [2] Method according to claim 1, wherein the surroundings of the vehicle are detected with one or more sensors (RS) of the vehicle and sensor data (RSD) are generated for objects and / or structures in the surroundings of the vehicle and a plausibility check of the detection of the surrounding object is carried out by comparing the sensor data (RSD) generated by the vehicle with the swarm data (SD). [3] Method according to claim 2, wherein in the case of a negative result of the plausibility check, the sensor data (RSD) generated by the vehicle are discarded or corrected. [4] Method according to claim 2 or 3, wherein in the case of a negative result of the plausibility check, the sensor data (RSD) generated by the vehicle and / or information about the negative result of the plausibility check are transmitted to the central server (S). [5] Method according to one of the preceding claims, wherein highly accurate localization data for the position of the vehicle when traveling along the route are determined by means of a digital map of the surroundings and a sensory detection of landmarks recorded in the digital map of the surroundings. [6] Method according to claim 5, wherein the plausibility check comprises evaluating the averaged lane of the swarm vehicles and declaring an environmental object detected by a sensor (RSD) of the vehicle to be invalid if, according to the digital map, it is located next to the lane, but the sensor detection shows that the environmental object is located on this lane. [7] Method according to claim 5, wherein the digital map is evaluated to determine whether the vehicle is about to enter an environment for which an impairment of the sensors is known and, in this case, a warning about a possible impairment of the sensors is issued. [8] Method according to claim 5, wherein the digital map is evaluated to determine whether it has an environmental object in the current surroundings of the vehicle which should be detected unimpaired by a sensor (RSD) of the vehicle and, in the event that this environmental object is nevertheless not detected, a warning is issued about a possible defect, a misalignment or a contamination of the sensors. [9] Method according to claim 5, wherein for an environmental object detected by a sensor (RSD) of the vehicle, which according to the digital map and / or further sensor data runs over an extended area substantially parallel to the direction of travel, but for which the sensor data has a jump, the detected environmental object is declared invalid. [10] Device for detecting objects in the vicinity of a vehicle, with - at least one sensor (S) which detects the surroundings of the vehicle (F); - a position determination unit (N) for determining the position of the vehicle; - a communication unit (C) for transmitting localization data (LD) for the determined position of the vehicle to a central server (S) and for receiving swarm data (SD) for a future position following a route from the central server, wherein the swarm data has been generated by statistically evaluating sensor data that were recorded at an earlier point in time at the future position by a plurality of other vehicles and comprise information about an object lying ahead on the vehicle's route and / or a structure lying ahead on the vehicle's route with a negative plausibility check result for a plurality of vehicles; and - an evaluation and control unit (AS) which uses the received swarm data (SD) to check whether a detection of the object and / or structure in front with the at least one sensor (S) is impaired by using the information contained in the swarm data about the object and / or structure in front to check the plausibility of the detection. [11] Device according to claim 10, wherein at least one sensor (S) is designed as a radar sensor.

Images