Method and system for adjusting a vehicle's perception of its surroundings

Abstract

The present invention relates to a method for adjusting the environmental perception of an ego-vehicle, wherein the ego-vehicle comprises means for V2X communication, wherein the ego-vehicle is controllable or controlled by an ADAS and / or an ADS, wherein the ego-vehicle has a plurality of sensors observing its environment and a storage medium, wherein data (11) measured by the sensors are interpreted by the ADAS / ADS for the execution of autonomous driving operations, wherein the following steps are performed: storing at least one driving scenario for which the ADAS / ADS can or does perform control of the ego-vehicle; storing sensor data measured during a journey in the storage medium; storing V2X data (12) transmitted during the journey in the storage medium; identifying (13) the at least one driving scenario in the data measured during the journey;Comparison (14) of the sensor data originating from sensors specified according to the at least one driving scenario with V2X data transmitted during the identified driving scenario, which are considered to be true data; use of any deviation detected during the comparison to adjust (16) the respective sensors and / or adjust the interpretation of the measured data by the ADAS / ADS. Furthermore, a system (10) for adjusting a vehicle's environmental perception is presented.

Description

[0001] The present invention relates to a method for adjusting the perception of the surroundings for ADAS / ADS of a vehicle. Furthermore, a system for adjusting the perception of the surroundings in ADAS / ADS is presented.

[0002] In "vehicle-to-X" communication, which connects an ego vehicle with an external communication participant, standardized, wireless interfaces with technical systems in the vehicle's environment are used to exchange information. This applies, for example, to communication between vehicles, known as "vehicle-to-vehicle" communication, in which vehicles report their braking decelerations to each other to prevent rear-end collisions, or report a planned lane change to other vehicles to enable merging. However, communication with installed infrastructure is also possible; for example, traffic lights can report the duration until the next green phase to the vehicle, or surveillance cameras installed at intersections can report detected objects to vehicles for trajectory planning.

[0003] Regardless, the ego vehicle requires its own sensors, which provide measurement data for assessing a given driving situation. On the other hand, "vehicle-to-X" communication in vehicles surrounding the ego vehicle collects information relevant to the assessment of a given driving situation. Therefore, it would be particularly useful for an ego vehicle that is at least partially controlled by a driver assistance system (ADAS) and / or an automated driving system (ADS) to process such information.

[0004] The document DE 10 2018 115 198 A1 discloses a virtual crowdsourcing sensor generator, in which sensor data from several different motor vehicles are processed and sensor data for a target vehicle is generated from this.

[0005] The document US 11,198,443 B2 describes a V2X communication for exchanging sensor information from different motor vehicles.

[0006] The document DE 10 2020 205 988 A1 discusses a method for coordinating a maneuver of a motor vehicle using V2X communication.

[0007] Against this background, it is an object of the present invention to present a method for comparing data transmitted through communication with a road user surrounding the ego vehicle and data acquired by the vehicle's sensors, particularly when the self-acquired data is used by an ADAS / ADS. Furthermore, a system on which the method is implemented is to be presented.

[0008] To solve the aforementioned problem, a method for adjusting the environment perception of an ego vehicle is proposed, in which the ego vehicle comprises means for V2X communication, referred to as V2X in vehicle-to-X communication. In V2X communication, V is the ego vehicle and X is a target vehicle or an object of traffic infrastructure that has means for V2X communication. The ego vehicle is controllable by or is controlled by an ADAS, referred to as ADAS in a driver assistance system, and / or an ADS, referred to as ADS in an automated driving system. The ego vehicle has a plurality of sensors monitoring its environment and a storage medium. The ADAS / ADS interprets data measured by the sensors to execute autonomous driving processes. The following steps are performed: • Storing at least one driving scenario for which control of the ego vehicle can be carried out or is carried out using the ADAS / ADS; • Storing data measured by the sensors during a journey in the storage medium; • Storing V2X data transmitted during the journey in the storage medium; • Identifying at least one driving scenario in the measured data during the drive; • Comparison of the sensor data originating from sensors specified in accordance with at least one driving scenario with V2X data transmitted during the identified driving scenario, whereby the V2X data are considered as true – i.e. error-free – data, • Use of a deviation identified during the comparison to adjust the respective sensors and / or adjust the interpretation of the measured data by the ADAS / ADS.

[0009] The method according to the invention thus advantageously utilizes both data from the vehicle's own sensors and data from V2X communication to identify driving scenarios relevant for autonomous control by ADAS / ADS. Within these driving scenarios, selected data from the vehicle's own sensors are compared with the data from V2X communication.

[0010] In one embodiment of the method according to the invention, at least one sensor observing the environment is selected from the following list: radar, lidar, camera.

[0011] In one embodiment, an automated driving system designed as “Adaptive Cruise Control,” abbreviated to ACC, detects a “braking target vehicle” driving scenario in which a vehicle directly ahead reduces its speed. The speed of the target vehicle is determined by merging radar and camera data. At the same time, the target vehicle reports its actual speed via V2V communication to the ego vehicle, whose ADAS / ADS then compares it with the speed of the target vehicle previously determined by itself. For certain data, such as the transmitted speed of the target vehicle, a several times higher accuracy can be assumed, e.g. 1% instead of 10%. A deviation between the two speeds can therefore be used to calculate, for example,an uncertainty in the speed determination based on the vehicle's own sensors must be carried out, whereby in the event of a possibly increased uncertainty value, the ego vehicle must be braked earlier than was previously the case.

[0012] Another example is the identification of a driving scenario for a turning intention and lateral acceleration of the target vehicle, which can be compared via a reported steering angle and a measured lateral acceleration of the target vehicle.

[0013] Another example is the sensor-based perception of the respective position of other road users, which is compared with the positions reported back via V2X communication from an infrastructure camera at, for example, intersections. The respective relative positions are determined by offsetting them against the position of the ego vehicle perceived by the infrastructure camera.

[0014] It is also conceivable to align a sensor-based perception of speed limits or traffic light displays with the respective information transmitted via V2X information with the respective infrastructure, such as traffic lights or intelligent guide posts.

[0015] Another example is a sensor-based perception of parking spaces and lane boundaries in a parking garage, which can be compared with information from a perception sensor system in the parking garage.

[0016] In a further embodiment of the method according to the invention, based on the detected deviation, the field of view of a sensor monitoring the surroundings is changed and / or the ADAS / ADS is re-parameterized and / or re-applied, and / or a performance report is created for the ADAS / ADS. The performance report or capability report of driver assistance systems can also include a summary of the behavior of the vehicle's own sensors in various driving scenarios.

[0017] In yet another embodiment of the method according to the invention, the detected deviation is used to expand training data for machine learning methods - which also includes AI-based perception software in the field of artificial intelligence.

[0018] In a further embodiment of the method according to the invention, sensor data and V2X data stored for a duration of the identified driving scenario are transmitted to a server outside the vehicle. Since only data relating to already identified driving scenarios are transmitted, the amount of data transmitted can advantageously be kept small compared to the total amount of recordable data.

[0019] Furthermore, a system for adjusting the environment perception of an ego vehicle is claimed, wherein the system comprises an ego vehicle with means for V2X communication, with vehicle-to-X referred to as V2X, and V as the ego vehicle and X as a target vehicle or an object of traffic infrastructure, an ADAS, which refers to a driver assistance system, and / or an ADS, which refers to an automated driving system. The ego vehicle is controllable by the ADAS / ADS. The ego vehicle has a plurality of sensors monitoring an environment and a storage medium. The ADAS / ADS is configured to interpret data measured by the sensors to execute autonomous driving operations. The system is designed to perform the following steps: • Storing at least one driving scenario for which control of the ego vehicle can be carried out or is carried out by the ADAS / ADS; • Storing data measured by the sensors during a journey in the storage medium; • Storing V2X data transmitted during the journey in the storage medium; • Identifying at least one driving scenario in the measured data during the drive; • comparing the sensor data originating from sensors specified according to the at least one driving scenario with V2X data transmitted during the identified driving scenario, wherein the V2X data represents true data; • Use of a deviation identified during the comparison to adjust the respective sensors and / or adjust the interpretation of the measured data by the ADAS / ADS.

[0020] In one embodiment of the system according to the invention, at least one sensor observing the environment is selected from the following list: radar, lidar, camera.

[0021] In a further embodiment of the system according to the invention, the system is designed to change a field of view of a sensor monitoring the environment based on the detected deviation and / or to re-parameterize and / or apply the ADAS / ADS based on the detected deviation and / or to create a performance report for the ADAS / ADS.

[0022] In yet another embodiment of the system according to the invention, the system is designed to provide extended training data for machine learning methods by means of the detected deviation.

[0023] In a further embodiment of the system according to the invention, the system is designed to transmit stored sensor data and V2X data to a server outside the vehicle for a period of time of the identified driving scenario.

[0024] Since validation of on-board sensors and perception software implemented in the ego vehicle represents a major challenge in the release of the systems due to functional inadequacies of ADAS / ADS, the system according to the invention can support a validation process and make it more efficient.

[0025] Further advantages and embodiments of the invention will become apparent from the description and the accompanying drawings.

[0026] It is understood that the features mentioned above and those to be explained below can be used not only in the combination specified in each case, but also in other combinations or on their own, without departing from the scope of the present invention.

[0027] Fig. 1 shows schematically an embodiment of the system according to the invention.

[0028] In Fig.1 schematically shows an embodiment of the system 10 according to the invention for adjusting the environment perception of an ego vehicle. Within the ADAS / ADS, a scenario identification 13 is performed using measured data 11 from sensors of the ego vehicle monitoring the environment, and a deviation is determined using V2X data 12 received from vehicle-to-X communication during an identified driving scenario during a comparison 14. The deviation determined during the comparison 14 is used, on the one hand, to create a performance report 15 for the ADAS / ADS and to provide an additional training data set for parameterizing or applying the ADAS / ADS, and, on the other hand, to adjust the sensors of the ego vehicle 16 and / or to adjust their field of view. List of reference symbols 10 System scheme for adjusting environmental perception 11 Data from ego vehicle with sensors for environmental monitoring 12 Data from V2X communication 13 Scenario identification 14 Comparison of data 15 Performance report, set from new training data 16 Sensor adjustment, field of view change QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2018 115 198 A1

[0004] US 11,198,443 B2

[0005] DE 10 2020 205 988 A1

[0006]

Claims

[1] A method for adjusting an environment perception of an ego vehicle, in which the ego vehicle comprises means for V2X communication, referred to as V2X in vehicle-to-X, wherein in the V2X communication V is the ego vehicle and X is a target vehicle or an object of a traffic infrastructure, which has means for V2X communication, in which the ego vehicle is controllable or is controlled by an ADAS, referred to as ADAS in a driver assistance system, and / or an ADS, referred to as ADS in an automated driving system, in which the ego vehicle has a plurality of sensors observing an environment and a storage medium, in which data (11) measured by the sensors is interpreted by the ADAS / ADS to carry out autonomous driving processes, in which the following steps are carried out: • Storing at least one driving scenario for which control of the ego vehicle can be carried out or is carried out by the ADAS / ADS; • Storing data (11) measured by the sensors during a journey in the storage medium; • Storing V2X data (12) transmitted during the journey in the storage medium; • Identifying (13) the at least one driving scenario in the measured data (11) during the journey; • comparing (14) the sensor data originating from sensors specified according to the at least one driving scenario with V2X data (12) transmitted during the identified driving scenario, wherein the V2X data (12) are considered as true data; • Use of a deviation determined during the comparison to adjust (16) the respective sensors and / or adjust the interpretation of the measured data by the ADAS / ADS. [2] Method according to claim 1, wherein at least one sensor observing the environment is selected from the following list: radar, lidar, camera. [3] Method according to one of the preceding claims, in which, on the basis of the detected deviation, a field of view of a sensor observing the surroundings is changed and / or the ADAS / ADS is re-parameterized and / or applied and / or a performance report (15) is created for the ADAS / ADS. [4] Method according to one of the preceding claims, in which training data for machine learning methods are extended by means of the detected deviation. [5] Method according to one of the preceding claims, in which data (11) and V2X data (12) stored for a period of time of the identified driving scenario are transmitted to a server outside the vehicle. [6] System for adjusting an environment perception of an ego vehicle, wherein the system (10) comprises an ego vehicle with means for V2X communication, with vehicle-to-X referred to as V2X and V as ego vehicle and X as a target vehicle or an object of a traffic infrastructure, an ADAS, which refers to a driver assistance system, and / or an ADS, which refers to an automated driving system, wherein the ego vehicle is controllable by the ADAS / ADS, wherein the ego vehicle has a plurality of sensors observing an environment and a storage medium, wherein the ADAS / ADS is configured to interpret data (11) measured by the sensors for carrying out autonomous driving operations, wherein the system (10) is designed to carry out the following steps: • Storing at least one driving scenario for which control of the ego vehicle can be carried out or is carried out by the ADAS / ADS; • Storing data measured by the sensors during a journey in the storage medium; • Storing V2X data (12) transmitted during the journey in the storage medium; • Identifying (13) the at least one driving scenario in the measured data (11) during the journey; • comparing (14) the sensor data originating from sensors specified according to the at least one driving scenario with V2X data (12) transmitted during the identified driving scenario, wherein the V2X data (12) represent true data; • Use of a deviation determined during the comparison to adjust (16) the respective sensors and / or adjust the interpretation of the measured data by the ADAS / ADS. [7] System according to claim 6, wherein at least one sensor observing the environment is selected from the following list: radar, lidar, camera. [8] System according to one of claims 6 or 7, wherein the system is designed to change a field of view of a sensor observing the environment on the basis of the detected deviation and / or to re-parameterize and / or apply the ADAS / ADS on the basis of the detected deviation and / or to create a performance report (15) for the ADAS / ADS. [9] System according to one of claims 6 to 8, wherein the system is designed to provide extended training data for machine learning methods by means of the detected deviation. [10] System according to one of claims 6 to 9, wherein the system is designed to transmit stored data (11) and V2X data (12) for a period of the identified driving scenario to a server outside the vehicle.

Images