Method and device for detecting malfunction of environmental model of automatic driving function

Abstract

The invention relates to a method for detecting a malfunction of an environmental model used by an automatic driving function of a motor vehicle, wherein the method comprises ascertaining a first deviation between a target trajectory ascertained by the environment model and an actual trajectory traveled by the motor vehicle and / or a second deviation between a road profile ascertained by the environment model and a road profile ascertained by the camera software, and identifying a fault based on the first and / or second deviation.

Description

Technical Field

[0001] The present disclosure relates to a method for identifying a fault in an environment model of an automated driving function. Additionally or alternatively, a data processing device is provided, which is configured to at least partially perform the method. Additionally or alternatively, an automated motor vehicle is provided, which at least partially includes a data processing device. Additionally or alternatively, a computer program is provided, which includes instructions, which, when executed by a computer, cause the computer to at least partially perform the method. Additionally or alternatively, a computer-readable medium is provided, which includes instructions, which, when executed by a computer, cause the computer to at least partially perform the method. Background Art

[0002] In the case of an automated driving function of a motor vehicle, it can be disturbing for the driver of the motor vehicle if the motor vehicle incorrectly detects the road geometry or the road course and as a result the motor vehicle does not follow the road as intended, for example because the motor vehicle is misled. In this case, the driver's intervention may be necessary to take over the control of the lateral and / or longitudinal guidance of the motor vehicle. If frequent interventions are required, the driver's confidence in the automated driving function decreases.

[0003] When developing automated driving functions, therefore, special attention is paid to situations in which the motor vehicle is misled and does not follow the actual course of the road.

[0004] The test driver who detects this incorrect behavior of the motor vehicle pays attention and manually starts data recording in order to fix the event.

[0005] These data records can be analyzed by development and can determine which part of the automated driving function is responsible for the erroneous behavior. However, many manual steps are required to record the data and analyze it.

[0006] However, the cause of the erroneous behavior and which part of the automated driving function triggered it are generally not clear to the test driver.

[0007] Traditionally, the cause of the erroneous behavior is found by developers through manual visual analysis of the recorded data. In this case, the developers visually compare the road geometry calculated by the environment model with reality (trajectory or camera image).

[0008] Furthermore, it is known to use trajectories driven by a driver as training data for automated driving functions (so-called behavior cloning). Summary of the invention

[0009] Against the background of this prior art, the object of the present disclosure is to specify a device and a method which are each suitable for improving the prior art described above.

[0010] This object is achieved by the features of the independent claim. The dependent claims contain preferred developments of the invention.

[0011] The object is then achieved by a method for detecting a fault in an environment model used by an automated driving function of a motor vehicle. The method comprises determining a first deviation between a target trajectory determined by the environment model and an actual trajectory driven by the motor vehicle. Additionally or alternatively, the method comprises determining a second deviation between a road course determined by the environment model and a road course determined by camera software. The method also comprises detecting a fault based on the first and / or second deviation.

[0012] The method described above may be a computer-implemented method, ie, one, multiple or all steps of the method may be executed by a computer or a data processing device.

[0013] An automated driving function may be understood to be a function of a motor vehicle which is designed to at least partially take over parts of the driving task or the entire driving task, if necessary together with other automated driving functions.

[0014] The environment model may be a so-called road environment model. The environment model may be determined based on sensor data from a plurality of, in particular different, sensors in the motor vehicle. For this purpose, the sensor data may be combined. The road environment model may include information about the course of the road and the target trajectory of the motor vehicle.

[0015] A trajectory can be understood as the path along which the motor vehicle should travel (so-called target trajectory) or the path actually traveled (so-called actual trajectory). In addition to the position information, the trajectory can also have a time component, ie when the motor vehicle is or where it should be.

[0016] The two possibilities described above for ascertaining the deviation have the common inventive concept of automatically detecting and, if necessary, subsequently analyzing faults in the automated driving function in the area of ​​the (road) environment model.

[0017] Therefore, the explanations concerning the two possibilities can be combined with one another and apply equally to both possibilities, as far as technically feasible.

[0018] In addition to reducing costs and accelerating the development of automated driving functions, the method also provides a possibility for improving the quality of automated driving functions by automating manual processes and data-driven development.

[0019] Possible refinements of the above-described method are described in detail below.

[0020] Determining a first deviation between a target trajectory determined by the environment model and an actual trajectory traveled by the motor vehicle may include determining a position and / or a curvature of a center line of the roadway as the target trajectory using the environment model, and determining the first deviation based on a deviation between the position and / or curvature of the center line and the position and / or curvature of the actual trajectory.

[0021] Determining a second deviation between the road course determined by the environmental model and the road course determined by the camera software may include determining the position and / or curvature of lane markings and / or a center line of the lane as the road course with the aid of the environmental model, determining the position and / or curvature of lane markings and / or a center line of the lane as the road course with the aid of the camera software, and determining the second deviation based on a deviation between the position and / or curvature of the lane markings and / or the center line determined with the aid of the environmental model and the position and / or curvature of the lane markings and / or the center line determined with the aid of the camera software.

[0022] The method includes determining that a predetermined ambient condition exists and recognizing that no fault is present despite a second deviation.

[0023] The ascertainment of the first and / or second deviation can take place during driving in the motor vehicle (ie online) and / or after driving by means of a data processing device external to the motor vehicle (ie offline).

[0024] If the first and / or second deviation is ascertained during driving, ie online, the data used by the automated driving function can be stored in a ring memory.

[0025] If a fault is detected based on the first deviation and / or the second deviation, the data stored in the ring memory can be sent from the motor vehicle to a data processing device external to the motor vehicle.

[0026] If the first deviation and / or the second deviation exceeds a corresponding predetermined limit value, a fault may be detected.

[0027] The method described above may be summarized in other words and with reference to more specific configurations as described below without limitation to the present disclosure.

[0028] Manual steps should be avoided when analyzing and evaluating the quality of the road geometry calculated by the automated driving function. The quality evaluation can be performed both online and offline. Online quality evaluation (self-monitoring) can be used to automatically identify deviations during driving. This makes it possible to automatically identify faults, request the driver to take over and / or automatically record data for offline analysis. Since the detection of problematic situations is automatic, data can be recorded not only by the test driver with the aid of special equipment, but also in the customer vehicle. The increased number of vehicles for which data can be recorded makes it easier to record rarely occurring events or faults. The recorded data can be automatically or automatically analyzed and / or recorded in a training data set.

[0029] The disclosure provides for a plurality of measures for automatically detecting and analyzing erroneous behaviors of an automated driving function in the context of a road environment model.

[0030] In one sub-aspect, the present disclosure determines a deviation between a road course determined by an automated driving function (road environment model) and an actual driving trajectory.

[0031] For example, a lateral offset between a center line calculated by the environment model and the actual driving trajectory can be used as the deviation. Alternatively or additionally, the deviation can also be calculated based on the steering angle.

[0032] The deviation can be determined both online and offline in the vehicle.

[0033] The environment model may have multiple sub-models (lane merging, crowd trajectories, high-definition maps). The present invention may be used to evaluate a single model.

[0034] When the deviation is determined online in the vehicle, the data can be automatically recorded for further analysis if the deviation is too large.

[0035] Data recording can also be performed when other critical values ​​are detected, such as excessive curvature, particularly small or large lane widths, very short or unstable detection of lane markings by the camera, etc.

[0036] In another sub-aspect of the present disclosure, the above description of the first sub-aspect applies mutatis mutandis, i.e. can be combined with this, and the geometry calculated by the road environment model (i.e. the output of the road environment model) can be compared with the detected geometry of the lane markings and / or estimated center line provided by the camera software (i.e. the input of the road environment model).

[0037] When there is a deviation, at least two reasons can be derived.

[0038] On the one hand, the current recognition of the camera may be wrong, for example the camera confuses an asphalt joint or a shadow edge with a lane marking and / or incorrectly detects the geometry (especially the curvature) of the lane marking. Therefore, the recognition can be set up in the automated driving function to recognize certain special situations (i.e. predetermined scenarios) in which it is known that camera false detections often occur. In this case, the current camera detection can be discarded and not taken into account when calculating the road environment model. In this case, the deviation may be consciously desired.

[0039] On the other hand, although the input data is correct, a deviation may also be formed due to incorrect processing in the environment model. This may be an undesirable deviation indicating a possible processing error of the environment model.

[0040] The special circumstances (scenarios) found can be used as criteria for finding the causes of deviations.

[0041] Another indication of an erroneous behavior of the road environment model may be by the driver taking over control (so-called driver takeover). The driver may be asked to take over control of the vehicle, for example when self-monitoring determines an error or the driver may take over control consciously, for example when the driver determines an erroneous behavior.

[0042] The driver taking over can be used as a trigger for the data recording. However, the driver taking over can also, ie, additionally or alternatively, be used as a criterion in the analysis of the causes of the vehicle's incorrect behavior.

[0043] If the geometry of the road environment model is consistent with the camera detection at the time the driver takes over, this may indicate an error in the camera software, but if there is a deviation, this may indicate an error in the road environment model.

[0044] Data logging can utilize a circular buffer so that when logging begins, the period preceding the event or fault is also logged. This enables analysis of the onset of a deviation.

[0045] The recorded data can be transmitted from the motor vehicle, for example from a customer vehicle, to a cloud (for example of a vehicle manufacturer). As a result, statistical data, in particular global statistical data, can be obtained for anomalies from all customer vehicles and / or predetermined customer vehicles.

[0046] Therefore, the method can be used for error analysis. The statistical analysis and evaluation can focus on the most common error images.

[0047] Automatic logging can be used to identify geographic hotspots of erroneous behavior.

[0048] This approach can be used to identify faulty components of automated driving functions (e.g., camera software, lane marking merging, localization, crowd trajectory maps, high-definition maps, etc.).

[0049] Object and / or obstacle information may also be included in order to determine the reasons for the detour or deviation from the trajectory.

[0050] The recorded data may be automatically received into a training database for the automated driving function and / or may be provided to a supplier of the automated driving function and / or camera software to improve the automated driving function or the camera software.

[0051] In addition, a data processing apparatus is provided, which includes a device for executing the above-described method. The data processing apparatus may also be referred to as a data processing system or a data processing apparatus.

[0052] The data processing device may be designed to be installed at least partially in and / or on board an automated vehicle and / or may be at least partially part of a cloud.

[0053] The data processing device may be part of a driver assistance system or an automatic driving function or may constitute and / or execute this part. The data processing device may have an electronic control unit (ECU in English). The data processing device may have an intelligent processor-controlled unit, which may communicate with other modules, for example, via a central gateway (CGW) and may form an on-board network, for example, with a telematics controller, via a field bus such as a CAN bus, a LIN bus, a MOST bus and a FlexRay or via automotive Ethernet, if necessary. It is conceivable that the controller controls functions related to the driving behavior of the motor vehicle, such as engine control, power transmission, a braking system and / or a tire pressure monitoring system. In addition, driver assistance systems, such as parking assistance, adaptive cruise control (ACC), lane keeping assistance, lane change assistance, traffic sign recognition, light signal recognition, start assistance, night vision assistance, emergency brake assistance and / or intersection assistance can be controlled by the controller.

[0054] It is conceivable that the data processing device controls the automated driving of the automated motor vehicle at least partially and / or temporarily based on the output of the environment model.

[0055] What was described above with respect to the method also applies analogously to the data processing device and vice versa.

[0056] Furthermore, an automated motor vehicle may be provided which has a data processing device as described above.

[0057] The motor vehicle may be a passenger vehicle, in particular a car.

[0058] The automated motor vehicle may be configured to take over longitudinal guidance and / or transverse guidance at least partially and / or at least temporarily by means of an automated driving function during automated driving of the motor vehicle. The automated driving may be controlled at least partially and / or temporarily by means of an automated driving function by means of a data processing device.

[0059] The automated driving can be performed in such a way that the forward movement of the motor vehicle is performed (largely) autonomously.

[0060] The motor vehicle may be a motor vehicle of level 0 autonomy, ie the driver takes over the task of dynamic driving, even if assisting systems (eg ABS or ESP) are present.

[0061] The motor vehicle may be a Level 1 autonomy motor vehicle, ie having certain driver assistance systems that assist the driver in operating the vehicle, such as an adaptive cruise control (ACC).

[0062] The motor vehicle may be a motor vehicle with a Level 2 autonomy level, ie may be partially automated, such that functions such as automatic parking, lane keeping or lateral guidance, general longitudinal guidance, acceleration and / or deceleration are taken over by a driver assistance system.

[0063] The motor vehicle may be a motor vehicle with a Level 3 autonomy level, i.e., may be conditionally automated so that the driver does not have to constantly monitor the vehicle systems. The motor vehicle autonomously performs functions such as triggering a turn signal, lane change and / or lane keeping. The driver may turn to other things, but if necessary, the system requests to take over the guidance within the warning period.

[0064] The motor vehicle may be a motor vehicle with a level 4 autonomy, ie may be highly automated, so that the control of the vehicle is continuously taken over by the vehicle system. If the system no longer has control over the driving task, the driver may be asked to take over the control.

[0065] The motor vehicle may be a motor vehicle with a Level 5 autonomy level, ie may be fully automated, so that no driver is required to perform the driving task. No human intervention is required except for determining the target and turning on the system.

[0066] What has been described above with respect to the method and the data processing device also applies analogously to the motor vehicle and vice versa.

[0067] Furthermore, a computer program is provided. The computer program is characterized in that it comprises instructions which, when the program is executed by a computer, cause the computer to at least partially carry out the described method.

[0068] The program code of the computer program may be present as any desired code, in particular a code suitable for motor vehicle control.

[0069] Furthermore, a computer-readable medium, in particular a computer-readable storage medium, is provided. The computer-readable medium is characterized in that it comprises instructions which, when the program is executed by a computer, cause the computer to at least partially execute the method described above.

[0070] That is, a computer-readable medium including the computer program defined above can be provided. The computer-readable medium can be any digital data storage, such as a USB flash drive, a hard drive, a CD-ROM, an SD card or an SSD card. The computer program does not necessarily have to be stored on such a computer-readable storage medium in order to be available to the motor vehicle, but can also be obtained from the outside via the Internet or other means.

[0071] What was described above with respect to the method, the data processing device and the automated vehicle also applies analogously to the computer program and the computer-readable medium, and vice versa. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Refer to the following Figures 1 to 3 An embodiment is described.

[0073] Figure 1 A flow chart schematically shows a method for detecting a fault in an environment model used by an automated driving function of a motor vehicle,

[0074] Figure 2 A first top view of a road surface is schematically shown for explaining the ascertainment of a first deviation between a target trajectory ascertained by an environment model and an actual trajectory traveled by the motor vehicle, and

[0075] Figure 3 A second top view of the roadway plane is schematically shown in order to explain the ascertainment of a first deviation between a target trajectory ascertained by the environment model and an actual trajectory traveled by the motor vehicle. DETAILED DESCRIPTION

[0076] As from Figure 1 It turns out that the method for detecting a fault in an environment model used by an automated driving function of a motor vehicle essentially has two steps S1 and S2 .

[0077] In a first step of the method, a first deviation between a target trajectory determined by the environment model and an actual trajectory traveled by the motor vehicle is determined. Additionally or alternatively, a second deviation between the road course determined by the environment model and the road course determined by the camera software is determined. Figures 2 to 3 Some possibilities of how it can be implemented are described in detail, wherein these possibilities do not constitute a final description of the implementation possibilities but are only used for illustrative purposes.

[0078] Ascertaining a first deviation between a target trajectory ascertained from the environment model and an actual trajectory traveled by the motor vehicle may include determining a position of a center line of the roadway as the target trajectory using the environment model and determining the first deviation based on a deviation between the position of the center line and the position of the actual trajectory.

[0079] This is for example the case for a global coordinate system in Figure 2 The driving trajectory or actual trajectory 1 (which is obtained by the odometer of the motor vehicle and Figure 2 The odometer data, in particular the position information and the center lines t1, t2 due to the time interval, are input into the global coordinate system. Figure 2 In FIG. 1 , this is shown, for example, for two different time points t1 and t2. Subsequently, at different time points, the corresponding deviation between the real trajectory or actual trajectory 1 and the calculated center line t1, t2 of the environment model is determined at the position or location of the motor vehicle at the time of the determination of the center line. The deviation can be the vertical distance between the actual trajectory 1 and the calculated center line t1, t2, or Figure 2 As shown, the shortest distance d1 or d2 between the actual trajectory 1 and the calculated center line t1, t2. The distance d1, d2 between the actual trajectory and the calculated center line can be included in the calculation or determination of the first deviation at least. Additionally or alternatively, the curvature of the center line and the curvature of the actual trajectory can be obtained at the corresponding position of the motor vehicle, and the difference between these curvatures can be included in the calculation or determination of the first deviation at least. For this purpose, the steering angle of the motor vehicle can be used, that is, the corresponding curvature of the center line t1, t2 at different positions can be obtained and the corresponding curvature can be obtained with the help of the actual steering angle of the motor vehicle at the corresponding position. It is also conceivable to pre-process the center line of the environment model, for example, to create a unique global center line from the center lines of multiple time steps or time points t1, t2 and to obtain, for example, the deviation of the maximum distance between the actual trajectory and the global center line.

[0080] like Figure 3 As shown, the coordinate system of the motor vehicle, ie the relative coordinate system, can also be used for determining the first deviation. Figure 2As described, the current position 2 or actual position of the motor vehicle can be compared with the current target position of the motor vehicle on the center line t1 and the distance d1 can be determined from this. This can be done continuously. If the first deviation is determined online during driving and can be done continuously, it may be particularly suitable to determine the first deviation via a relative coordinate system. The center line t1 used for this purpose can be determined in the same time step or in an earlier time step. That is, the current position 2 of the vehicle can be compared with the target position that has been determined by the environment model at a previous time point (for example t=0). Similarly to the above method, the curvature of the center line t1 can be compared with the current steering angle.

[0081] It is also conceivable that the center lines t1 and t2 are determined not only by the environment model but also by the camera software. Then, the position and / or curvature of the center lines t1 and t2 can be determined not only by the camera software but also by the environment model, and the deviation between the position and / or curvature of the center lines can be obtained. The deviation between these center lines, such as the lateral offset of the center line (see above), can be at least included in the calculation or determination of the second deviation. It is also conceivable that the lane marking is used in this way in addition or alternatively to the center line. Then, the position and / or curvature of the lane marking can be determined not only by the camera software but also by the environment model, and the deviation between the position and / or curvature of the lane marking can be obtained. The deviation between these lane markings, such as the lateral offset of the lane marking (see above), can be at least included in the calculation or determination of the second deviation.

[0082] In a second step S2 of the method, a fault is identified based on the first deviation and / or the second deviation, wherein the fault is identified when the first and / or second deviation exceeds the corresponding predetermined limit value. It can be considered that if it is determined that a predetermined environmental condition exists, the second deviation is not taken into account. That is, the recognition can be set in the automatic driving function to recognize a predetermined environmental condition or a special condition (i.e. a predetermined scene) in which it is known that camera false detections frequently occur. In this case, the current camera detection can be discarded and not taken into account when calculating the environmental model. In this case, the deviation may be consciously desired. Therefore, in this case, it can be recognized that there is no fault even if the second deviation exceeds the limit value.

[0083] The above-described determination of the first deviation and / or the second deviation can be performed during driving in the motor vehicle and / or after driving with the aid of a data processing device outside the motor vehicle. The same applies to the above-described fault identification based on the first and / or second deviation. When one or both of these steps occur during driving, the data used by the autonomous driving function can be stored in a ring buffer. When a fault is identified based on the first and / or second deviation, the data stored in the ring buffer can be sent from the motor vehicle to a data processing device outside the motor vehicle. Additionally or alternatively, data can be recorded during the entire driving period or during part of the driving period with the aid of a recording device, in particular a data log, regardless of whether a fault is identified, and / or the data can be sent to a data processing device outside the motor vehicle, in particular streamed to a backend or a cloud.

[0084] Reference numerals list

[0085] 1 Actual trajectory

[0086] 2 Current location of the vehicle

[0087] Target trajectory at different time points t1 and t2

[0088] d1, d2 are the distances between the actual trajectory and the corresponding target trajectory at different time points

[0089] Steps of S1 and S2 methods

Claims

1. A method for identifying a fault in an environment model used by an automated driving function of a motor vehicle, the method comprising: - determining a first deviation between a target trajectory (t1, t2) determined by the environment model and an actual trajectory (1) traveled by the motor vehicle and / or a second deviation between a road course determined by the environment model and a road course determined by the camera software, and - identifying the fault based on the first deviation and / or the second deviation.

2. The method according to claim 1, wherein determining the first deviation between the target trajectory (t1, t2) determined by the environment model and the actual trajectory (1) traveled by the motor vehicle comprises: - determining the position and / or curvature of the center line of the roadway as a target trajectory (t1, t2) with the aid of the environment model, and - determining said first deviation based on a deviation between said position and / or said curvature of said center line and a position and / or a curvature of said actual trajectory (1).

3. The method according to claim 1 or 2, wherein determining a second deviation between the road course determined by the environment model and the road course determined by the camera software comprises: - determining the position and / or curvature of lane markings and / or a center line of the lane as the road course using the environment model, - determining the position and / or curvature of the lane markings and / or the center line of the lane as the road course by means of the camera software, and - determining the second deviation based on a deviation between the position and / or curvature of the lane marking and / or the center line determined by means of the environment model and the position and / or curvature of the lane marking and / or the center line determined by means of the camera software.

4. The method according to any one of claims 1 to 3, wherein the method comprises: - verify that the predetermined environmental conditions exist, and - Recognition: Despite the presence of the second deviation, no fault exists. 5 . The method according to claim 1 , wherein the first deviation and / or the second deviation is determined during driving in the motor vehicle and / or after the driving by means of a data processing device external to the motor vehicle. 6 . The method according to claim 5 , wherein the data used by the automated driving function are stored in a ring memory when the first deviation and / or the second deviation is determined during the driving process. 7 . The method according to claim 6 , wherein the data stored in the ring memory are sent from the motor vehicle to the data processing device outside the motor vehicle when the fault is detected based on the first deviation and / or the second deviation. 8 . The method according to claim 1 , wherein the fault is detected when the first deviation and / or the second deviation exceeds a respective predetermined limit value.

9. An apparatus for data processing, comprising means for executing the method according to any one of claims 1 to 8.

10. A computer program and / or a computer readable medium, comprising instructions, respectively, which, when the program or the instructions are executed by a computer, cause the computer to perform the method according to any one of claims 1 to 8.

Images