Computer-implemented method and system for monitoring trajectory suggestions for an automated operating mode of a vehicle

Abstract

A computer-implemented method and system are proposed that enable meaningful verification and evaluation of trajectories (21) proposed by a planning module (20) for an automated operating mode of a vehicle. At least the following procedural steps are carried out: • Aggregating situation-specific information, • Generating an environment model based on situation-specific information (analysis step 11), • Determining possible vehicle behaviors (31 to 34) based on situation-specific information and the environment model, where each behavior (31 to 34) is described by a set of boundary conditions (analysis step 12), and • Prioritizing the possible behaviors (31 to 34) of the vehicle based on the boundary conditions in conjunction with a predefined set of rules (analysis step 13). According to the invention, a compatibility check is performed for a proposed trajectory (21) and at least one of the identified behaviors (31, 32) by verifying whether the proposed trajectory (21) satisfies the boundary conditions of the respective behavior (31, 32). The proposed trajectory (21) is then evaluated based on the results of these compatibility checks and the prioritizations of the respective behaviors (31, 32).

Description

State of the art

[0001] The invention relates to a computer-implemented method and a computer-implemented system for monitoring trajectories proposed by at least one planning module for an automated operating mode of a vehicle.

[0002] Such methods and systems are used in the context of behavior planning for partially or fully automated vehicles.

[0003] Automated vehicles must also operate in complex situations that can change very quickly and unexpectedly. Appropriate behavior planning requires methods and systems that meet very high safety standards. Therefore, systems with a multi-path architecture are frequently used in practice. A first path, the so-called performance path, includes at least one planning module for generating trajectories, each describing a possible behavior of the automated vehicle for the given situation, which can then be implemented by the vehicle's actuators. These trajectories can be generated using rules. However, planning modules that employ artificial intelligence (AI) methods for trajectory planning are also frequently used.To ensure that a trajectory proposal from the planning module meets the requirements of the given situation and is also safe, this proposal is reviewed and evaluated in a second path, the so-called safety path, by an independent component. This component is also referred to as the 'Safety Monitor'.

[0004] The verification of trajectory proposals, particularly with regard to their functional safety, as well as a meaningful and informative evaluation of trajectory proposals, present particular challenges.

[0005] The method according to the invention first analyzes the given traffic situation and performs the following process steps for this purpose: • Aggregating situation-specific information, • Generating an environmental model based on situation-specific information, • Determining possible vehicle behaviors based on situation-specific information and the environmental model, where each behavior is described by a set of boundary conditions, and • Prioritizing the possible behaviors of the vehicle based on the boundary conditions in conjunction with a predefined set of rules.

[0006] For this analysis of the given situation, the so-called SOCA method, described in the publication, can be used, for example. M. Butz et al., “SOCA: Domain Analysis for Highly Automated Driving Systems,” 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020, pp. 1-6, doi: 10.1109 / ITSC45102.2020.9294438., presents the SOCA method. Using this method, traffic situations are analyzed with the aim of determining boundary conditions or requirements for the behavior of an automated vehicle, hereinafter referred to as an EGO vehicle, in the respective traffic situation. For this purpose, an abstract description of the traffic situation to be analyzed is first generated. This description uses so-called zone graphs. A zone graph abstracts the traffic situation to be analyzed by representing the real road situation with a corresponding abstract representation.

[0007] A traffic infrastructure element (static road geometry) is represented by different zones that are relevant for realizing the driving intention of the EGO vehicle, but which are initially neither specified in terms of their size nor their position. The different zones can represent different map areas, possible traffic flows, objects, etc. Based on this abstract description of the traffic situation, the possible developments or behavior of the road users involved are then determined and morphologically analyzed in order to establish boundary conditions for the behavior of the EGO vehicle in the analyzed traffic situation. It is noteworthy that the results or boundary conditions obtained in this way initially apply to all traffic situations with the same zone graph. Specification only occurs by applying the situation-specific parameters of the analyzed traffic situation to the results.

[0008] Prioritization places the possible vehicle behaviors identified for a given situation into a ranking determined by a predefined set of rules. German patent application 10 2023 201 983.3 describes such a method for prioritizing the possible behaviors of a vehicle based on the boundary conditions defining these behaviors and in conjunction with a predefined set of rules. Prioritization is automated and performed dynamically at system runtime. It requires no specific metric but relies solely on the decision-making process structure of the rule set. Disclosure of the invention

[0009] According to the invention, it has been recognized that such an analysis of the given traffic situation can not only be used for behavioral planning, but is also very well suited for verifying trajectory suggestions from a planning module, specifically for a verification that takes the context of the traffic situation into account. The result of this verification then also enables a meaningful and informative evaluation of the proposed trajectories.

[0010] According to the invention, a compatibility check is performed for a proposed trajectory and at least one of the identified possible behaviors of the vehicle in the given situation. This check verifies whether the proposed trajectory fulfills the boundary conditions of the respective behavior. Based on the results of these compatibility checks and the prioritization of the respective behaviors, the proposed trajectory is then evaluated.

[0011] Compatibility checks are used to first verify the validity of a trajectory. This means they first check whether the trajectory corresponds to at least one behavior identified as a possible behavior of the vehicle in the given situation. These compatibility checks can also determine, for example, which boundary conditions for which behaviors are not met. The quality of the trajectory is then evaluated. This evaluation considers not only its validity with regard to the individual behaviors, but also their ranking or prioritization. Since this is determined by the predefined set of rules, user-defined evaluation criteria can be considered, such as violations of traffic regulations, endangering other road users, etc.

[0012] In principle, there are many possibilities for evaluating the trajectories within the framework of the inventive method.

[0013] In a preferred embodiment, a trajectory is simply evaluated based on the highest priority of all tested behaviors whose boundary conditions it satisfies, and / or based on the lowest priority of all tested behaviors whose boundary conditions it does not satisfy. These priorities can simply be used as the evaluation criteria. Alternatively, it would be conceivable to consider the priorities of all behaviors whose boundary conditions the trajectory satisfies, or only the priorities of behaviors whose boundary conditions the trajectory does not satisfy, and then weight these priorities, for example, depending on which boundary conditions are violated and / or met.All these assessments are highly meaningful with regard to the given traffic situation and user-defined quality criteria, as the priorities represent the quality of the respective behavior in relation to the predefined set of rules.

[0014] The method according to the invention can advantageously be used in the context of behavior planning for a vehicle when one of several proposed trajectories must be selected for implementation in automated operating mode. These trajectories can be proposed by a planning module. However, the method according to the invention can also be advantageously integrated into a multi-path architecture with several different planning modules that independently propose trajectories. In any case, the method according to the invention enables a comparison of the different trajectory proposals based on the results of compatibility tests and / or the evaluations of the proposed trajectories. According to the invention, this comparison takes into account quality criteria that have been determined for the given traffic situation and is therefore particularly informative.

[0015] In a preferred embodiment of the invention, the results of the compatibility checks and / or the evaluations of the proposed trajectories are temporarily stored or saved. This data can then advantageously be used for documentation purposes. This is of particular interest, for example, if an emergency situation arose in which a critical trajectory was executed because the operational plan did not identify any non-critical alternatives. Based on the saved results of the compatibility checks and the trajectory evaluations, such a decision can be easily traced and justified.

[0016] In an advantageous further development of the method according to the invention, the results of the compatibility tests and / or the evaluation of a trajectory are made available to the planning component that proposed the trajectory. This feedback can be used to improve the performance of the planning component.

[0017] The method according to the invention can also be used to detect critical situations. These are generally characterized by an unexpected development in the traffic situation. The analysis of such traffic situations within the framework of the method according to the invention then increasingly reveals vehicle behaviors that are not compliant with regulations or are only given very low priority based on the predefined set of rules. In most cases, the trajectory suggestions of the planning module are only partially adequate for such situations. This has a significant impact on the results of the compatibility tests, but above all on the evaluation of these trajectory suggestions according to the invention. Accordingly, critical situations can be identified relatively easily based on the results of the compatibility tests and / or the evaluation of the proposed trajectory.

[0018] This capability can be used advantageously to generate a so-called "take-over request" when a critical situation has been detected, i.e., a signal to terminate the automated operating mode and transition to a manual operating mode.

[0019] Furthermore, the inventive method can also be used to trigger the permanent storage of continuously aggregated and buffered situation-specific information when a critical situation is detected. This data can then be used to better handle future critical situations and, ideally, to prevent them altogether. For this purpose, the situation-specific information, in particular sensor data and internal status data, is temporarily stored in a ring buffer during operation. When a critical situation is detected, the data from the ring buffer is transferred to persistent storage, from which it can then be transmitted anonymously as soon as a stable data connection is established. A data set generated in this way can, for example, be used for the further development and verification of automated driving functions.

[0020] As already mentioned, the present invention relates not only to a computer-implemented method but also to a computer-implemented system for monitoring trajectories proposed by at least one planning module for an automated operating mode of a vehicle. The system according to the invention comprises a perception level for aggregating situation-specific information from vehicle-integrated and external information sources. Vehicle-integrated information sources can be vehicle-integrated sensors, such as lidar sensors, radar sensors, and / or RGB cameras installed on the vehicle, which detect the vehicle's surroundings, or sensors that detect vehicle status data, such as speed, orientation, etc. External information sources can be sensors, such as lidar sensors, radar sensors, and / or RGB cameras, that are installed on infrastructure elements or other road users.Other potential information sources include stored map data, as well as queryable weather and road condition information, traffic situation data, etc. The information from these various sources is aggregated at the perception level and, if necessary, pre-processed into contextual information.

[0021] The system according to the invention further comprises an evaluation module for generating an environment model based on the aggregated situation-specific information and an analysis module for determining possible vehicle behaviors based on the situation-specific information and the environment model, wherein each behavior is described by a set of boundary conditions. Furthermore, the system according to the invention has a predefined set of rules available, which is used to evaluate and prioritize the possible behaviors based on their respective boundary conditions.

[0022] An essential component of the system is an evaluation module for the proposed trajectories, designed to perform a compatibility check for a proposed trajectory and at least one of the identified behaviors by verifying whether the proposed trajectory meets the boundary conditions of the respective behavior. This evaluation module is further designed to then assess the proposed trajectory based on the results of these compatibility checks and the prioritization of the respective behaviors.

[0023] The evaluation module can also be designed to evaluate several trajectories proposed by one or more planning modules in parallel and to compare the results of the compatibility checks and / or the assessments of the proposed trajectories. This proves particularly advantageous when using the system according to the invention as a safety monitor of a behavioral planner with a multi-path architecture.

[0024] The evaluation module is preferably designed to make the results of the compatibility checks and / or the evaluation of the proposed trajectory available via at least one interface, in particular to a downstream control module for selecting a trajectory that is then to be implemented by the vehicle's actuators. This interface can also be used for feedback to the planning module that proposed the trajectory.

[0025] Advantageously, the system according to the invention further has at least one storage module for storing selected trajectories together with the associated results of the compatibility tests and / or the associated evaluations, so that it can log the trajectory proposals of a planning module and their evaluations and can be used like a kind of tachograph.

[0026] In a further advantageous embodiment, the system according to the invention is equipped with a warning function. In this case, the evaluation module is designed to detect, based on the results of the compatibility tests and / or the evaluation of the proposed trajectory, whether a critical situation exists. The evaluation module can then generate a "takeover request," i.e., a signal to terminate the automated operating mode and transition to a manual operating mode. drawing

[0027] Exemplary embodiments and advantageous further developments of the invention are explained in more detail below in conjunction with the figures. Fig. Figure 1 illustrates the measures according to the invention for monitoring trajectories proposed by a planning module within the framework of behavior planning for a vehicle. Fig. Figure 2 shows a block diagram of a behavior planning system with a multi-path architecture, into which a safety monitor according to the invention is integrated. Description of exemplary implementations

[0028] The starting point for behavioral planning for an automated driving function of a vehicle pursuing a predetermined destination is always the state of the traffic environment at a given planning point in time, and in particular the state of all participants in that environment at that time. The state of the traffic environment is described by situation-specific information that is aggregated from various vehicle-integrated and / or external information sources at the planning point or over a certain period before and up to that point. For this purpose, the vehicle is typically equipped with a perception level that includes the necessary sensors and usually also suitable data processing equipment.

[0029] In connection with Fig. In the embodiment described in Figure 1, a planning module 20 generates a trajectory 21 based on the specified destination and the aggregated situation-specific information, which is to be checked and evaluated using the method according to the invention.

[0030] To this end, the given traffic scene is first analyzed based on the specified destination and the aggregated situation-specific information. In a first analysis step 11, a specific environmental model is generated. In this case, this is an abstract description of the traffic situation according to the SOCA method mentioned earlier. In a second analysis step 12, possible vehicle behaviors for the given situation are derived from this abstract description of the traffic situation in conjunction with the situation-specific information, with each behavior being described by a set of boundary conditions. These behaviors are then prioritized or ranked in a third analysis step 13 using a predefined set of rules.

[0031] The predefined set of rules assigns a semantic meaning to the possible behaviors and enables their prioritization based on safety considerations, with behaviors receiving progressively lower priority as boundary conditions become increasingly relaxed. This relaxation of boundary conditions can be broadly categorized as functional degradation, violation of traffic rules, and endangering or harming other road users. Functional degradation occurs, for example, when a vehicle is instructed to stop before an intersection instead of proceeding through it. A violation of traffic rules would include, for instance, crossing a stop line because the vehicle cannot brake in time, or driving at excessive speed.An example of a relaxation of the category of endangerment or harm to road users would be crash mitigation, which accepts damage to the vehicles in order to protect the lives of vulnerable road users.

[0032] Fig. Figure 1 shows the result of this prioritization in the form of a corresponding list 30 of behaviors 31 to 34 in descending order of priority. These behaviors 31 to 34 are in the right half of the Fig. Figure 1 is again presented scenically, specifically for a given traffic situation with a vehicle 1 on a two-lane roadway 2 and an obstacle 3 in the direction of travel in front of vehicle 1. The zones prohibited for vehicle 1 are shown hatched. The proposed trajectory 21 is shown as a line. The individual behaviors 31 to 34 are characterized or defined by the following boundary conditions: 31 “Drive on lane” - Driving around obstacle 1 without leaving the lane, 32 “Evade blockade” - Bypassing obstacle 1 by changing lanes to the oncoming lane, 33 “Stop at blockade” - Stop before the obstacle 1, 34 “Stop at blockade & leaving the lane” - Stop before the obstacle 1, while performing a lane change to the oncoming lane.

[0033] According to the invention, the trajectory 21 proposed by the planning module 20 is subjected to a compatibility check by verifying whether it fulfills the boundary conditions of at least one of the behaviors 31 to 34. For this purpose, the individual behaviors 31 to 34 are checked in the ranking order of list 30.

[0034] The highest-priority behavior, 31, stipulates that vehicle 1 should drive around obstacle 3 without leaving its lane. As can be seen from the scene depiction 31, trajectory 11 does not meet these boundary conditions, which is symbolized here by an x.

[0035] The next possible behavior 32 from list 30 involves driving around obstacle 1, even if this requires changing lanes to the oncoming lane, since no oncoming traffic is expected. Trajectory 11, symbolized here by a hook, satisfies this behavior 32. In the embodiment described here, the compatibility check is terminated at this point, as a possible behavior has been identified that the proposed trajectory fulfills.

[0036] The proposed trajectory 21 is now evaluated based on the results of the compatibility checks and the prioritizations of the respective behaviors. In this case, trajectory 21 is simply assigned a rating corresponding to the priority of behavior 32.

[0037] It should be noted at this point that, within the framework of the inventive method, further behaviors from the list of possible behaviors can also be tested, which is useful, for example, if a more differentiated evaluation of the trajectory is to be carried out.

[0038] The more boundary conditions need to be relaxed and the more critical the relaxation, the worse an input trajectory is rated. In particular, trajectories that violate traffic rules or even unacceptably endanger other road users are generally rejected as invalid and only accepted in absolute emergencies where the vehicle has no other option. In these cases, it proves advantageous to save the critical input trajectory, along with the results of the compatibility checks and the resulting rating, in a log file. Based on this documentation, both the incident and the safety rationale behind the decision underlying the input trajectory can be traced.

[0039] In the present embodiment, the results of the compatibility checks and / or the evaluation of the proposed trajectory 21 are fed back to the planning module 20 in a feedback loop 22. Specifically, the highest-priority, fulfilled behavior 32 is reported back to the planning module 20. Furthermore, it is reported which higher-priority behaviors—behavior 31—the input trajectory 21 does not fulfill.

[0040] As already mentioned at the outset, a safety monitor according to the invention can advantageously be integrated into the multi-path architecture of a vehicle behavior planning system. Such an embodiment is described in Fig. 2 shown.

[0041] The system 100 presented here comprises a perception level 110 for aggregating and, if necessary, preprocessing scene-specific or situation-specific information from a traffic scene. The processed situation-specific information is fed to several planning modules 101, 102, ...10N, which, in parallel and independently of each other, propose trajectories for the vehicle based on the situation-specific information. The planning modules 101, 102, ...10N can be identical. However, they can also be different planning modules that use rule-based and / or AI-based methods to generate trajectories.

[0042] The situation-specific information is also fed to a first module 120 of a safety monitor according to the invention. This module 120 analyzes the given traffic situation based on the situation-specific information and then generates a prioritized list of possible vehicle behaviors in the given situation using a predefined set of rules. In the embodiment described here, the module 120 thus comprises the functionality of an evaluation module for generating an environment model based on the situation-specific information, the functionality of an analysis module for determining possible vehicle behaviors based on the situation-specific information and the environment model, wherein each behavior is described by a set of boundary conditions, and a predefined set of rules for evaluating and prioritizing the possible behaviors based on their respective boundary conditions.

[0043] The Safety Monitor described here further includes, for each planning module 101, 102, ... 10N, an evaluation module 121, 122, ... 12N, to which the prioritized list of possible behaviors is provided. These evaluation modules 121, 122, ... 12N are each located in the performance path of the associated planning module 101, 102, ... 10N, so that the trajectory proposals of the planning modules can be checked and evaluated in parallel. The check includes compatibility checks, as described in connection with... Fig. 1. described in detail, from which an evaluation is then derived in conjunction with the prioritized list of behaviors.

[0044] It should be noted here that the trajectories of a planning module can also be checked and evaluated by an evaluation module other than the one assigned to it, for example, if the functionality of an evaluation mode is limited. This possibility is indicated here for planning module 101, whose trajectories are forwarded to both evaluation module 121 and evaluation module 122.

[0045] The individual trajectory proposals, along with the results of the respective evaluation modules 121, 122, ...12N, are fed into a selection module 130. Therefore, evaluation modules 121, 122, ...12N are each represented here with a first output arrow for the trajectory proposal and a second output arrow for the evaluation results. In selection module 130, the evaluation results are compared to select exactly one trajectory for controlling 140 of the vehicle's actuator. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2023 201 983.3

[0008] Cited non-patent literature

[0000] M. Butz et al., “SOCA: Domain Analysis for Highly Automated Driving Systems,” 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020, pp. 1-6, doi: 10.1109 / ITSC45102.2020.9294438

[0006]

Claims

[1] Computer-implemented method for monitoring trajectories (21) proposed by at least one planning module (20) for an automated operating mode of a vehicle, in which at least the following process steps are performed: a. Aggregating situation-specific information, b. Generating an environment model based on the situation-specific information (analysis step 11), c. Determining possible behaviors (31 to 34) of the vehicle based on the situation-specific information and the environment model, whereby each behavior (31 to 34) is described by a set of boundary conditions (analysis step 12), and d. Prioritizing the possible behaviors (31 to 34) of the vehicle based on the boundary conditions in conjunction with a predefined set of rules (analysis step 13); characterized by, that for a proposed trajectory (21) and at least one of the identified behaviors (31, 32) a compatibility check is carried out by checking whether the proposed trajectory (21) meets the boundary conditions of the respective behavior (31, 32), and that the proposed trajectory (21) is then evaluated on the basis of the results of these compatibility checks and the prioritizations of the respective behaviors (31, 32). [2] Method according to claim 1, characterized by , that the evaluation of the proposed trajectory (21) is based on the highest prioritization of all tested behaviors (31, 32) whose boundary conditions it fulfills and / or on the lowest prioritization of all tested behaviors (31, 32) whose boundary conditions it does not fulfill. [3] Method according to one of claims 1 or 2, characterized by, that several trajectories proposed by one or more planning modules (101, 102, ...10N) are compared with each other based on the results of the compatibility checks and / or the evaluations of the proposed trajectories in order to select a trajectory for implementation in automated operating mode based on this comparison. [4] Method according to any one of claims 1 to 3, characterized by that the results of the compatibility checks and / or the evaluation of the proposed trajectory are cached or saved. [5] Method according to any one of claims 1 to 4, characterized by , that the results of the compatibility checks and / or the trajectory evaluation are made available to the planning component (20) that proposed the trajectory (21). [6] Method according to any one of claims 1 to 5, characterized by, that based on the results of the compatibility tests and / or the evaluation of the proposed trajectory, it is determined whether a critical situation exists. [7] Method according to claim 6, characterized by , that a signal to terminate the automated operating mode and transition to a manual operating mode is generated when a critical situation is detected. [8] Method according to one of claims 6 or 7, characterized by that the aggregated situation-specific information is continuously cached and that this cached situation-specific information is permanently stored, especially when a critical situation has been detected. [9] Computer-implemented system for monitoring trajectories proposed by at least one planning module (101, 102, ...10N) for an automated operating mode of a vehicle, comprising at least: a. A perception level (110) for aggregating situation-specific information from vehicle-internal and vehicle-external information sources, b. An evaluation module for generating an environmental model based on situation-specific information, c. An analysis module for determining possible vehicle behaviors based on situation-specific information and the environment model, where each behavior is described by a set of boundary conditions, d. A predefined set of rules for evaluating and prioritizing possible behaviors based on their respective boundary conditions, and e. An evaluation module (121, 122, ...12N) for the proposed trajectories, designed to to perform a compatibility check for a proposed trajectory and at least one of the identified behaviors by checking whether the proposed trajectory satisfies the boundary conditions of the respective behavior, and The proposed trajectory should then be evaluated based on the results of these compatibility checks and the prioritization of the respective behaviors. [10] System according to claim 9, characterized by that the evaluation module is designed to evaluate several trajectories proposed by one or more planning modules in parallel and to compare the results of the compatibility checks and / or the assessments of the proposed trajectories. [11] System according to one of claims 9 or 10, characterized bythat the evaluation module is designed to make the results of the compatibility checks and / or the evaluation of the proposed trajectory available via at least one interface, in particular to a downstream control module and / or the planning module that proposed the trajectory. [12] System according to any one of claims 9 to 11 comprising at least one storage module for storing selected trajectories together with the associated results of the compatibility tests and / or the associated evaluations. [13] System according to any one of claims 9 to 12, characterized by, that the evaluation module is designed to detect, based on the results of the compatibility tests and / or the evaluation of the proposed trajectory, whether a critical situation exists, and to generate a signal to terminate the automated operating mode and transition to a manual operating mode when a critical situation has been detected.

Images