Computer-implemented method for generating an instruction

Abstract

The invention relates to a computer-implemented method for generating an instruction for a control system of an automated driving function of a motor vehicle, the method comprising the following steps: - generating an action request by the control system of the automated driving function, wherein the action request comprises an environment model and the position of the vehicle; - Classifying the action request using a machine learning algorithm; and - Generating an action instruction from the classification of the action request, whereby the action instruction depends on the position of the vehicle.

Description

[0001] The invention relates to the generation of instructions, in particular for automated or partially automated driving functions in motor vehicles. State of the art

[0002] Automated vehicles have been a trend for several years and are now already on the market and on public roads in certain forms. However, approvals are always limited to an Operational Design Domain (ODD).

[0003] The term "operational design domain" refers to the area within which vehicles with an automated driving function can operate safely. It defines the specific conditions under which the vehicle is capable of operating safely, including environmental conditions, traffic scenarios, speed ranges, and other relevant parameters.

[0004] Establishing a clear ODD is crucial to ensure that automated driving functions are only used in situations for which they were designed. A comprehensive understanding of the ODD is important to ensure that the vehicle can respond appropriately to various challenges and that its automated driving functions are reliable.

[0005] With regard to automated driving functions, the ODD can include factors such as weather conditions, road types, traffic density, speed ranges, and other environmental conditions. By clearly defining the ODD, developers and operators of automated driving functions can ensure that they are only activated in situations where they are capable of navigating and operating safely.

[0006] Vehicles with automated driving functions are only allowed to operate within a defined ODD on a regular basis. When they leave the ODD, concepts are conceivable in which an operator is connected to manually release the ODD extension.

[0007] For example, a vehicle may drive outside the officially drivable area after an operator has verified via video transmission and / or using additional sensors that the maneuver can be performed safely. In another example, an operator is only called in if the vehicle cannot handle the situation encountered on its own and safely. This could be the case, for example, if the vehicle is required to negotiate an obstacle, but the obstacle's classification is unclear. The operator intervenes and confirms the classification or performs it himself.

[0008] An important aspect in the approval of vehicles with automated driving functions is the conformity of the vehicle's behavior with the law and local traffic regulations.

[0009] A conformity check is particularly conceivable in two cases. Before registration, conformity must be established to ensure proper functioning and maintain roadworthiness. If the vehicle is involved in an accident, conformity must also be checked. This can serve, in particular, to demonstrate that the vehicle has complied with the law.

[0010] For liability reasons, the vehicle manufacturer must provide proof of legal compliance. However, this is complicated by the fact that laws and traffic regulations can vary from country to country. Therefore, approval for automated driving functions in vehicles can only be granted on a country-specific basis. Therefore, an operator must always check the approval of a maneuver specifically against the locally applicable rules and regulations.

[0011] Furthermore, legal texts and traffic regulations may contradict each other or, in some cases, at least be ambiguous. This has so far resulted in a case-by-case review. Furthermore, legal texts are occasionally interpreted, refined, or supplemented by court rulings. In particular, situation-specific details and partial aspects are occasionally further defined by court rulings.

[0012] Court rulings and legal texts are largely stored in public databases and are generally accessible. However, not every court ruling is valid, as rulings can be overturned by corresponding rulings of higher courts.

[0013] The invention is therefore based on the object of proposing a method with which instructions for automated or partially automated driving functions in vehicles can be generated, with which contradictory or unsolvable situations can be resolved.

[0014] The problem is solved by the subject matter of the independent claims. Disclosure of the invention

[0015] According to a first aspect of the invention, this object is achieved by a computer-implemented method for generating an action instruction for a control system of an automated driving function. The method comprises the following steps: - generating an action request by the control system of the automated driving function, wherein the action request comprises an environment model and the position of the vehicle; - Classifying the action request using a machine learning algorithm; and - Generating an action instruction from the classification of the action request, whereby the action instruction depends on the position of the vehicle.

[0016] If the vehicle is outside its ODD, it may encounter situations it is unfamiliar with. In this case, the control system generates an action request.

[0017] The action request includes an environment model and the vehicle's position. Using the environment model and position, the action request can be evaluated by a machine learning algorithm.

[0018] A machine learning algorithm is an algorithm designed to automatically detect patterns and relationships in data and make predictions or decisions. It is created by training on existing data and can then be applied to new, unknown data to generate predictions or classifications.

[0019] A machine learning algorithm can take various forms, such as linear models, decision trees, support vector machines, neural networks, and many others. It is optimized by learning from the training data, identifying patterns and rules to make the best possible predictions or classifications for new data.

[0020] The effectiveness of a machine learning algorithm depends on several factors, including the quality and quantity of training data, the choice of model, the model configuration, and the evaluation of the model using evaluation metrics. The model can be continuously improved and optimized to maximize accuracy and performance.

[0021] The linear regression model assumes a linear relationship between a dependent variable and one or more independent variables and is used to make predictions about continuous values.

[0022] Support vector machines (SVMs) are a model used for classification or regression that detects patterns in the data. They search for the optimal separation between different classes or attempt to fit a continuous function to the data.

[0023] Decision trees are a model that creates decision rules in the form of a tree diagram. They divide data based on features and enable predictions or classifications.

[0024] A probabilistic model is Naive Bayes, which is based on Bayes' theorem and is used for classification. It assumes that features are independent of each other and calculates the probability of a particular class based on the given features.

[0025] Neural networks refer to models that are primarily used for processing a wide variety of data. The architecture of a neural network comprises multiple nodes, neurons, or nodes, arranged in layers.

[0026] Preferably, a machine learning algorithm is used to classify the action request, which is particularly suitable for classification tasks.

[0027] Classification is used to categorize the vehicle's situation. This can be used, for example, to examine how other vehicles, and possibly even human drivers, have behaved in a similar situation. Depending on the vehicle's position, other traffic rules or instructions may arise that must also be observed. Therefore, it is important that the vehicle's position is also used to classify the action request. The position can, for example, be a position relative to the environment model or an absolute position, especially a geo-position.

[0028] The classification divides the vehicle's situation into one of several classes. The classes can, for example, consider one or more parameters: a danger level, an urgency level, the presence of a rule conflict, the type of rule conflict, and / or the vehicle's potential scope of action. For example, the danger level may be high if people are present in the environment model. Urgency, in turn, could be a measure of how time-critical the action request is. Several parameters can be correlated with each other, resulting in different classes.

[0029] Once the class is determined, an action instruction is generated. The action instruction can be specifically designed to resolve a rule conflict present in the vehicle. However, not all action requests necessarily involve a rule conflict. It is also possible that an action request is generated when the control system is confronted with a situation that it is unfamiliar with and therefore cannot handle independently.

[0030] An action instruction comprises a control command for the vehicle's control system. For example, the action instruction can include acceleration or deceleration, as well as steering or waiting. With the action instruction, the vehicle is able to overcome the situation that triggered the action request. The method thus achieves the object of the invention.

[0031] In one embodiment, the action request may comprise two or more possible actions, wherein the action instruction comprises a command to execute one of the possible actions.

[0032] In one embodiment, the environment model comprises a map and / or object information, in particular relative positions and speeds to objects in the environment of the vehicle.

[0033] Maps are particularly well-suited for depicting an environment. Position information can be used to indicate not only the vehicle's location, but also objects in the vehicle's immediate vicinity. Furthermore, speeds can be assigned to the objects, allowing their movement to be predicted.

[0034] Preferably, the most precise information about the environment and the objects in the environment is used to classify the action request and generate an action instruction. The more precise the information in the action request, the higher the quality of the generated action instruction.

[0035] In one embodiment, the action request comprises sensor data of the vehicle, wherein the environment model is generated from the sensor data.

[0036] Information about the environment can be collected directly or indirectly by the vehicle's sensors. This can include, in particular, radar sensors, lidar sensors, ultrasonic sensors, GPS sensors for determining position, or cameras for recording the environment.

[0037] When the vehicle collects sensor data to create the environment model, the model is based on the vehicle's perception. The resulting environment model then corresponds to what a driver of the vehicle would perceive, or even more, because the combination of the various sensors allows for a significantly broader perceptual apparatus than that represented by the human senses.

[0038] The environment model does not necessarily have to be a 3D model of the environment. The environment model can include a mathematical description of objects, especially their positions and velocities relative to the vehicle. For example, the environment itself can be represented as a function and objects as vectors.

[0039] In one embodiment, the environment model comprises a natural language description of the sensor data or a classification result based on the sensor data.

[0040] Descriptions of the environment written in natural language can advantageously be compared particularly easily with traffic regulations, laws, and case law regarding situations that appear contradictory to the vehicle. This allows the description text to be easily compared with other texts, particularly legal texts such as laws and court rulings, regarding relevant situations when classifying the action request. A large language model is preferably used for this embodiment.

[0041] A large language model is a term used to describe language models that use large amounts of text data to learn the ability to process and generate natural language. These models are commonly used in machine text processing and can perform various tasks such as machine translation, text generation, text classification, question-answering systems, and more.

[0042] Large language models are special models of neural networks. The architecture of a neural network comprises several nodes, neurons, or nodes arranged in layers. Each node receives an input value and generates an output value from it. The input values can be the input or parts of it, but also output values from nodes in previous layers.

[0043] Due to their structure, neural networks, especially when trained as a large language model, are particularly well suited for text processing.

[0044] In one embodiment, the action instruction comprises a condition and at least one alternative action in the event that the condition is not met.

[0045] It may happen that the situation the vehicle is in has not yet occurred during training or algorithm development, or that the machine learning algorithm cannot generate a clear instruction due to ambiguous legal requirements. In this case, the instruction creates a condition that must be comprehensively verified by the vehicle or its perception system and its sensors.

[0046] The condition may, for example, include an independent classification of a single or multiple objects in the vehicle's surroundings.

[0047] For example, if the vehicle encounters an obstacle that is large enough to be easily driven over, but the object is moving, it could be an animal, such as a cat, hedgehog, or something similar. In such a case, the instruction could be: Drive over the object if it is not moving, has not moved, and / or will not move. Otherwise, wait until the object has moved away and then continue driving.

[0048] In this way, even complex situations that cannot be clearly classified can be processed with the proposed method.

[0049] In one embodiment, the action request further includes a time of the action request.

[0050] In some situations, time may play a role in classifying the action request. There are traffic regulations that are time-dependent, such as night-time driving bans. Furthermore, a dark environment may appear like night or a heavily overcast sky to a machine. Specifying the time provides another parameter that can be considered when classifying the action request and generating the action instruction.

[0051] In one embodiment, the machine learning algorithm was trained using temporally and / or spatially limited data sources.

[0052] Traffic rules can vary locally, particularly due to house rules, for example, in a parking garage or on company premises, or due to national traffic laws and regulations. Furthermore, different traffic rules may apply at different times of day.

[0053] If the machine learning algorithm is trained on these different situations using temporally and / or spatially limited data sets, this improves the classification of the action request when applying the procedure.

[0054] In one embodiment, the action instruction, the data sources relevant for the action instruction, the time of the action request and the action request are stored in a memory.

[0055] In particular, the memory can be a protected memory that is specially protected against mechanical damage. Furthermore, the memory can be used in a tachograph to reconstruct the cause of the accident even after a serious accident.

[0056] By saving the instructions, the data that may have been responsible for the accident can be analyzed. It can be particularly useful to save the requests for action and, above all, the generated instructions when retrospectively determining who was at fault for the accident and clarifying the insurance obligations of those involved in the accident.

[0057] In one embodiment, the machine learning algorithm generates a counter query to the environment model when classification of the action request cannot be performed due to insufficient information in the environment model, wherein the counter query specifies the information that the machine learning algorithm lacks to classify the action request.

[0058] In this embodiment, it is provided that the machine learning algorithm can determine itself whether sufficient information is available for classifying the action request and generating an action instruction.

[0059] This embodiment is particularly suitable for cases where the vehicle's control system provides the machine learning algorithm with only selected information, rather than all available information, along with the action request. This can have several reasons, such as bandwidth limitations for transmission or the lack of a requirement in the normal case, thus allowing for more efficient resource utilization.

[0060] The counter-request can specify the required information in such a way that it simply needs to be read from the control system's memory. Furthermore, the counter-request can be coded so that the control system automatically recognizes the counter-request upon receipt and processes it accordingly. Using the information the machine learning algorithm receives in response to the counter-request, it can generate the action instruction for which a conflict previously existed.

[0061] This embodiment can be combined, in particular, with the embodiment in which a condition and an alternative course of action are generated. If the information obtained cannot conclusively resolve the conflict, the condition can be established and sent to the control system along with the alternative course of action. In this case, the vehicle's perception system is prompted to gather further information from the environment and then execute one of the alternative courses of action. Furthermore, multiple alternative courses of action can also be generated.

[0062] In one embodiment, the machine learning algorithm is executed on a classification system separate from the vehicle, the method further comprising the steps of: - transmitting the action request from the vehicle to the classification system for classifying the action request by the machine learning algorithm; and - Transferring the action instruction from the classification system to the vehicle to execute the action instruction.

[0063] The classification system may be a computer system external to the vehicle, for example a web-based system or one that is otherwise communicatively connected to the vehicle.

[0064] By separating the classification system from the vehicle, it is possible to continuously develop the classification system and train the machine learning algorithm implemented on it with data from other vehicles. This training can preferably take place in the background, without the vehicle having to visit a workshop for an update. Furthermore, the machine learning algorithm can be updated with current changes to traffic laws and regulations, as well as recent rulings on specific cases. Specifications from the manufacturer or developer can also be incorporated into the machine learning algorithm in the background.

[0065] In a further aspect, the invention relates to a computer-implemented method for training a machine learning algorithm for generating an action instruction for a control system of an automated driving function, the method comprising the following steps: - generating an action request by the control system of the automated driving function, wherein the action request comprises an environment model and the position of the vehicle; - Classifying the action request using a machine learning algorithm; - generating an action instruction from the classification of the action request, whereby the action instruction depends on the position of the vehicle; and - Using the generated instruction as ground truth for training the machine learning algorithm to control the vehicle.

[0066] In this aspect of the invention, the machine learning algorithm is trained with the information provided by the vehicle's action request. Preferably, the use of the information as ground truth is preceded by confirmation from the control system.

[0067] The confirmation can be used to inform the machine learning algorithm that the generated action instruction was appropriate to the vehicle's situation.

[0068] In one embodiment, the action request further includes a time of the action request. The machine learning algorithm was trained using temporally and / or spatially limited data sources, and the action instruction, the data sources relevant to the action instruction, the time of the action request, and the action request are stored in a memory.

[0069] These features provide the same advantages as for the proposed method, in which the machine learning algorithm is not retrained.

[0070] In a further aspect, the invention relates to a computer program with program code for carrying out a method as described above when the computer program is executed on a computer

[0071] In a further aspect, the invention relates to a computer-readable data carrier with program code of a computer program for carrying out a method as described above when the computer program is executed on a computer.

[0072] In a further aspect, the invention relates to a system for generating an action instruction for a control system of an automated driving function, wherein the system is designed to carry out a method as described above.

[0073] In summary, the present invention provides a computer-implemented method for generating an instruction for a control system of an automated driving function, a computer-implemented method for training a machine learning algorithm for generating an instruction for a control system of an automated driving function, a computer program with program code, a computer-readable data carrier and a system for generating an instruction for a control system of an automated driving function.

[0074] Furthermore, the action request can also be received and then processed by a human-machine interface. This allows, for example, a developer of an automated driving function to generate the action instruction during the development of the automated function and incorporate it into the development process.

[0075] The described designs and further training courses can be combined as desired.

[0076] Further possible embodiments, developments and implementations of the invention also include combinations of features of the invention described previously or below with regard to the exemplary embodiments that are not explicitly mentioned. Short description of the drawing

[0077] The accompanying drawings are intended to provide a further understanding of embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain principles and concepts of the invention.

[0078] Other embodiments and many of the aforementioned advantages will become apparent upon review of the drawings. The elements shown in the drawings are not necessarily drawn to scale.

[0079] It shows: Fig. 1 schematically shows the sequence of the proposed method according to one embodiment.

[0080] In the figure of the drawing, the same reference symbols designate the same or functionally identical elements, parts or components, unless otherwise stated.

[0081] Fig. 1 schematically shows the flow of the computer-implemented method according to one embodiment.

[0082] The procedure is used when a vehicle is outside its ODD in a situation where the control system cannot generate a clear instruction for action.

[0083] The method therefore begins in step S10 by generating an action request. The action request includes an environmental model and the position of the vehicle. This information is used to describe the situation in which the vehicle is located.

[0084] The action request is input into a machine learning algorithm and classified in step S12. The classification of the action request allows the machine learning algorithm to contextualize the vehicle's situation. The classification result forms the basis for the next step S14.

[0085] In step S14, the machine learning algorithm generates an action instruction. The control system can use the action instruction to automatically guide the vehicle out of the situation.

[0086] The machine learning algorithm can be trained, in particular, with traffic rules and complex situations, especially with analyzed court rulings, in order to resolve complex and / or contradictory situations for the vehicle.

Claims

[1] Computer-implemented method for generating an instruction for a control system of an automated driving function of a motor vehicle, the method comprising the following steps: - generating an action request by the control system of the automated driving functions, wherein the action request comprises an environment model and the position of the vehicle; - Classifying the action request using a machine learning algorithm; and - Generating an action instruction from the classification of the action request, whereby the action instruction depends on the position of the vehicle. [2] Computer-implemented method according to claim 1, wherein the environment model comprises a map and / or object information, in particular relative positions and speeds to objects in the environment of the vehicle. [3] Computer-implemented method according to one of the preceding claims, wherein the action request comprises sensor data of the vehicle and wherein the environment model is generated from the sensor data. [4] Computer-implemented method according to one of the preceding claims, wherein the environment model comprises a natural language description of the sensor data or a classification result based on the sensor data. [5] Computer-implemented method according to one of the preceding claims, wherein the action instruction comprises a condition and at least one alternative action in the event that the condition is not met. [6] A computer-implemented method according to any one of the preceding claims, wherein the action request further comprises a time of the action request. [7] Computer-implemented method according to claim 6, wherein the machine learning algorithm was trained using temporally and / or spatially limited data sources. [8] Computer-implemented method according to claim 7, wherein the action instruction, the data sources relevant for the action instruction, the time of the action request and the action request are stored in a memory. [9] A computer-implemented method according to any one of the preceding claims, wherein the machine learning algorithm generates a counter-query to the environment model if a classification of the action request cannot be performed due to insufficient information in the environment model, the counter-query specifying the information that the machine learning algorithm lacks to classify the action request. [10] A computer-implemented method according to any one of the preceding claims, wherein the machine learning algorithm is executed on a classification system separate from the vehicle, the method further comprising the steps of: - transmitting the action request from the vehicle to the classification system for classifying the action request by the machine learning algorithm; and - Transferring the instruction from the classification system to the vehicle to execute the instruction. [11] A computer-implemented method for training a machine learning algorithm for generating an instruction for a control system of an automated driving function of a motor vehicle, the method comprising the following steps: - generating an action request by the control system of the automated vehicle, the action request comprising an environment model and the position of the vehicle; - Classifying the action request using a machine learning algorithm; - generating an action instruction from the classification of the action request, whereby the action instruction depends on the position of the vehicle; and - Using the generated instruction as ground truth for training the machine learning algorithm to control the vehicle. [12] Computer-implemented method according to claim 11, wherein the action request further comprises a time of the action request, wherein the machine learning algorithm was trained using temporally and / or spatially limited data sources, wherein the action instruction, the data sources relevant for the action instruction, the time of the action request and the action request are stored in a memory. [13] A computer program comprising program code for carrying out a method according to any one of claims 1 to 10 when the computer program is executed on a computer. [14] A computer-readable data carrier comprising program code of a computer program for carrying out a method according to any one of claims 1 to 10 when the computer program is executed on a computer. [15] System for generating an action instruction for a control system of an automated driving function of a motor vehicle, wherein the system is designed to carry out a method according to one of claims 1 to 10.

Images