Computer-implemented method for controlling a vehicle

The integration of a large language model in vehicle control systems addresses the challenge of planning trajectories for unforeseen situations by utilizing 'world knowledge', resulting in improved safety and comfort for autonomous vehicle operations.

DE102023211809A1Pending Publication Date: 2025-05-28ZF FRIEDRICHSHAFEN AG

Patent Information

Application Number
DE102023211809
Authority / Receiving Office
DE · DE
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-11-27
Publication Date
2025-05-28

AI Technical Summary

Technical Problem

Fully autonomously driving vehicles face challenges in planning satisfactory movement trajectories for unforeseen or untrained driving situations, leading to potential discomfort and safety issues for occupants and other road users.

Method used

A computer-implemented method utilizing a speech module, specifically a large language model, to determine and explain movement trajectories for vehicles. This method converts actual environment information into a semantic language model, allowing the speech module to plan and explain trajectories based on 'world knowledge' beyond traditional vehicle-specific training.

Benefits of technology

The method enhances the reliability and comfort of movement trajectories by leveraging broader knowledge domains, improving handling of unpredictable situations and providing clearer explanations to vehicle occupants.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

A computer-implemented method (100) for controlling a vehicle (400), comprising providing (110) an environment module (220) configured to determine an environment model based on environment information characterizing an environment of the vehicle (400); determining (120), by means of the environment module (220), an actual environment model of the vehicle (400) based on actual environment information of the vehicle (400); converting (130) the actual environment model into an actual language model, wherein the actual language model describes the environment of the actual environment model in a semantic language; providing (140) a language module (230) and inputting (150) the actual language model into the large language module (230); Using (160) the language module (230) to plan and / or explain a movement trajectory of the vehicle (400) based on the actual language model.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The invention relates to a computer-implemented method and a device for controlling a vehicle, a vehicle, a computer program product and a computer-readable storage medium.

[0002] At least partially, and especially fully, autonomous vehicles have a trained neural network designed to determine a vehicle's movement trajectory and associated control instructions. For this purpose, the vehicle's surroundings are also detected so that the movement trajectory can be determined based on this environment.

[0003] Training the neural network requires appropriate data sets, which ideally describe various driving situations. However, the data sets are usually limited to those recorded by vehicles. This means, among other things, that not all driving situations can be trained in advance. Especially in poorly trained or even untrained driving situations, it can be difficult for a neural network to plan a movement trajectory that is satisfactory for a vehicle occupant and / or other road users.

[0004] Because planning is performed by the neural network and is not influenced by control instructions from the driver or the vehicle's occupants, the planned movement trajectory can lead to misunderstandings and even a lack of driving comfort and safety. For example, an abrupt deceleration that is unpredictable for the vehicle occupant can cause them to be thrown forward and injured.

[0005] It is an object of the invention to at least improve one or more of the aforementioned disadvantages. In particular, it is an object of the invention to provide improved control and / or explanation of the movement trajectory for an at least partially, in particular fully, autonomously driving vehicle.

[0006] According to a first aspect, the object is achieved by a computer-implemented method for controlling a vehicle. The vehicle can be at least partially, in particular fully, autonomously driving. The method comprises the following steps: - Providing an environment module which is designed to determine an environment model based on environment information characterising an environment of the vehicle; - Determining, by means of the environment module, an actual environment model of the vehicle based on actual environment information of the vehicle; - Converting the current environment model into a current language model, whereby the current language model describes the environment of the current environment model in a semantic language; - Providing a language module and entering the actual language model into the language module; - Using the language module to plan and / or explain a vehicle movement trajectory based on the current language model.

[0007] The inventors have recognized that by using a speech module, particularly a large language module (Large Language Module or Model), so-called "world knowledge" can be used to control the vehicle. While regular neural networks are primarily trained on vehicle information, speech modules are trained and queried for a variety of purposes and are thus not limited to the vehicle domain. Accordingly, the speech module is trained in significantly more domains. Particularly in driving situations that are difficult or even impossible to predict, a speech module can provide a more satisfactory solution based on this "world knowledge." The speech module can be used to plan a movement trajectory and explain the movement trajectory, as well as to explain a movement trajectory planned by a planning module, particularly a neural network of the vehicle.

[0008] The speech module can be a trained speech module. A speech module can be defined as a speech module if it consists of more than approximately 100 million parameters.

[0009] The vehicle may include the trained neural network. The method may include providing the neural network with the environment module and / or additional modules. The neural network may further include a planning module for planning the movement trajectory and / or a perception module for providing the environment information. The neural network may include additional modules that are configured for and / or used for at least partially, in particular fully autonomously, controlling the vehicle.

[0010] The method may further comprise detecting the environment and providing the environmental information based on the detected environment. The detection and / or provision may be performed by means of the perception module.

[0011] The speech module can be designed to output a semantic output.

[0012] The language module can be a large language module and / or a large language module.

[0013] Planning the movement trajectory, in particular by means of the speech module, may further comprise providing a control instruction to a controller for controlling the vehicle. The controller may be comprised of the vehicle. The control instruction may be at least one of acceleration, steering, a vehicle position of the vehicle, a position of another road user or object, and a distance to one or the other road user or one or the object of the vehicle.

[0014] Planning, in particular using the speech module, of the movement trajectory may further include: - specifying a semantic action space to the speech module within which the movement trajectory is to be determined, wherein semantic actions of the action space are assigned to predetermined parameters for controlling the vehicle; - Determining a first semantic description with semantic actions based on the actual language model, wherein the first semantic description describes the movement trajectory planned by means of the language module; - Converting the actions of the first semantic description into the corresponding parameters of the action space; - Determining a control instruction based on the associated parameters.

[0015] The control instruction can be provided to the vehicle's control system.

[0016] The parameters can be the steering angle, acceleration, vehicle position, the position of the other road user or object, and / or the distance. Consequently, the parameters can be provided to the controller in the form of a control instruction.

[0017] Planning the movement trajectory, in particular by means of the language module, can further comprise inputting a first prompt task to the language module. The prompt task can be input by a user and / or generated automatically by the environment model. If the task is generated by the environment model, the prompt task can be sent to the language model using a calculation rule. The language model can be expandable with further predefined instructions. A description of the further instructions can be included in the prompt task. Previous and subsequent prompt tasks can be prompt engineering tasks. The first prompt task can characterize a first task formulated in the semantic language for the language module, wherein the first task characterizes the determination of the movement trajectory within the action space.Consequently, the speech module is asked to determine the movement trajectory using the first prompt task. The goal of prompt engineering can be to get the speech module to make one or more unambiguously interpretable statements, in this case regarding the movement trajectory.

[0018] The actual environmental information can be recorded sensorially, in particular by means of one or more sensors, and provided and / or read in via an interface.

[0019] An example of such an action space is described by Shalev-Shwartz, Shai, Shaked Shammah, and Amnon Shashua in their 2017 paper "On a formal model of safe and scalable self-driving cars." The action space can be a decision tree. Here, the alternative goals of the task can be assigned their own IDs. Within the goals, a precise quantification of the parameters can be specified based on a look-up table.

[0020] The conversion into and out of semantic language can be carried out as described by Kuo, Yen-Ling, et al., “Trajectory prediction with linguistic representations.” 2022 International Conference on Robotics and Automation (ICRA), IEEE, 2022.

[0021] The conversion into semantic language can be done using the environment module. The conversion from semantic language can be done using the language module.

[0022] Explaining the movement trajectory may further comprise providing output information for output to a vehicle occupant of the vehicle and / or an external unit, which output information characterizes an acoustic and / or visual explanation of the movement trajectory of the vehicle. The explanation may, in particular, be in semantic language. This output information may be output by means of an output unit of the vehicle and / or communicated to the external unit by means of a communication unit of the vehicle.

[0023] Explaining the movement trajectory can further include explaining the movement trajectory planned using the speech module. Consequently, the movement trajectory can be planned and explained using the speech module. The explanation is based on the current language model. Since the large speech module is familiar with the environment, it can incorporate it into the explanation. For example, a neighboring vehicle attempted to change into the lane of the driver's own vehicle, causing the driver's own vehicle to brake. The explanation can thus be provided by combining the current language model and the planned movement trajectory.

[0024] An actual model, for example, the actual environment model and / or the actual language model, can represent an actual situation and / or a current situation of the vehicle. Alternatively, the actual model can identify a model to be used to plan and / or explain the movement trajectory. The steps of the method can also be executed based on past environmental information and past movement trajectories, for example, to explain a past movement trajectory.

[0025] Alternatively and / or in addition to the planning of the movement trajectory by the large language module, the movement trajectory can be carried out by a planning module. The method can include: - Providing the planning module for planning movement trajectories; - Determine, using the planning module, the movement trajectory based on the actual environment model; - Converting the movement trajectory determined by the planning module into a second semantic description of the movement trajectory determined by the planning module; - Providing the second semantic description to the large language module, whereby the explanation of the movement trajectory is further based on the second semantic description.

[0026] The planning module can be part of the neural network or a planning module without artificial intelligence.

[0027] The perception module and / or the environment module can be part of the neural network or modules without artificial intelligence.

[0028] Explaining the movement trajectory may further comprise inputting a second prompt task to the large language module. The second prompt task may characterize a second task formulated in the semantic language to the large language module, wherein the second task characterizes explaining the movement trajectory planned by the planning module. Consequently, the large language module is prompted by the second prompt task to determine an explanation of the movement trajectory.

[0029] The explanation of the movement trajectory can be carried out and / or restricted to only taking place and / or taking place in particular when a critical driving situation arises. A critical driving situation can be an unmanageable driving situation and / or an out-of-distribution (OOD) case. The large speech module can communicate this situation, including the reason, to the driver or the external unit, for example a control center. The driver or the control center can then explicitly issue authorizations or tasks to a driving function of the vehicle via an interface, which lead to the critical driving situation being managed. Such critical driving situations can be saved and used for training or retraining of the driving function and / or the speech module.

[0030] The large speech module can be further trained based on recorded traffic scenarios and the first and / or second task. The motion trajectory can be generated using differentiable operations and fed as input to an objective function. Using this objective function, the weights of the speech module can be adjusted using backpropagation and thus fine-tuned based on the respective task.

[0031] According to a second aspect, the object is achieved by a device for controlling a vehicle, in particular an at least partially, preferably fully autonomously driving vehicle, comprising means for carrying out the method according to the first aspect. For this purpose, the device can comprise at least one processor, a memory and / or a communication unit. The processor, the memory and / or the communication unit can be referred to as a hardware component. The device can comprise a computer or be one which comprises one or more of the hardware components. A computer program product mentioned below can be stored on the memory. The device, in particular a hardware component of the device, can be designed to execute the computer program product.The communication unit can be configured to transmit and / or receive information, which in particular relates to information about the method according to the first aspect. The device, in particular the communication unit, can be configured to transmit the control instruction to the controller. Alternatively, the device can comprise the controller.

[0032] Method features described with respect to the method according to the first aspect may be embodied as device features of the device according to the second aspect.

[0033] According to a third aspect, the object is achieved by a computer program product comprising instructions which, when the program is executed by a hardware component of a computer, cause the computer to execute the method according to the first aspect when the program is loaded onto the hardware component and / or executed by the hardware component. Alternatively or additionally, the computer program product may comprise instructions which, when the program is executed by the device according to the second aspect, cause the device to execute the method according to the first aspect.

[0034] The object is achieved according to a fourth aspect by a computer-readable storage medium, comprising instructions which, when executed by a hardware component of a computer or the computer, cause the computer to execute the method according to the first aspect when the program is loaded onto the hardware component and / or executed by the hardware component. Alternatively or additionally, the storage medium can comprise instructions which, when the program is executed by the device according to the second aspect, cause the device to execute the method according to the first aspect.

[0035] According to a fifth aspect, the object is achieved by a vehicle. The vehicle can be an at least partially, in particular fully, autonomous vehicle. The vehicle comprises a device according to the second aspect and / or a memory for storing the computer program product according to the third aspect and a processor for executing the computer program product. The memory and / or the processor can be referred to as a hardware component. The vehicle can comprise a computer, wherein the computer comprises at least one of the memory and the processor.

[0036] Preferred embodiments are explained using the accompanying figures. They show: Fig. 1 a schematic representation of a computer-implemented method for controlling a vehicle; Fig. 2 a schematic representation of the method for planning a movement trajectory and explaining it; Fig. 3 a schematic representation of the method for explaining a movement trajectory planned by means of a planning module; Fig. 4 is a schematic representation of a device for controlling a vehicle; and Fig. 5 a vehicle with such a device.

[0037] In the figures, identical or essentially functionally identical or similar elements are designated by the same reference numerals.

[0038] Fig. 1 shows a schematic representation of a computer-implemented method 100 for controlling a vehicle 400 (see Fig. 5).

[0039] The method 100 may be stored in the form of a computer program product.

[0040] The method 100 may include providing a perception module 210. The perception module 210 is configured to provide actual environment information characterizing the actual environment based on an actual environment detected by sensors of the vehicle 400 or a current environment.

[0041] The method 100 comprises providing 110 an environment module 220 which is designed to determine an environment model based on the environment information characterizing the environment of the vehicle 400.

[0042] The perception module 210, the environment module 220, and / or a subsequently mentioned planning module 250 can be part of a neural network for planning movement trajectories of the vehicle 400. Alternatively, individual modules 210, 220, 250 can be provided without artificial intelligence and / or can be provided outside the neural network.

[0043] The method 100 further comprises determining 120, by means of the environment module 220, an actual environment model of the vehicle 400 based on actual environment information of the vehicle 400.

[0044] In order to be able to provide the information from the current environment model as input to a subsequently named language module 230, here a large language module 230 (English: "Large Language Model"), the method 100 further comprises converting 130 the current environment model into a current language model, wherein the current language model describes the environment of the current environment model in a semantic language. Consequently, the conversion can be understood as translating the current environment model into the semantic language.

[0045] The method 100 further comprises providing 140 the large language module 230 and inputting 150 the actual language model into the large language module 230.

[0046] The method 100 further comprises using 160 the large language module 230 to plan and / or explain a movement trajectory of the vehicle 400 based on the actual language model. In particular, planning the movement trajectory may further comprise providing a control instruction SA to the controller 240 for controlling the vehicle 400. The controller 240 may be included in the vehicle 400. The control instruction SA may, in particular, be based on the movement trajectory. The control instruction SA may be configured such that the vehicle 400 is controlled according to the movement trajectory.

[0047] Consequently, the movement trajectory can be planned and explained using the large language module 230. Alternatively, the movement trajectory can be explained solely using the large language module 230, for example, if it is explained by the planning module 250, see Fig. 3, was planned. By providing the current language model, which semantically describes the current environment, the large language module 230 is provided with information about the current environment. With the knowledge of the current environment, the large language module 230 can then plan the movement trajectory and output a semantic description of the movement trajectory.

[0048] Fig. 2 shows the planning of the movement trajectory using the large speech module 230. The arrows of the Fig. 2 and Fig. 3 an information flow. The environment module 220 converts the actual environment model into the actual language model and outputs it to the large language module 230. Planning the movement trajectory using the large language module 230 can include specifying a semantic action space to the large language module 230, within which the movement trajectory is to be determined. Semantic actions of the action space are assigned to predetermined parameters for controlling the vehicle. These parameters can, for example, characterize a steering angle, a vehicle position of the vehicle, and / or a distance to another road user. Thus, the semantic actions, in particular the associated parameters, can be transferred into the control instruction.

[0049] To this end, the method 100 may further comprise inputting a first prompt task to the large language module 230, wherein the first prompt task characterizes a first task formulated in the semantic language to the large language module. The first and / or each further prompt task may include and / or characterize the action space. The first task characterizes the determination of the movement trajectory within the action space. For example, the first prompt task may be: "You are the driver of a vehicle. Your speed is 50 km / h with an acceleration of 0 m / s^2, while you are in the center of the lane. You are approaching an intersection where a cyclist is approaching quickly from the right (cyclist's speed 30 km / h; deceleration 1 m / s^2, center of the lane). You intend to cross the intersection. There are no traffic lights or signs. There is only one lane in each direction. The speed limit is 50 km / h. Solve the task using the following semantic action space. The semantic action space defines 1) lateral and 2) longitudinal goals, as well as 3) the degree of aggressiveness required to achieve them. 1) Lateral goals are desired positions in the lane coordinate system (e.g., "My goal is to be in the center of lane number 2"). 2) There are three types of longitudinal goals. The first goal is a relative position and speed with respect to other vehicles (e.g., "My goal is to be behind car number 3, at the same speed and within 2 seconds of it"). The second goal is a speed goal (e.g., "Drive at the speed limit for this road times 110%). The third goal is a speed limit at a specific location (e.g., when approaching an intersection: "speed of 0 at the stop line" or when negotiating a sharp curve: "speed of no more than 60 km / h at a specific location on the curve").For the third objective, a “speed profile” can be used instead (some discrete points on the route and the desired speed at each of these points). A reasonable number of lateral targets is limited by 16 = 4 × 4 (4 positions in at most 4 relevant lanes). A reasonable number of longitudinal targets of the first type is limited by 8 × 2 × 3 = 48 (8 relevant cars, whether in front or behind them, and 3 relevant distances). A reasonable number of absolute speed targets is 10 (0%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90%, 100% of the maximum speed), and a reasonable upper limit for the number of speed limits is 2 (0 at stop lines; maximum 60 km / h at sharp bends). To reach a specific longitudinal or lateral target, one must first accelerate and then decelerate (or vice versa). Target aggressiveness is the maximum (expressed in absolute terms) acceleration / deceleration required to reach the target. Determine the combination between the longitudinal and lateral targets (e.g., "Start with the lateral target, and right in the middle, start using the longitudinal target as well"). You can specify a range of blending and three levels of aggressiveness (0 m / s^2, 4 m / s^2, 8 m / s^2 for longitudinal targets and 0 m / s^2, 1 m / s^2, 3 m / s^2 for lateral targets). Blending means they can change their target five times until they have solved the task. Provide a precise sequence of the next five objectives, with quantified values ​​and levels of aggressiveness to be followed and when to mix them. Do not impede traffic with overly cautious driving maneuvers. Mixing is not an order and should therefore not be mentioned in the sequence of objectives.

[0050] The first prompt task may further include: "At the end of your answer, provide a list of IDs for the next action, where the IDs are generated according to the following rule: The first number represents the target type: - 0: Relative position in relation to other vehicles - 1: Speed ​​target - 2: Speed ​​limit - 3: Desired position in the roadway coordinate system If we have the target with ID 0, three numbers follow. The following numbers are defined by: ID: 1-8 (vehicle ID with relative position) + 0-1 (0: in front / 1: behind) + 0-2 (relevant distances: 0: 3m, 1: 10m, 2: 50m) If we have the target with ID 1, there is a following number. The following numbers are defined by: ID: 0-10(0: 0%,1: 10%,2: 20%,3: 30%,4: 40%,5: 50%,6: 60%,7: 70%,8: 80%,9: 90%,10: 100% of maximum speed) If we have the target with ID 2, there is a following number. The following numbers are defined by: ID: (0: at stop lines; 1: maximum 60 km / h in sharp curves) If we have the target with ID 3, there are two following numbers. The following numbers are defined by: ID: 0-3 (lane ID; 0: current lane, 1: lane to the right of the current lane, 2: lane to the left of the current lane, 3: lane to the right of the right lane) + 0-3 (positions; 0: center, 1: 20 cm to the right of the lane center, 2: 20 cm to the left of the lane center, 40 cm to the left of the lane center). The last number of each target ID is the acceleration in m / s^2 (0 m / s^2, 4 m / s^2, 8 m / s^2 for longitudinal gates and 0 m / s^2, 1 m / s^2, 3 m / s^2 for transverse gates). At the end, a string of target IDs separated by semicolons should be created. A target ID consists of several unseparated numbers. Only one ID can be specified per target.

[0051] Here, the IDs can correspond to the parameters and / or actions of the action space. The large language module 230 can then generate a sequence of actions that can be translated into a parameterized movement trajectory and, through it, into steering and acceleration commands or control instructions SA.

[0052] Thus, planning the movement trajectory can further include: - determining a first semantic description with semantic actions based on the language model, wherein the first semantic description describes the movement trajectory planned by means of the large language module; - Converting the actions of the first semantic description into the corresponding parameters of the action space; - Determining a control instruction SA based on the associated parameters.

[0053] To explain the movement trajectory, the large speech module 230 may further comprise providing output information AI for output to a vehicle occupant of the vehicle 400 and / or an external unit, which characterizes an acoustic and / or visual explanation of the movement trajectory of the vehicle.

[0054] For this purpose, a further second prompt task can be issued to the large speech module 230, which requests the large speech module 230 to justify the movement trajectory.

[0055] The controller 240 can control the vehicle 400 based on the control instruction SA.

[0056] Fig. 3 shows the method 100 in which, compared to the Fig. 2 the movement trajectory is adopted by a planning module 250. The method 100 comprises: - Providing the planning module 250 for planning movement trajectories; - Determining, by means of the planning module 250, the movement trajectory based on the actual environment model; - converting the movement trajectory determined by means of the planning module 250 into a second semantic description of the movement trajectory determined by means of the planning module 250; - Providing the second semantic description to the large language module 230, wherein the explanation of the movement trajectory is further based on the second semantic description.

[0057] Thus, the language module 230, in particular the large language module 230, is provided with the actual language model as well as the second semantic description of the movement trajectory planned by the planning module 250. Based on this information and an associated prompt task, the large language module 230 can be requested to determine an explanation of the planned movement trajectory. The output information A1 is a human-understandable explanation of the planned movement trajectory of the autonomous vehicle 400.

[0058] By means of the proposed method 100, safer and more comfortable movement trajectories and also explanations of these movement trajectories can be provided.

[0059] Fig. 4 shows a schematic representation of a device 300 for controlling a vehicle 400. The device 300 comprises a memory 310, a processor 320, and a communication unit 330. The device 300 can be implemented in a vehicle 400. A computer program product can be stored on the memory 310, comprising instructions which, when executed by the processor 320, cause the device 300 to carry out the method 100. Information can be sent and / or received by means of the communication unit 330. The communication unit 330 can be used to send the movement trajectory determined by means of the speech module 230, in particular the large speech module 230, in particular in a suitable description transferred to the controller 240, for example in the form of the control instruction SA, to the controller 240.

[0060] Fig.Figure 5 shows a schematic representation of a vehicle 400 with a device 300. Alternatively or additionally, the vehicle 400 may include the controller 240, a processor, and a memory. The memory is configured to store the computer program product, and the processor is configured to execute the computer program product. Reference symbol 100 Computer-implemented method for controlling a vehicle 110 Providing an environment module 120 Determining an actual environment model 130 Converting the current environment model into a current language model 140 Providing a language module 150 Entering the actual language model into the large language module 160 Using the speech module to plan and / or explain a movement trajectory 210 Perception module 220 environment module 230 large speech module 240 Control All output information SA tax instruction 250 Planning Module 300 Device for controlling a vehicle 310 memory 320 processor 330 communication unit 400 vehicles QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] Shalev-Shwartz, Shai, Shaked Shammah, and Amnon Shashua. “On a formal model of safe and scalable self-driving cars” from 2017

[0019] Kuo, Yen-Ling, et al. „Trajectory prediction with linguistic representations.“ 2022 International Conference on Robotics and Automation (ICRA), IEEE aus 2022

[0020]

Claims

[1] Computer-implemented method (100) for controlling a vehicle (400), comprising: Providing (110) an environment module (220) configured to determine an environment model based on environment information characterizing an environment of the vehicle (400); Determining (120), by means of the environment module (220), an actual environment model of the vehicle (400) based on actual environment information of the vehicle (400); Converting (130) the actual environment model into an actual language model, wherein the actual language model describes the environment of the actual environment model in a semantic language; Providing (140) a language module (230) and inputting (150) the actual language model into the large language module (230); Using (160) the language module (230) to plan and / or explain a movement trajectory of the vehicle (400) based on the actual language model. [2] The method (100) of claim 1, wherein planning the movement trajectory further comprises providing a control instruction to a controller (240) for controlling the vehicle (400). [3] Method (100) according to one of the preceding claims, wherein planning the movement trajectory further comprises: Predetermining a semantic action space to the large language module (230) within which the movement trajectory is to be determined, wherein semantic actions of the action space are assigned to predetermined parameters for controlling the vehicle (400); Determining a first semantic description with semantic actions based on the actual language model, wherein the first semantic description describes the movement trajectory planned by means of the language module (230); Converting the actions of the first semantic description into the corresponding parameters of the action space; Determining a control instruction (SA) based on the associated parameters. [4] Method (100) according to one of the preceding claims, wherein planning the movement trajectory further comprises inputting a first prompt task to the large speech module (230), where the first prompt task characterizes a first task formulated in the semantic language to the large language module (230), The first task characterizes the determination of the movement trajectory within the action space. [5] Method (100) according to one of the preceding claims, wherein explaining the movement trajectory further comprises providing output information (AI) for output to a vehicle occupant of the vehicle (400) and / or an external unit, which characterizes an acoustic and / or visual explanation of the movement trajectory of the vehicle (400). [6] The method (100) according to any one of the preceding claims, wherein explaining the movement trajectory further comprises explaining the movement trajectory planned by means of the speech module (230). [7] Method (100) according to one of claims 1 to 5, further comprising: Providing a planning module (250) for planning movement trajectories; Determining, by means of the planning module (250), the movement trajectory based on the actual environment model; Converting the movement trajectory determined by means of the planning module (250) into a second semantic description of the movement trajectory determined by means of the planning module (250); Providing the second semantic description to the large language module (230), where the explanation of the movement trajectory is further based on the second semantic description. [8] Method (100) according to one of the preceding claims, wherein explaining the movement trajectory further comprises inputting a second prompt task to the large speech module (230), wherein the second prompt task characterizes a second task formulated in the semantic language to the large language module (230), wherein the second task characterizes the explanation of the movement trajectory planned by means of the planning module (250). [9] Method (100) according to one of the preceding claims, wherein the explanation of the movement trajectory occurs when a critical driving situation exists. [10] Device (300) for controlling a vehicle (400), comprising means (310; 320; 330) for carrying out the method (100) according to one of claims 1 to 9. [11] A computer program product comprising instructions which, when the program is executed by a hardware component of a computer, cause the hardware component to execute the method (100) according to any one of claims 1 to 9 when the program is loaded onto and / or executed by the hardware component. [12] A computer-readable storage medium comprising instructions which, when executed by a hardware component of a computer, cause the hardware component to execute the method (100) according to any one of claims 1 to 9 when the program is loaded onto and / or executed by the hardware component. [13] Vehicle (400), comprising: a device (100) according to claim 10; and / or a memory for storing the computer program product according to claim 11 and a processor for executing the computer program product.

Citation Information

Patent Citations

  • E2e learning-based evaluator for an autonomous driving vehicle

    EP4082856A2

Cited By

  • Vehicle parking method, electronic equipment and storage medium

    CN121106198A