Method and device for operating a vehicle

Abstract

In a procedure for operating a vehicle (102), the vehicle's (102) driving situation is determined. Driving situation data corresponding to this situation is generated. This driving situation data is processed using a pre-trained generative transformer to generate evaluation data that corresponds to a legal and / or moral assessment of the driving situation. Based on this evaluation data, at least one functional unit (104) of the vehicle (102) is controlled.

Description

[0001] The invention relates to a method for operating a vehicle. The invention further relates to a device for operating a vehicle.

[0002] Decision-making methods are known from the state of the art, particularly for fully autonomous vehicles. These methods allow, for example, maneuver or trajectory planning based on the vehicle's sensor data. Known methods are usually based on predefined rules that define vehicle behavior depending on specific driving situations.

[0003] US Patent No. 10,127,214 B2 discloses a method for analyzing court decisions that uses a natural language model trained on legal texts. It is proposed that this method can be implemented in a vehicle computer.

[0004] Regarding the prior art, reference is also made to US 2023 / 0109379 A1, which discloses a diffusion-based method for generating data based on large language models. Furthermore, reference is made to CN 115115913 A, which discloses a data processing method using large language models.

[0005] The object of the invention is to provide a method and a device for operating a vehicle which can react particularly flexibly to a driving situation of the vehicle.

[0006] This object is achieved by a method having the features of claim 1 and by a device having the features of the independent device claim. Advantageous further developments are specified in the dependent claims.

[0007] In the proposed method for operating a vehicle, a driving situation of the vehicle is determined. Driving situation data corresponding to the driving situation is generated. The driving situation data is processed using a generative, pre-trained transformer to generate evaluation data corresponding to a legal and / or moral evaluation of the driving situation. Based on the evaluation data, at least one functional unit of the vehicle is controlled.

[0008] First, the driving situation is classified appropriately. The classification of the driving situation is described in more detail below in connection with embodiments of the method. The result of the classification is then used as a prompt for the generative pre-trained transformer to obtain the legal and / or moral assessment of the driving situation. With the help of the generative pre-trained transformer, the driving situation can then be interpreted in a manner that is as close as possible to a human interpretation. In the proposed method, the generative pre-trained transformer thus takes the place of a human road user who legally and / or morally assesses the driving situation. Based on the assessment, the vehicle can then be controlled, for example, or a decision-making aid can be provided to the driver.The advantage of the proposed method is that an assessment of the driving situation by the generative pre-trained transformer is particularly flexible, especially in comparison with, for example, a rule-based assessment.

[0009] The driving situation is recorded, for example, using vehicle sensors. The recording of the driving situation is described in more detail below in connection with embodiments of the method. The method can, in particular, record the course of the driving situation, for example, for a limited period of time or related to a section of road, such as an intersection, a roundabout, or an entrance or exit ramp.

[0010] The driving situation is characterized in particular by at least one of the following parameters: a spatial distribution of other road users; at least one movement parameter of the other road users; a spatial distribution of immobile objects in the vicinity of the vehicle; a relative position to certain types of traffic signs, lane markings, and traffic lights; information about the vehicle's right of way; information about an action of another road user in the vicinity of the vehicle, for example honking, flashing headlights, pushing, overtaking the vehicle, or an overtaking attempt. Furthermore, the driving situation can be characterized by one or more parameters related to relevant traffic regulations, traffic signs, right of way, traffic lights, and / or traffic light phases.

[0011] The vehicle is, in particular, a motor vehicle that can be controlled by the driver. However, the vehicle can also be a fully autonomous vehicle. In particular, it can also be an unmanned vehicle.

[0012] The generative pre-trained transformer is also called a generative pre-trained transformer or GPT. The generative pre-trained transformer can also be a neural network with a corresponding architecture. Specifically, the generative pre-trained transformer is trained using training data that includes text information related to: - Legal information, in particular traffic law, laws, regulations, court rulings, precedents, data and / or protocols of previous traffic occurrences and their evaluation, which were prepared in particular by legally qualified persons; and / or - Moral information, publications, in particular specialist literature, opinions of commissions, journalistic publications, posts on social media or the Internet, concerning moral principles and / or precedents.

[0013] Through specific retraining, the generative pre-trained transformer can be adapted to the specific laws and morals of certain countries, societies, legal systems, and / or groups of people. Furthermore, the generative pre-trained transformer can be adapted to the personal preferences of the vehicle driver. For example, the driver can set a higher weighting for the moral aspect compared to legal and / or material aspects. The driver can also, for example, set a higher weighting for the protection of animals that may be encountered in road traffic.

[0014] The assessment can, for example, determine a probable distribution of legal and / or moral guilt or liability. Furthermore, the assessment can determine consequences of the driving situation, for example from the perspective of or for the driver and / or operator of the vehicle. In particular, the consequences can be costs. The assessment can also predict possible unlawful and / or immoral actions, in particular driving actions, in particular actions of the driver and / or a driver assistance system of the vehicle. Furthermore, a legal and / or moral report can be generated. The report can, in particular, be stored in a coded and / or encrypted form with the relevant data, for example time value, position value, characteristics and / or identification of one or more parties involved in the driving situation.

[0015] In one embodiment, at least one piece of sensor information related to the driving situation is acquired. Sensor data corresponding to the sensor information is generated. The sensor data is processed to generate the driving situation data. The sensor information can be acquired, in particular, by a sensor of the vehicle. Examples of vehicle sensors are mentioned below in connection with exemplary embodiments. Based on the sensor data, a model of the vehicle's environment can be created, which represents the driving situation. The evaluation can then be carried out very reliably based on the environmental model.

[0016] In another embodiment, the sensor data is processed using a tokenizer to generate the driving situation data in the form of tokens. The term "token" originates from the field of artificial intelligence and refers to an information element. Using the tokenizer, a sequence of tokens is created from a sequence of sensor data, which can then be processed particularly easily by the generative pre-trained transformer. Alternatively or additionally, a natural language text can be generated from the sensor data and fed to the generative pre-trained transformer as an input (prompt).

[0017] In a further embodiment, the driving situation is predicted at least based on the sensor data. The driving situation data are generated according to the predicted driving situation. In this embodiment, the evaluation is based on a predicted future driving situation, i.e., a driving situation that will occur with a certain probability. This allows the driving situation to be evaluated proactively, i.e., before it has occurred. As a result, by controlling the vehicle depending on the evaluation, measures can be taken, in particular, to bring about a legally and / or morally desirable outcome of the driving situation.

[0018] In a further embodiment, the sensor information comprises at least one image with a depiction of the surroundings of the vehicle. The sensor data comprises image data that corresponds to the image. In particular, a two-dimensional image is captured using a camera of the vehicle, which in particular captures light in the optical spectrum and / or in the infrared spectrum. Capturing light in the infrared spectrum has the additional advantage that the camera can capture the image even in poor lighting conditions, for example at night. Alternatively, a three-dimensional image of the surroundings of the vehicle can be captured, in particular in the form of a point cloud. A time-of-flight camera, a RADAR system, or a LIDAR system of the vehicle can be used for this purpose. Furthermore, an image sequence comprising successive images with a depiction of the surroundings of the vehicle can be captured. In such an embodiment, the image data corresponds to the image sequence.From the image sequence, reliable models of the vehicle's environment can be extracted, from which the driving situation can be determined.

[0019] In a further embodiment, the image data is processed using the generative pre-trained transformer or another generative pre-trained transformer operating as an image-to-text transformer to generate at least a portion of the driving situation data. Based on the image data, tokens and / or human language text are generated that correspond to the semantic content of the image. Based on this information, the driving situation can be more accurately determined.

[0020] In a further embodiment, communication data is received from at least one other road user and / or from traffic infrastructure. The communication data is processed to generate at least part of the driving situation data. The communication can take place in particular via Dedicated Short Range Communication (DSRC), for example Car-2-Car or Car-2-X. For example, an intention can be received from the other road user, for example whether a vehicle ahead is braking. Situational context can be provided from the traffic infrastructure, for example whether a traffic light signal change is imminent. The generative pre-trained transformer can consider more information based on the communication data and thus enable a more reliable assessment.

[0021] In a further embodiment, at least one output unit of the vehicle is controlled based on the evaluation data in order to inform the driver of the vehicle about the result of the evaluation of the driving situation. The output can be provided, in particular, visually, via a display unit of the vehicle, or audibly, for example, as a generated voice output. The result of the evaluation serves as a decision-making aid for the driver. For example, the driver can consider the likely legal and / or moral consequences of their actions in their decision. Alternatively or additionally, several decision variants can be provided to the driver.

[0022] In another embodiment, the functional unit is only activated based on the evaluation data once the driver has made a confirming input. By outputting the result, the driver is informed of possible consequences and can decide for themselves whether a suggested action should be taken. This supports the driver without being patronizing.

[0023] In another embodiment, the control of the functional unit based on the evaluation data can be interrupted and / or aborted by a corresponding input from the driver. In this embodiment, a suggested action is taken. However, the driver can interrupt or abort this action to take control of the vehicle themselves. This also supports the driver without being patronizing.

[0024] In a further embodiment, at least one functional unit of the vehicle that influences the movement of the vehicle, in particular a drive system, a braking system, and / or a steering system, is controlled based on the evaluation data. In this embodiment, the movement of the vehicle is controlled depending on the result of the evaluation. For example, the vehicle can be braked or induced to swerve to avoid negative legal and / or moral consequences.

[0025] In a further embodiment, at least one sensor unit of the vehicle is controlled based on the evaluation data. For example, sensors of the vehicle can be controlled to detect the surroundings of the vehicle and / or the vehicle interior. In particular, this can involve data recording on an increased scale and / or with a higher level of detail, for example a more extensive video recording, audio recording and / or recording of the status of electronic systems of the vehicle. For example, a 60 fps or fps recording or a wide-angle recording can be activated for a time interval. The data recorded in this step can be used in particular for evidentiary purposes, for example to clarify liability in the event of an accident.

[0026] In a further embodiment, at least one control command for assisted, automated, or autonomous driving of the vehicle is generated based on the evaluation data. Based on the legal and / or moral assessment of the driving situation, a driver assistance system can be activated, for example, to support the driver in the driving situation and help minimize negative legal and / or moral consequences. In an automated or fully autonomous vehicle, trajectory planning or maneuver planning can also be performed based on the legal and / or moral assessment of the driving situation.

[0027] In this document, assisted driving refers to automation levels 1 and 2 according to the Federal Highway Research Institute. Automated driving refers to automation levels 2, 2+, and 3 according to the Federal Highway Research Institute. Autonomous driving refers to automation levels 4 and 5 according to the Federal Highway Research Institute. Driver assistance systems within the meaning of this document include, for example, adaptive cruise control, lane keeping assist, parking assist, longitudinal guidance assist, or a combination of the aforementioned driver assistance systems, such as traffic jam assist.

[0028] The invention further relates to a device for operating a vehicle. The device comprises a determination unit configured to determine a driving situation of the vehicle and generate driving situation data corresponding to the driving situation, and a processing unit configured to process the driving situation data using a generative pre-trained transformer to generate evaluation data corresponding to a legal and / or moral evaluation of the driving situation. The device further comprises a control unit configured to control at least one functional unit of the vehicle based on the evaluation data.

[0029] The device has the same advantages as the claimed method. In particular, the device can be further developed with the features of the dependent claims directed to the method. Furthermore, the method described above can be further developed with features described in this document in connection with the device.

[0030] Embodiments of the invention are explained in more detail below with reference to the figures, in which: Fig. 1 shows a schematic representation of a device for operating a vehicle; and Fig. 2 a flowchart of a method for operating the vehicle.

[0031] Fig. 1 shows a schematic representation of a device 100 for operating a vehicle 102.

[0032] The device 100 is used to legally and / or morally evaluate a driving situation of the vehicle 102. To perform the evaluation, the device 100 detects and interprets the driving situation. The driving situation is then evaluated according to legal and / or moral criteria using a generative, pre-trained transformer. Based on the legal and / or moral evaluation, at least one functional unit 104 of the vehicle 102 is then controlled. The functional unit 104 can, in particular, be an output unit of the vehicle 102, with the aid of which a driver of the vehicle 102 is informed of the result of the evaluation. The functional unit 104 can also be a drive system, a braking system and / or a steering system, i.e., the movement of the vehicle 102 is influenced based on the evaluation.

[0033] The device 100 comprises a detection unit 106 for detecting at least one piece of sensor information related to the driving situation. To detect the sensor information, the detection unit 106 comprises, in particular, sensors 108, 110 of the vehicle 102. In the Fig. In the embodiment shown in Figure 1, the detection unit 106 comprises, purely by way of example, a camera 108 of the vehicle 102, which is arranged and configured to capture an image depicting an area outside the vehicle 102. The camera 108 generates image data corresponding to the image. As an alternative to an optical camera that captures a two-dimensional image of the surroundings of the vehicle 102, the detection unit 106 can also comprise a RADAR system or LIDAR system, each of which can capture a three-dimensional image of the surroundings of the vehicle 102. The detection unit 106 also comprises a distance sensor 110, which is configured to detect a distance between the vehicle 102 and other road users or stationary objects in the surroundings of the vehicle 102. The distance sensor 110 can be configured, for example, as an ultrasonic sensor or as a RADAR system or LIDAR system.The distance sensor 110 generates distance data corresponding to the distance. The image data and the distance data are provided by the acquisition unit 106 as sensor data.

[0034] The detection unit 106 is configured to process the sensor data to determine the driving situation and to generate driving situation data corresponding to the driving situation. In one embodiment, the detection unit 106 can further be configured to predict the driving situation based at least on the sensor data, ie, to determine one or more driving situations likely to occur in the future. In such an embodiment, the driving situation data corresponds to the predicted driving situation.

[0035] To generate the driving situation data, the acquisition unit 106 uses, in particular, a tokenizer that generates a sequence of tokens from a sequence of sensor data that can be processed by the generative pre-trained transformer. The tokens form at least part of the driving situation data. Alternatively or additionally, the sensor data can also be processed using the aforementioned generative pre-trained transformer or another generative pre-trained transformer to generate the driving situation data, for example, in the form of a natural language text. In particular, the acquisition unit 106 can be configured to process the image data using the generative pre-trained transformer or another generative pre-trained transformer that operates as an image-to-text transformer.This allows the acquisition unit 106 to generate a text from the image data that reflects the semantic content of the image. This text can form at least part of the driving situation data.

[0036] The device 100 further comprises a processing unit 112, which is shown purely by way of example as part of a central control unit 114 (ECU) of the vehicle 102. The processing unit 112 is configured to process the driving situation data using the generative pre-trained transformer to generate evaluation data corresponding to the legal and / or moral evaluation of the driving situation. The central control unit 114 also comprises a control unit 116 configured to control the functional unit 104 of the vehicle 102 depending on the driving situation data.

[0037] The central control unit 114 further comprises a communication interface 118 for communicating with other road users or traffic infrastructure, for example, a traffic light system. The central control unit 114 can receive communication data from the other road users or the traffic infrastructure and provide it, for example, to the detection unit 106 and / or the processing unit 112. For example, the detection unit 106 can use the communication data to predict the driving situation. The communication data can, in particular, also be part of the driving situation data.

[0038] Fig. 2 shows a flowchart of the method for operating the vehicle 102.

[0039] The method can be carried out in particular by the central control unit 114 of the vehicle 102, which uses Fig. 1. Alternatively, any other computing unit of the vehicle 102 can be used to calculate the Fig. 1 to control the device 100 described.

[0040] The method is started in step S200. In step S202, the detection unit 106 is controlled to detect the driving situation. In particular, the sensor information related to the driving situation is detected and the sensor data is generated. For example, the camera 108 captures the image of the surroundings of the vehicle 102 and generates the image data. In step S204, the detection unit 106 generates the driving situation data. For example, the detection unit 106 processes the sensor data using the tokenizer to generate at least a portion of the driving situation data in the form of tokens. In another example, the detection unit 106 processes the image data using the further generative pre-trained transformer, which is operated as an image-to-text transformer, to generate at least a portion of the driving situation data in the form of natural language text.In step S206, the processing unit 112 then processes the driving situation data using the generative pre-trained transformer to generate the evaluation data.

[0041] In step S208, the functional unit 104 of the vehicle 102 is controlled based on the legal and / or moral assessment performed in step S206. For example, an output unit of the vehicle 102 is controlled to output the result of the assessment to the vehicle driver. The output can be in the form of text and / or pictograms, for example in the form of text or images generated by the generative pre-trained transformer as at least part of the assessment data. However, the output can also be audible, for example as a computer-generated voice output generated by the generative pre-trained transformer as at least part of the assessment data. Alternatively or additionally, in step S208, a functional unit 104 related to the movement of the vehicle 102 is controlled to perform a maneuver and / or adapt the trajectory of the vehicle 102 according to the assessment of the driving situation.Optionally, the maneuver and / or the trajectory adjustment are only performed if the driver has previously provided a confirming input. Alternatively or additionally, the driver can pause and / or abort the process by making a corresponding input. The method is terminated in step S210.

[0042] In the case of the Fig. 1 and Fig. 2, at least the detection unit 106, the processing unit 112 and the control unit 116 form the device 100 for operating a vehicle 102. Further in the Fig. 1 and Fig. Elements and features shown in Figure 2 and mentioned in the preceding description may be part of the claimed device 100. Likewise, method steps described with reference to the device 100 may be part of the claimed method. List of reference symbols 100 Device 100 102 vehicles 104 functional unit 106 registration unit 108 Camera 110 Distance sensor 112 processing unit 114, 116 control unit 118 Communication interface QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] US 10,127,214 B2

[0003] US 2023 / 0109379 A1

[0004] CN 115115913 A

[0004]

Claims

[1] Method for operating a vehicle (102), in which a) a driving situation of the vehicle (102) is determined and driving situation data corresponding to the driving situation are generated; b) the driving situation data are processed using a generative pre-trained transformer to generate evaluation data corresponding to a legal and / or moral evaluation of the driving situation; and c) at least one functional unit (104) of the vehicle (102) is controlled on the basis of the evaluation data. [2] Method according to claim 1, wherein at least one piece of sensor information related to the driving situation is detected, sensor data corresponding to the sensor information is generated, and the sensor data is processed to generate the driving situation data. [3] The method of claim 2, wherein the sensor data is processed using a tokenizer to generate the driving situation data in the form of tokens. [4] Method according to claim 2 or 3, wherein the driving situation is predicted at least on the basis of the sensor data and the driving situation data are generated according to the predicted driving situation. [5] Method according to one of claims 2 to 4, wherein the sensor information comprises at least one image with a depiction of the surroundings of the vehicle (102) and the sensor data comprises image data corresponding to the image. [6] The method of claim 5, wherein the image data is processed using the generative pre-trained transformer or another generative pre-trained transformer operating as an image-to-text transformer to generate at least a portion of the driving situation data. [7] Method according to one of the preceding claims, wherein communication data are received from at least one further road user and / or from traffic infrastructure and the communication data are processed to generate at least part of the driving situation data. [8] Method according to one of the preceding claims, wherein on the basis of the evaluation data at least one output unit of the vehicle (102) is controlled in order to inform a driver of the vehicle (102) about the result of the evaluation of the driving situation. [9] Method according to claim 8, wherein the control of the functional unit (104) on the basis of the evaluation data only takes place when the vehicle driver has made a confirming input. [10] Method according to claim 8 or 9, wherein the control of the functional unit (104) on the basis of the evaluation data can be interrupted and / or aborted by a corresponding input from the vehicle driver. [11] Method according to one of the preceding claims, wherein on the basis of the evaluation data at least one functional unit (104) of the vehicle (102) influencing the movement of the vehicle (102) is controlled, in particular a drive system, a braking system and / or a steering system. [12] Method according to one of the preceding claims, wherein at least one sensor unit of the vehicle (102) is controlled on the basis of the evaluation data. [13] Method according to one of the preceding claims, wherein at least one control command for assisted, automated or autonomous driving of the vehicle (102) is generated on the basis of the evaluation data. [14] Device (100) for operating a vehicle (102), comprising a determination unit configured to determine a driving situation of the vehicle (102) and to generate driving situation data corresponding to the driving situation; a processing unit (112) configured to process the driving situation data using a generative pre-trained transformer to generate evaluation data corresponding to a legal and / or moral evaluation of the driving situation; and a control unit (116) which is designed to control at least one functional unit (104) of the vehicle (102) on the basis of the evaluation data.

Images