Method and apparatus for determining the cause of an error in vehicles using large language models

Abstract

The invention relates to a method for diagnosing a vehicle system of a vehicle, having the following steps: - providing (S1) a system configuration of the vehicle system at least with the program code of the software functions implemented in at least one control device; - providing (S2) a sequence of vehicle events during past operation of the vehicle; - evaluating (S3) the system configuration and sequence of vehicle events using a trained large language model to obtain the cause of an error or a predicted error, wherein the large language model is trained on the basis of a plurality of automotive and non-automotive program codes and a plurality of sequences of vehicle events for different vehicles; and - carrying out (S4) error handling as a function of the cause of the error or of the predicted error.

Description

[0001] Description

[0002] title

[0003] Method and device for determining the cause of a fault in vehicles using large language models

[0004] Technical area

[0005] The invention relates to the diagnosis of vehicles, and in particular to the detection of fault causes after errors or undesirable behavior have occurred.

[0006] Technical background

[0007] Locating the cause of a fault after detecting anomalies or undesirable behavior in a vehicle is generally difficult and is often performed intuitively by a workshop employee or with guided troubleshooting tools. As a result, determining the cause of the fault largely depends on the experience of the workshop employee or the troubleshooting tools available.

[0008] A vehicle system typically comprises a number of control units connected to one another via a communication link. The control units are used to acquire state variables via sensors and to control actuators, including position sensors and drive motors, in order to implement control functions or locomotion functions. The vehicle functions are thus generally realized by the hardware architecture and the software functions implemented in the control units. If an error, an anomaly, or other undesirable behavior occurs, the cause of the error may be the failure of a hardware element or faulty software code. Determining the cause of the error largely depends on the experience of the software developer or debugger.

[0009] For example, the "GitHub Co-Pilot" is known for a-priori detection of errors in software code. It supports a programmer in programming a software code and points out errors.

[0010] Disclosure of the invention

[0011] According to the invention, a method for determining the cause of a fault in a vehicle according to claim 1 and a corresponding device according to the independent claim are provided.

[0012] Further embodiments are specified in the dependent claims.

[0013] According to a first aspect, a method for diagnosing a vehicle system of a vehicle is provided, comprising the following steps:

[0014] Providing a system configuration of the vehicle system at least with the program code of the software functions implemented in at least one control unit;

[0015] Providing a sequence of vehicle events during the past operation of the vehicle;

[0016] - Evaluating the system configuration and the sequence of vehicle events using a trained large language model to obtain a fault cause or a predicted fault, wherein the large language model is trained based on a variety of automotive and non-automotive program codes and a variety of sequences of vehicle events for different vehicles;

[0017] Perform error handling depending on the cause of the error or the predicted error.

[0018] Furthermore, the system configuration may further include a description of a hardware configuration of the vehicle system. In addition to providing the program code of a system configuration of the vehicle system, a build configuration of the software function may also be provided. This allows errors that are not found in the software logic but are caused by combinations of compilers and / or dependent software libraries to be detected.

[0019] Troubleshooting may include a software update or patch, changing configuration settings, or limiting at least one feature.

[0020] Error handling can include a priori warnings about potential errors, allowing various program codes, value pairs, and the like, that led to problems to be identified during development. Furthermore, error handling can also include root-cause analysis, which identifies the cause of an actual error.

[0021] Furthermore, the large language model can be trained with training data that includes the following information: a large number of generally available program codes for software functions not used in vehicles; a large number of program codes for software functions used in different vehicle types; a large number of descriptions of hardware configurations with regard to control units, components and bus systems; a large number of bus system protocols and message formats on vehicle buses of different vehicle types; a large number of sequences or time series of vehicle events for different vehicles; a large number of bug fixes for generally available program codes, i.e. which locations were repaired in the past and in what way to enable a transfer to existing program code.

[0022] Vehicle systems typically comprise a multitude of control units, making troubleshooting very difficult when an anomaly or error occurs. These control units can be microcontroller-based or microprocessor-based (e.g., vehicle computers) and have varying levels of performance. The complexity of this system lies primarily in the fact that, in addition to all possible hardware-based error causes, each control unit implements software functions that result in a multitude of interactions and dependencies that are impossible to oversee. Troubleshooting such a system requires a comprehensive understanding of the network and its dependencies, so debugging requires a thorough understanding of both the hardware and software functions and their interactions.

[0023] Existing approaches to locating errors and root causes are based on diagnostic events, which contain valuable information about system performance, errors, and operational statistics. Using specialized software tools, these diagnostic events can be analyzed to detect errors and anomalies that indicate system failures.

[0024] Large language models (LLMs) have recently become popular with the development of ChatGPT. Large language models make it possible to analyze language in context and generate new texts. Such LLMs can also evaluate abstract descriptions. For example, LLMs can be used to diagnose diseases based on symptom descriptions.

[0025] The ability of LLMs to generate context-dependent findings based on symptom descriptions or abstract descriptions can also be used for troubleshooting in vehicles. According to the above method, it is intended to predict a possible fault cause and future vehicle events based on a specified software code of one or more control units of a control unit system in a vehicle, a hardware description of the control units, and the associated components and vehicle events.

[0026] In other words, the system configuration specifications in the form of the hardware configuration and the implemented software functions, as well as the vehicle events that occurred, can be used for evaluation in a trained large-language model to determine a possible cause of the failure and / or predict expected failure events. The vehicle events can include operational events, repair events, error events, and diagnostic events.

[0027] Vehicle events can also include dumps or a sequence of vehicle states or states of the respective software functions, particularly variable values ​​or values ​​communicated via protocols. The corresponding data is either stored internally on the control units or could be, for example, CAN or Ethernet bus dumps or time series of dumps.

[0028] The operating events include all events that characterize the driver-specific operation, such as driving at high speed above a speed threshold and for more than a predetermined driving time, an impact due to driving on uneven ground or due to hitting an environmental object with an acceleration effect above a predetermined threshold, an environmental condition such as rain, a very low or very high outside temperature (each defined by exceeding or falling below limit values) for a predetermined minimum period of time, the times and extent of software updates, the input and modification of presets by the driver, the triggering of the alarm system, a restricted view of an outside camera and the like.

[0029] Error events can be errors detected by an automatic monitoring function in the control units. These are stored in an error log and, if necessary, transmitted to a central unit, where the chronological sequence of errors that occurred is logged. Furthermore, error events can also be errors detected and logged through driver observations.

[0030] Repair events correspond to events of a specific repair, replacement of a vehicle part or the like, which are carried out, for example, when the vehicle is taken to a workshop for repair or routine maintenance.

[0031] In addition, tests can be performed during a workshop visit that lead to results that can be provided as diagnostic events or diagnostic results. The diagnostic results can include, for example, battery voltage measurements, error log readings, and the like. The repair measures are usually also stored electronically and are thus available as vehicle events for the individual vehicle.

[0032] The LLM can be pre-trained with general training data and vehicle-specific training data to learn context dependencies. In particular, the LLM can be trained with a large amount of general program code, including error descriptions and bug fixes, as well as with program code for software functions implemented in vehicles, such as those used in ECUs of various vehicles, so that the peculiarities of vehicle software and the specific behavior of the ECUs in vehicles and the vehicle system are learned.

[0033] Furthermore, hardware configurations and various possible formats of data communication between the control units, such as in the form of a CAN matrix and the like, can be specified.

[0034] Vehicle events, operational events, diagnostic events, vehicle status data, sequences of communication signals on vehicle buses and the like from different vehicles can also be used as training data.

[0035] By training the LLM with a combination of the program code, the software functions, especially the bug fixes and the vehicle events, it is possible to find the causes of errors in software functions or hardware components and to predict vehicle events.

[0036] Furthermore, the large language model can be implemented as a foundation model, so that the register or memory dumps can also be processed in the form of image data. In this way, large data volumes can be considered. According to a further aspect, a device for implementing the above method is provided.

[0037] Brief description of the drawings

[0038] Embodiments are explained in more detail below with reference to the attached drawings. They show:

[0039] Figure 1 is a schematic representation of a vehicle system with a plurality of control units connected to other components via a vehicle bus; and

[0040] Figure 2 is a flowchart illustrating a method for finding a cause of a fault and / or predicting possible vehicle events.

[0041] Description of embodiments

[0042] Figure 1 shows a vehicle system with a plurality of control units 2, which are connected to further components 3, such as sensors, actuators and the like.

[0043] The control units 2 and the components 3 communicate via a vehicle bus 4, such as CAN or the like. Software functions described by program code are implemented in the control units 2. One or more of the control units 2 can record vehicle events, which may include operational events, error events, and the like.

[0044] Figure 2 describes a method in which a large language model can be used to predict the cause of an error that has occurred or potentially occurring error events.

[0045] For this purpose, a description of the vehicle system is first provided in step S1. Vehicle system 1 is described in the form of the program code of the software functions of control units 2, as well as the hardware configuration of control units 2, the communication via the vehicle bus, and the components connected to control units 2.

[0046] Furthermore, in step S2, known vehicle events in the form of operating events, error events, repair events, and diagnostic events can be provided. Furthermore, vehicle events can include the provision of a register or memory dump or a sequence of vehicle states or states of software functions, in particular variable values ​​or values ​​communicated via protocols. The data is either stored internally on the control units or could be, for example, CAN or Ethernet bus dumps or time series of dumps.

[0047] The information thus obtained is fed to a diagnostic system via a suitable user interface in step S3. The diagnostic system is implemented on a conventional data processing device and may include cloud services. A large-language model is implemented in the diagnostic system.

[0048] Large Language Models such as GPT-4 (Generative Pre-trained Transformer) are complex AI systems designed to understand and generate natural language. They are trained using machine learning and use a transformer architecture consisting of many layers of neurons. These models are trained with large amounts of text data to learn language patterns and structures. Such models can be used for a wide variety of applications, including automatic text generation, translation, chatbots, speech recognition, text analysis, and much more. They can also assist with tasks such as question-answering systems or text writing.

[0049] The information obtained in steps S1 and S2 is evaluated using the large language model

[0050] The large language model is appropriately trained to indicate, as a result of the information provided, a current error cause in case an error has occurred previously or to predict a possible error that will occur with a high probability.

[0051] The large language model can be designed as a unified model or as an ensemble model.

[0052] To enable the evaluation of state data and variables with the LLM, a state vector can be generated, for example, representing raw data from vehicle events, such as dumps. Such a state vector can then be "translated" into text for evaluation in the Large Language Model. For example, a data item from a specific packet must be assigned its meaning, so that a text such as "The temperature of the control unit is 120 °C" can be generated.

[0053] Furthermore, a Foundation model can be used as a large language model, so that sound data or image data can also be processed.

[0054] This can be signaled accordingly in step S4 or processed in an error handling unit to repair the error. For example, the error can be addressed through a software update or patch, by making configuration settings, or by limiting functions, such as limiting a maximum speed. Furthermore, software development can also be notified of possible error causes so that bug fixes can be generated.

[0055] The training of the large language model can be carried out with training data comprising the following information: a large number of generally available program codes for software functions not used in vehicles; a large number of program codes for software functions used in different vehicle types; a large number of descriptions of hardware configurations with regard to control units, components and bus systems; a large number of bus system protocols and message formats on vehicle buses of different vehicle types; a large number of sequences or time series of vehicle events for different vehicles.

[0056] A plurality of dumps or a sequence of vehicle states or states of the respective software functions, in particular variable values ​​or values ​​that are communicated via protocols, are provided;

[0057] Error descriptions and root-cause analyses including references or data of bug fixes in the software and update descriptions;

[0058] Build and toolchain parameters such as the versions of the compilers used.

[0059] The training can be carried out in a manner known for language models.

Claims

Claims 1. Computer-implemented method for diagnosing a vehicle system of a vehicle, comprising the following steps: Providing (S1) a system configuration of the vehicle system at least with the program code of the software functions implemented in at least one control unit; Providing (S2) a sequence of vehicle events during the past operation of the vehicle; Evaluating (S3) the system configuration and the sequence of vehicle events using a trained large language model to obtain a fault cause or a predicted fault, wherein the large language model is trained based on a variety of automotive and non-automotive program codes and a variety of sequences of vehicle events for different vehicles; and performing (S4) fault handling depending on the fault cause or the predicted fault.

2. The method of claim 1, wherein the system configuration further comprises a description of a hardware configuration of the vehicle system.

3. The method according to claim 1 or 2, wherein vehicle events comprise at least one of operational events, error events, repair events and diagnostic events, wherein in particular 4. The method according to any one of claims 1 to 3, wherein the error handling comprises a software update or patch, a change of configuration settings or a limitation of at least one function.

5. The method according to any one of claims 1 to 4, wherein the large language model, which is in particular designed as a foundation model, is trained with training data comprising the following information: a variety of generally available program codes for software functions not used in vehicles; a variety of program codes for software functions used in different vehicle types; a variety of descriptions of hardware configurations with regard to control units, components and bus systems; a variety of bus system protocols and message formats on vehicle buses of different vehicle types; a variety of sequences or time series of vehicle events for different vehicles; Numerous error descriptions with root cause analysis, bug fixes in the program code and comments.

6. Apparatus for carrying out one of the methods according to one of claims 1 to 5.

7. A computer program product comprising instructions which, when the program is executed by at least one data processing device, cause the device to carry out the steps of the method according to one of claims 1 to 5.

8. A machine-readable storage medium comprising instructions which, when executed by at least one data processing device, cause the device to carry out the steps of the method according to one of claims 1 to 5.

Images