Computer-implemented method for identifying an erratic system behavior of a vehicle, and information technology system

Abstract

The invention relates to a computer-implemented method for identifying an erratic system behavior of a vehicle by way of a machine learning model. The computer-implemented method according to the invention is characterized by the following method steps: - recording the signals (1) transmitted via a field bus of a test vehicle during operation of the test vehicle and / or processed by a computing unit in the test vehicle in the form of first timeseries data (ZRD1); - training a machine learning model with the first timeseries data (ZRD1); - generating artificial timeseries data (KZRD) by way of the machine learning model; - recording the signals (1) transmitted via a field bus of a fleet vehicle during operation of the fleet vehicle and / or processed by a computing unit in the fleet vehicle in the form of second timeseries data (ZRD2); - transmitting the second timeseries data (ZRD2) from the fleet vehicle to a central computing device; - comparing the artificial timeseries data (KZRD) with the second timeseries data (ZRD2) by way of the central computing device; and - outputting a warning signal by way of the central computing device if the artificial timeseries data (KZRD) and the second timeseries data (ZRD2) differ from one another by more than a defined degree of deviation.

Description

[0001] Mercedes-Benz Group AG

[0002] Computer-implemented method for identifying erratic system behavior of a vehicle and information technology system

[0003] The invention relates to a computer-implemented method for identifying erratic system behavior of a vehicle using a machine learning model and to an information technology system for executing the method.

[0004] Software-based technology in vehicles can become the target of hacking attacks or manipulation. For example, so-called chip tuning or vehicle coding can be performed, which fundamentally alters the vehicle's system technology. This can increase performance or reduce fuel consumption. However, chip tuning can cause the vehicle to no longer behave according to its specifications or to fail certifications. In the worst-case scenario, the vehicle can even be damaged and become a safety risk. Such manipulations can also damage other vehicle components, such as control units connected to a manipulated processing unit. Therefore, it is desirable to be able to detect such manipulations of the vehicle's system technology.

[0005] A device for detecting engine tuning in a vehicle is known from KR 102356315 B1. Using suitable sensors, various parameters, such as the engine's intake pressure, are monitored. The device then checks whether these parameters correspond to standard values. If deviations occur, engine tuning is detected. A disadvantage of this method is that dedicated components for detecting engine tuning must be installed in the vehicle.

[0006] WO 2016 / 046819 A1 also describes a procedure for detecting cyberattacks in vehicles.

[0007] Furthermore, DE 102021 208 988 A1 discloses the detection of malfunctions in a vehicle based on the use of artificial intelligence. Using suitable environmental sensors, such as cameras or laser scanners, objects in the vehicle's vicinity are detected and classified. Object attributes are assigned to the corresponding objects. Information about the objects and their attributes, as well as their temporal behavior, is passed to an artificial learning system for training, enabling the learning system to learn typical object behavior. This allows the artificial learning system to identify malfunctions in the corresponding sensor-based object recognition.

[0008] The present invention is based on the objective of providing means by which it is possible to reliably detect manipulations of the system technology in the vehicle.

[0009] According to the invention, this problem is solved by a computer-implemented method for identifying erratic system behavior of a vehicle using a machine learning model with the features of claim 1. Advantageous embodiments and further developments, as well as an information technology system suitable for carrying out the method, are described in the dependent claims.

[0010] A generic computer-implemented method for identifying erratic system behavior of a vehicle using a machine learning model is further developed according to the invention by the following method steps:

[0011] - Recording the signals transmitted via a fieldbus of the test vehicle during operation and / or processed by a computing unit in the test vehicle in the form of initial time series data;

[0012] - Training a machine learning model with the initial time series data so that the machine learning model is able to artificially generate time series for the signals encompassed by the initial time series data while adhering to a specified tolerance level;

[0013] - Generating artificial time series data through the machine learning model, encompassing artificially generated time series of signals through the machine learning model;

[0014] - Recording the signals transmitted via a fleet vehicle's fieldbus during operation and / or processed by a computing unit in the fleet vehicle in the form of secondary time series data; - Transferring the secondary time series data from the fleet vehicle to a central computing facility;

[0015] - Comparison of the art time series data with the second time series data, by the central computing facility; and

[0016] - Issuing a warning signal by the central computing unit if the artificial time series data and the second time series data differ from each other by a defined deviation measure.

[0017] The method according to the invention makes it possible to identify erratic system behavior of a vehicle through the use of artificial intelligence-based algorithms. Such erratic system behavior can indicate unauthorized manipulation of the vehicle's system technology. Countermeasures can then be initiated accordingly. This ensures the reliable operation of the vehicle and maintains road safety.

[0018] The test vehicle allows for the generation of initial time series data. This initial data enables the description of the typical system behavior of a vehicle. The signals transmitted and / or processed by the vehicle can be of a diverse nature.For example, these are signals from a wide variety of sensors, such as temperature readings, mileage, speed, traction battery charge level, volume flow, mass flow, rotational speed, acceleration, steering angle, engine performance data, engine fuel consumption data, status information from a vehicle control unit, route information from a navigation system, information from the vehicle's infotainment system such as the selected radio station, information derived from a vehicle occupant's voice input, the active driving program, states of actuators such as the vehicle's brakes or accelerator pedal, and the like. These signals can be accessed via the respective fieldbus and / or read from a processing unit in the vehicle.The vehicle can have various fieldbus systems, such as CAN, Flex-Ray, LIN, Byteflight, Ethernet and the like.

[0019] The individual signals are aggregated over time, creating time series. These signal time series are then combined to form the initial time series data. This initial time series data is used as training data for the machine learning model. The training of the machine learning model can take place either in the vehicle or externally, particularly on the aforementioned central computing unit.

[0020] Training the machine learning model enables it to recognize patterns in the signal time series, ultimately allowing it to generate corresponding time series itself. The machine learning model is trained until the artificially generated time series match the actual signal time series recorded in the test vehicle within a defined tolerance range. This tolerance range can be specified by the vehicle manufacturer, for example. Signals can differ in two ways. First, a specific signal can have a different amplitude for the same point in time. Second, a characteristic value of a signal may be present in the initial time series data and the artificial time series data at different points in time.This can lead to a temporal difference as well as a difference in the value of a given signal. A signal-specific tolerance value is then defined for each type of deviation.

[0021] After sufficient training of the machine learning model, it is used to generate corresponding artificial time series data. This artificial time series data then serves as a reference for comparison with corresponding second time series data generated in the fleet vehicles. The computer-implemented procedure is then used to verify the system behavior of the fleet vehicles. For this purpose, the central computing unit compares the artificial time series data with the second time series data. Established data comparison methods can be used for this. In particular, artificial intelligence can also be employed. This comparison specifically checks not only for overlaps between the respective time series of signals, but also for the dependencies between the different signals. For example, a fully depressed accelerator pedal can lead to a typical longitudinal acceleration value.This longitudinal acceleration value is then established after a characteristic time period. If the fleet vehicle exhibits erratic system behavior, this longitudinal acceleration value may be higher or lower and / or occur after a shorter or longer time period than usual. The precise point at which such erratic system behavior occurs is specified for each signal by the defined deviation measure. This deviation measure can also be defined by the vehicle manufacturer for each specific signal. Alternatively, the deviation measure can be defined by the machine learning model itself. In this case, the machine learning model defines the signal-specific deviation measure based on the insights it gains from analyzing the initial time series data.

[0022] If erratic system behavior is detected, the central computing unit issues a warning signal. Depending on the warning signal issued, various subsequent reactions can be triggered, which will be discussed in more detail below.

[0023] In general, it would also be possible to directly compare the initial time series data with the subsequent time series data to identify erratic system behavior. However, the machine learning model allows for the generation of extensive datasets, providing a particularly large dataset in the form of artificial time series data for comparison. This eliminates the effort required to manually generate a sufficiently large initial time series dataset. Furthermore, there is always the risk that the fleet vehicle exhibits permissible system behavior that is not reflected in the initial time series data. By artificially generating such data using the machine learning model, these values ​​can also be produced, thus reducing the risk of a false positive detection of erratic system behavior.Furthermore, this makes it possible to define the deviation measure by the machine learning model itself, which, due to the superior pattern recognition capabilities of the machine learning model, enables a particularly reliable differentiation between erratic system behavior and permissible system behavior.

[0024] An advantageous further development of the method according to the invention provides for the use of a large language model as a machine learning model. Large language models are also referred to as "Large Language Models" (LLMs). Due to their ability to artificially generate information, large language models are particularly suitable for use in the method according to the invention. In particular, large language models are capable of reading and outputting corresponding time series data. For example, such time series can be formulated in the form of a table or a vector. For instance, a timestamp can be entered in each row, and the respective value of a signal can be entered in each column.

[0025] Preferably, the central computing unit integrates the large language model by issuing a prompt to the model to compare the art time series data with the secondary time series data. This prompt allows interaction with the large language model. The large language model can thus be used not only to artificially generate art time series data but also to compare it with the secondary time series data. Such a prompt could, for example, be: "What is the probability that the data deviation is due to data manipulation?" Since the large language model has been trained to artificially generate corresponding art time series data, it is also able to reliably identify such deviations between the art time series data and the secondary time series data.This allows for a more reliable identification of erratic system behavior and thus corresponding manipulations of the vehicle system components.

[0026] A further advantageous embodiment of the method according to the invention provides that a machine learning model pre-trained for processing time series is used. For example, this can be a so-called "foundation model" for time series data. The correspondingly pre-trained machine learning model or artificial neural network is already characterized by a high level of training for recognizing patterns in time series data, which allows the sufficient training to be completed using a smaller initial time series dataset.

[0027] According to a further advantageous embodiment of the method according to the invention, it is further provided that several machine learning models, dependent on the configuration of the test vehicle, are trained. To generate artificial time series data for the fleet vehicle, such a machine learning model is used, the training of which is based on initial time series data recorded from a test vehicle whose configuration matches that of the fleet vehicle. The time series of signals recorded in a respective vehicle can depend on the respective configuration of that vehicle. For example, if a more powerful engine is installed in a vehicle, correspondingly higher performance characteristics such as engine power, longitudinal acceleration, driving speed, and the like can be achieved.Accordingly, adapted machine learning models are trained for the various vehicle configurations. This allows for the generation of even more accurate artificial time series data for each vehicle and the identification of corresponding deviations between the artificial time series data and the secondary time series data.

[0028] To differentiate between differently trained machine learning models, these can be sorted into groups. Corresponding test vehicles or fleet vehicles can also be assigned to these groups. By comparing the artificial time series data generated for a test vehicle with the initial time series data recorded in the test vehicle, the quality of the machine learning model adapted to the vehicle's configuration can be verified.

[0029] A further advantageous embodiment of the method according to the invention further provides that the central computing unit, by issuing the warning signal, causes the following:

[0030] - a workshop appointment for the fleet vehicle is automatically booked;

[0031] - a remote diagnostic test is performed for the fleet vehicle;

[0032] - a warning message is issued in the fleet vehicle; and / or

[0033] - a vehicle function in the fleet vehicle is restricted or blocked.

[0034] If the warning signal is triggered, this indicates unauthorized manipulation of the vehicle system components. For example, the program code of a control unit's software may have been manipulated, additional devices may have been installed in the vehicle to manipulate signals exchanged via the relevant fieldbuses, and so on. Appropriate countermeasures must then be taken.

[0035] The central computer system can be connected to an online appointment booking system for workshops. Each fleet vehicle owner can have a user account with the vehicle manufacturer. This account can contain the owner's address or a preferred workshop. Using the address, a nearby workshop can be found, or the preferred workshop can be selected for booking an appointment. This allows the vehicle owner to conveniently schedule a workshop visit to rectify any tampering with their vehicle.

[0036] Additionally or alternatively, the central computing unit can also initiate remote diagnostics. For this purpose, fleet vehicles and the central computing unit can exchange data in a proven manner. For example, a fleet vehicle might be equipped with a telecommunications unit that allows the vehicle to connect to the internet via mobile network, Wi-Fi, or similar technology. The central computing unit can also be connected to the internet and, in this context, can also be referred to as a cloud server. Corresponding diagnostic data can then be transmitted via the internet from a given fleet vehicle to the central computing unit for analysis. This makes it possible to investigate the cause of erratic system behavior even more precisely. The analysis of the diagnostic data can be performed manually by a developer, assisted, or fully automated by a computer system.

[0037] The warning information can also be used to inform the vehicle owner or driver about the detected tampering. This warning information can be transmitted visually, audibly, and / or haptically. For example, a warning message can be displayed on a screen in the fleet vehicle, a warning tone can sound in the vehicle, and / or an actuator such as a piezoelectric actuator or a motor can be activated to, for example, transmit vibrations to the steering wheel. It is also possible for the respective vehicle owner or driver to be logged into their user account via an application running on a mobile device such as a smartphone. The mobile device can also be connected to the fleet vehicle, for example, via a wired USB cable or wirelessly, such as via Wi-Fi, Bluetooth, NFC, or similar technologies.Warning information can also be displayed on mobile devices. Furthermore, after detecting potential manipulation of vehicle system components, a vehicle function can be restricted or blocked to protect the fleet vehicle. For example, the fleet vehicle's performance could be illicitly increased through manipulation, causing vehicle components to operate outside their permitted operating range, resulting in excessive wear or damage. In such cases, the vehicle functions that affect the operating range of the respective components can be restricted or blocked. This prevents premature aging or damage to the components until the manipulation has been removed from the vehicle. Afterward, the respective vehicle functions can be reactivated and made available for unrestricted use.

[0038] According to a further advantageous embodiment of the method according to the invention, only those signals in the form of time series are processed which do not allow any conclusions to be drawn about personal information. This ensures data protection. For example, time series associated with route navigation can generate personal information, as this could allow conclusions to be drawn about the place of residence of the respective driver.

[0039] A further advantageous embodiment of the method according to the invention further provides that

[0040] - if the artificial time series data and the second time series data differ from each other by a defined outlier measure, where the outlier measure is larger than the deviation measure, the central computing unit adjusts the generation behavior of the machine learning model so that the machine learning model generates time series of the signals with a higher variance;

[0041] - the central computing facility generates outlier artificial time series data using the adapted machine learning model;

[0042] - the central computing unit compares the second time series data with the outlier artificial time series data; and

[0043] - the central computing unit refrains from issuing the warning signal if the difference between the comparison of the second time series data with the outlier artificial time series data is smaller than the difference between the comparison of the second time series data with the artificial time series data.

[0044] The differences detected between the artificial time series data and the secondary time series data may not only be due to erratic behavior resulting from manipulation of the vehicle's systems, but also to outliers in the underlying signals. Such outliers are characterized by a particularly large deviation from the standard behavior. Accordingly, the central processing unit can check whether there are differences between the artificial time series data and the secondary time series data that exceed the defined outlier threshold. This can be an indication of such outliers. In this case, the warning signal should be suppressed, as there is actually no manipulation of the vehicle's system components.

[0045] To validate outlier detection, the central computing unit generates artificial outlier time series data. If the machine learning model is a large language model, the so-called "temperature value" of the generation process can be adjusted to produce time series data with high variance or a high random component. This can lead to the large language model incorporating disproportionately high values ​​into the time series data, a phenomenon also known as "hallucinating" in the context of large language models.

[0046] By comparing the second time series data with the outlier artificial time series data, it can then be verified whether these are actually outliers or whether there is a manipulation of the vehicle system components.

[0047] An information technology system according to the invention for carrying out a method described above is characterized by a central computing unit comprising at least one machine learning model trained with first time series data from a test vehicle and a computer program product, the computer program product comprising machine-interpretable instructions which, when executed by a processor of the central computing unit, cause it to generate artificial time series data using the machine learning model, to compare second time series data received from a fleet vehicle with the artificial time series data and to output a warning signal depending on the comparison.

[0048] Thus, the invention relates not only to the method for identifying erratic system behavior of a vehicle, but also to the hardware and software components required for this purpose.

[0049] Further advantageous embodiments of the computer-implemented method according to the invention for identifying erratic system behavior of a vehicle also result from the exemplary embodiments which are described in more detail below with reference to the figures.

[0050] This shows:

[0051] Fig. 1 a diagram showing the course of signals transmitted via a vehicle's fieldbus and / or processed by a vehicle-internal computing unit in the form of time series;

[0052] Fig. 2 is a diagram showing the course of signals artificially generated by a machine learning model in the form of time series;

[0053] Fig. 3 is a diagram showing the difference between the signals from the diagrams shown in Fig. 1 and Fig. 2; and

[0054] Fig. 4 shows a flowchart of a method according to the invention for identifying erratic system behavior of a vehicle.

[0055] Modern vehicles, such as passenger cars, rely on the processing of signals by computing units, for example, in the form of control units. Using a "drive-by-wire" system, a desired power output can be requested from the vehicle's engine based on the detected accelerator pedal position. This can involve, for example, controlling a pump to deliver liquid fuel and increase the fuel flow rate injected into the engine, a throttle valve control unit to open the throttle valve, and similar components. Thus, during vehicle operation, a wide variety of signals are transmitted via the vehicle's fieldbus system and processed by the respective computing units installed in the vehicle. Figure 1 shows an example of a qualitative representation of the magnitude or characteristics of such signals 1, plotted against time t in the form of time series.Figure 1 shows so-called first time series data ZRD1, recorded in a test vehicle, and second time series data ZRD2, recorded in a fleet vehicle. The first time series data ZRD1 and the second time series data ZRD2 can be identical or differ from each other. It is possible that vehicle system components, particularly the respective control units and / or the software running on them, are manipulated. Such manipulation can occur as part of a cyberattack or, for example, through so-called chip tuning. The system behavior of a correspondingly manipulated vehicle will differ from the operating mode intended by the vehicle manufacturer. A method according to the invention for identifying such erratic system behavior is based on the use of artificial intelligence.For this purpose, a machine learning model is trained using the initial time series data ZRD1 recorded in the test vehicle, enabling the machine learning model to artificially generate corresponding time series data in the form of artificial time series data KZRD. The artificial time series data KZRD generated by the machine learning model are shown in Figure 2.

[0056] The aim is to investigate whether a fleet vehicle exhibits erratic system behavior. For this purpose, second time series data (ZRD2) are recorded in the fleet vehicle during operation and compared with the artificial time series data (KZRD) by a central computing unit. The difference resulting from this comparison is shown in Figure 3. The corresponding signals 1 can have a time offset from each other, indicated by corresponding horizontal arrows, and / or differ in their amplitude, indicated by a corresponding vertical arrow. Signal-specific limit values ​​can be defined for each signal, representing the deviations that can occur between the second time series data (ZRD2) and the artificial time series data (KZRD) during normal operation of the fleet vehicle. The sum of these limit values ​​is referred to as the deviation measure.As soon as at least one signal 1 exceeds this deviation threshold, the central computing unit interprets this as erratic system behavior, which is an indication of corresponding unauthorized manipulation of the vehicle system components. The central computing unit then issues a warning signal. The process of the method according to the invention is illustrated again with reference to Figure 4. In step 401, the signals 1 transmitted via the fieldbus of the test vehicle during operation and / or processed by a computing unit in the test vehicle are recorded in the form of initial time series data ZRD1.

[0057] In step 402, the machine learning model is trained with the first time series data ZRD1, for example by the central computing facility.

[0058] In step 403, the machine learning model generates art time series data KZRD.

[0059] In step 404, the signals 1 transmitted via the fleet vehicle's fieldbus during operation and / or processed by a computing unit in the fleet vehicle are recorded in the form of second time series data ZRD2.

[0060] In step 405, the second time series data ZRD2 is transferred from the fleet vehicle to the central computing unit. In step 406, the central computing unit compares the artificial time series data KZRD with the second time series data ZRD2.

[0061] Finally, in step 407, the central computing unit issues a warning signal if the artificial time series data KZRD and the second time series data ZRD2 differ from each other by the aforementioned defined deviation measure.

[0062] The method according to the invention makes it possible to reliably detect, with minimal technical effort, whether vehicle system components of the fleet vehicle have been manipulated. By initiating appropriate countermeasures after the warning signal is issued, potential damage to the fleet vehicle can be prevented. Furthermore, road safety can be increased, as it prevents such manipulated fleet vehicles from behaving dangerously in traffic.

Claims

Mercedes-Benz Group AG Patent claims 1. Computer-implemented method for identifying erratic system behavior of a vehicle using a machine learning model, characterized by the following process steps: - Recording the signals transmitted via a fieldbus of the test vehicle during operation and / or processed by a computing unit in the test vehicle (1) in the form of first time series data (ZRD1); - Training a machine learning model with the first time series data (ZRD1) so that the machine learning model is able to artificially generate time series for the signals (1) encompassed by the first time series data (ZRD1) while adhering to a specified tolerance measure; - Generating artificial time series data (ATD) by the machine learning model, comprising artificially generated time series of the signals (1) by the machine learning model; - Recording the signals transmitted via a fieldbus of the fleet vehicle and / or processed by a computing unit in the fleet vehicle (1) in the form of second time series data (ZRD2); - Transfer of the second time series data (ZRD2) from the fleet vehicle to a central computing facility; - Comparison of the art time series data (KZRD) with the second time series data (ZRD2) by the central computing unit; and - Issuing a warning signal by the central computing unit if the artificial time series data (KZRD) and the second time series data (ZRD2) differ from each other by a defined deviation measure.

2. The method according to claim 1, characterized in that a large language model (LLM) is used as a machine learning model.

3. Method according to claim 2, characterized in that the central computing unit integrates the large language model (LLM) by issuing an input request to the large language model (LLM) to compare the art time series data (KZRD) with the second time series data (ZRD2).

4. Method according to one of claims 1 to 3, characterized in that a machine learning model pre-trained for processing time series is used.

5. Method according to one of claims 1 to 4, characterized in that several machine learning models dependent on the configuration of the test vehicle are trained, wherein such a machine learning model is used to generate artificial time series data (KZRD) for the fleet vehicle, the training of which is based on first time series data (ZRD1) drawn from a test vehicle whose configuration matches that of the fleet vehicle.

6. Method according to any one of claims 1 to 5, characterized in that the central computing device causes the following by issuing the warning signal: - a workshop appointment for the fleet vehicle is automatically booked; - a remote diagnostic test is performed for the fleet vehicle; - a warning message is issued in the fleet vehicle; and / or - a vehicle function in the fleet vehicle is restricted or blocked.

7. Method according to one of claims 1 to 6, characterized in that only such signals (1) in the form of time series are processed which do not allow any conclusions to be drawn about personal information.

8. Method according to any one of claims 1 to 7, characterized in that - if the artificial time series data (KZRD) and the second time series data (ZRD2) differ from each other by a defined outlier measure, where the outlier measure is larger than the deviation measure, the central computing unit adjusts the generation behavior of the machine learning model so that the machine learning model generates time series of the signals (1) with a higher variance; - the central computing facility generates outlier artificial time series data using the adapted machine learning model; - the central computing unit compares the second time series data (ZRD2) with the outlier artificial time series data; and - the central computing unit refrains from issuing the warning signal if the difference between the comparison of the second time series data (ZRD2) with the outlier artificial time series data is smaller than the difference between the comparison of the second time series data (ZRD2) with the artificial time series data (KZRD).

9. Information technology system for executing a method according to one of claims 1 to 8, characterized by a central computing unit comprising at least one machine learning model trained with first time series data from a test vehicle and a computer program product, the computer program product comprising machine-interpretable instructions which, when executed by a processor of the central computing unit, cause it to generate artificial time series data (KZRD) using the machine learning model, to compare second time series data (ZRD2) received from a fleet vehicle with the artificial time series data (KZRD) and to output a warning signal depending on the comparison.

Images