METHOD FOR PROCESSING LOG FILES, DATA PROCESSING SYSTEM AND VEHICLE

Description

[0001] The invention relates to a method for processing log files with computer-aided data processing systems according to the type defined in more detail in the preamble of claim 1, a corresponding data processing system and a vehicle.

[0002] The fundamental prerequisite for the correct operation of an information technology system is the flawless interaction of hardware and software. Due to a hardware defect or errors in the program code, also known as a bug, the correct functioning of an information technology system can be limited, faulty, or even impossible.

[0003] For this reason, log files are generated, particularly during the development of hardware or software components, but also during the ongoing operation of an information technology system. These log files record the processes performed by the information technology system. Analyzing these log files often makes it possible to identify the cause of the error and implement appropriate measures to correct the error, such as adapting the program code.

[0004] The following problems occur in connection with processing log files: 1. Depending on the design of the information technology system and the software used to monitor the processes running on the information technology system, also known as a debugger, only incomplete log files are generated after an error occurs. These log files do not cover all relevant system parameters and / or do not cover the entire relevant time period. Often, only the most important parameters are included in a time window close to the error that occurred, for example, the last minute before the error occurred. A comprehensive problem analysis may not be possible if the cause of the error can be traced back to a system parameter that is not logged, especially outside the specified time window. 2. Log files can contain a very large number of system parameters, especially for long observation periods.Locating a specific system parameter at a specific point in time can then be very labor-intensive. Even the use of search algorithms can often only lead to the desired result after a lengthy search, since locating a relevant section of the log file requires an analysis of the system behavior stored and logged in the log file. Thus, despite the use of search and / or sorting algorithms, manual effort is required by the user. 3. Log files are often only retained for a running information technology system, for example, in a volatile storage medium such as random access memory (RAM). "Running" or "runtime" is understood below to mean the period of time during which a respective information technology system is activated between a boot process and a shutdown process or a crash.The period during which the information technology system performs monitoring and saves relevant information in a log file is referred to below as "system monitoring." One or more system monitoring sessions can be performed during a runtime. If the information technology system shuts down or crashes, the log file for that runtime may also be lost. This makes it impossible to reconstruct the system state of the information technology system before the crash.

[0005] Methods and devices for generating log files are well known. For example, WO 2012 / 106969 A1 and CN 112286896 A disclose methods and devices for generating and storing log files in a compressed and converted format. This allows corresponding log files to be stored with a smaller memory requirement and to be written and read more quickly. A log file can also be converted using cryptographic encryption, which prevents unauthorized third parties from processing the log file. However, this does not resolve the problems associated with processing log files mentioned above.

[0006] Furthermore, US 2021 / 226973 A1 discloses a device, a system, and a method for transmitting vehicle logs. A difference log is generated from a vehicle log using a difference generation log. The difference log is then transmitted to a server for analysis.

[0007] Furthermore, US 2021 / 112085 A1 discloses a device and a method for processing information. Coding instructions can be applied to a file to compress its size. The file can be a log, with the log being divided into a header and a payload. The log format used can be written as information in the header. Furthermore, it is provided to generate a small and a large log, with more information being stored in the large log than in the small log. If an error is detected in the small log, the large log is transmitted to another device for analysis.

[0008] Furthermore, US 2020 / 192779 A1 discloses managing event log information of a storage subsystem.

[0009] In addition, when writing data, it is well known that a new file is created as soon as a file to be written reaches a specified maximum file size to avoid excessively large files. In this context, reference is made to: SCHUSTER ET AL: "Introducing the Microsoft Vista event log file format," DIGITAL INVESTIGATION, ELSEVIER, AMSTERDAM, NL, Vol. 4, July 24, 2007 (2007-07-24), pages 65-72, XP022166451, ISSN: 1742-2876, DOI: 10.1016 / J.DIIN.2007.06.015.

[0010] The object of the present invention is to provide an improved method for processing log files with computer-aided data processing systems and a corresponding data processing system with the aid of which the problems described above can be at least mitigated, but preferably eliminated.

[0011] According to the invention, this object is achieved by a method for processing log files with computer-aided data processing systems having the features of claim 1 and a corresponding data processing system having the features of claim 10. Advantageous embodiments and further developments as well as a vehicle with such a data processing system or a component of such a data processing system also emerge from the dependent claims.

[0012] In a method for processing log files with computer-aided data processing systems, wherein a log file is generated, processed and / or at least temporarily stored by a data processing system in an encoded format and wherein the log file comprises a header and a payload, the following method steps are carried out: Reading coding instructions into a coding module; performing system monitoring and generating a stream of diagnostic data by means of a diagnostic module during system monitoring; reading the stream of diagnostic data into a coding module; generating an encoded log file by applying the coding instructions to the stream of diagnostic data by the coding module; reading the encoded log file into a decoding module; reading the coding instructions into the decoding module; and generating a decoded log file by the decoding module taking into account the coding instructions from the encoded log file; wherein the coding module assigns a version identifier of the encoding instructions used to the header of the encoded log file;the coding module appends a log entry as a payload to the encoded log file if the coding module detects a log event in the diagnostic data that affects a specific log parameter for the first time or that describes a change in the respective log parameter; and the coding module causes the encoded log file to be saved in its current state during system monitoring if the coding module has appended a log entry to the encoded log file or if a specified period of time has elapsed without another log entry being appended.

[0013] The method according to the invention makes it possible to record all protocol parameters to be logged during system monitoring in the log file and to make the log file available for later analysis purposes, even in the event of an unexpected shutdown of the computer-based data processing system. By saving the log file in coded form, the individual log entries can be executed in such a way that they can be found particularly quickly and easily in the log file using the decoding module. This reduces the manual effort required to evaluate the log file.

[0014] A data processing system is an information technology system. One or more computer-based data processing systems can be involved in processing the log files. For example, a first data processing system can be executed by an embedded system. The embedded system can be integrated into a vehicle such as a car. For example, it can be a control unit for controlling a head unit. The diagnostic module then monitors this embedded system, recording its status and outputting information processed by the embedded system in the form of diagnostic data. The diagnostic module can be part of the embedded system or integrated into another data processing system.The additional data processing system can also be part of the vehicle or temporarily connected to the vehicle, for example, as a mobile device connected to the vehicle via a CAN bus, a USB interface, or an Ethernet interface, such as a tablet computer or laptop. The coding instructions contain information on how the coding module should process the diagnostic data to generate the log file in the encoded format. Depending on the design of the data processing system to be monitored and the information processed or the protocol parameters to be monitored, the coding instructions can be designed differently. The coding instructions can therefore be described as data processing system-specific.

[0015] The encoded log file is then decoded by the decoding module, taking the same encoding instructions into account. The decoding module can be part of the embedded system or part of another data processing system. For example, the decoding module is provided by or executed on a developer system. The developer system can be, for example, a personal computer. Any modules mentioned above can be implemented in hardware, software, or as a combination of hardware and software.

[0016] The encoded log file generated by the encoding module contains a header and a payload appended to the header. The header includes a version identifier of the encoding instructions used. This allows the decoding module to decide which encoding instructions should be used to decode the encoded log file when decoding the encoded log file. The header of the encoded log file thus represents a component of the log file that is designed uniformly, regardless of system monitoring. New entries are appended to the log file by the encoding module when an analysis of the diagnostic data by the encoding module shows that a new protocol parameter to be logged is returning a value or a changed value for the first time. To do this, the encoding module detects the occurrence of protocol events in the diagnostic data. Such a protocol event can also be referred to as a so-called function call.Examples include starting or stopping the data processing system, loading specific software, changing a sensor value that is recorded and processed by the data processing system, completing a task, or the like. The individual protocol parameters can be any parameter that describes a state of the data processing system or a parameter that describes information processed by the data processing system. Examples of protocol parameters include a system time, a date, a vehicle's ignition status, a received signal strength, a speed value, a mass flow, an IP address, or the like.

[0017] The encoded log file generated by the encoding module can be stored in any data processing system. For example, the encoded log file can be stored on a physical storage medium in the same data processing system that contains the encoding module, or it can be transferred from this data processing system to another data processing system. This transfer can be wireless or wired. Any communication technology, such as Ethernet, a USB connection, Wi-Fi, Bluetooth, NFC, ZigBee, or similar, can be used.

[0018] By appending log entries as payload to the log file only when a log event indicates the first occurrence of a particular log parameter or a change in the corresponding value of that protocol parameter, the file size of the encoded log file can be kept particularly small. This ensures efficient use of computing resources.

[0019] The encoded log file is saved again whenever a new log entry is appended to the encoded log file or after the specified time period has elapsed. The specified time period can be freely selected and can be a fixed time interval, for example, five milliseconds, one second, or five minutes, or an adaptive time period, such as a period that variably executes depending on a log parameter or log event. By frequently saving the encoded log file, i.e. writing the encoded log file to a persistent storage medium, it can be ensured that the encoded log file is available for further analysis purposes even after an unplanned shutdown or crash of the data processing system.

[0020] In particular, the encoded log file is a text file in which the header and payload are stored as an alphanumeric character string, possibly supplemented by special characters and / or mathematical symbols or the like. The decoding module can then read the text file and convert it into another format. For example, the decoding module can generate a table from the encoded log file. If individual protocol parameters appear for the first time or change in the corresponding table, a corresponding cell entry for the protocol parameter can be highlighted, for example, by color-coding. This facilitates manual evaluation of the log file.

[0021] An advantageous further development of the procedure provides that a first encoded log file is at least temporarily cached in the data processing system after its generation during a first system monitoring; and the encoding module concatenates the first encoded log file with the second encoded log file when generating a second encoded log file during a second system monitoring.

[0022] This allows two coded log files to be combined into a single log. In the event of a system crash, this makes it possible to make the information logged during the first system monitoring runtime available for analysis after the data processing system is restarted. The first coded log file is stored in the respective data processing system, in particular at least during a shutdown period of the data processing system or the diagnostic module and coding module. If the data processing system is then restarted and a second system monitoring runtime is executed during a subsequent runtime, the second coded log file is generated, with the first coded log file being concatenated with the second coded log file. Concatenated means, for example, that the first coded log file is placed at the front of the second coded log file or appended to the end of the second coded log file.

[0023] According to a further advantageous embodiment of the method, a plurality of differently versioned coding instructions are stored in a first database. The first database can be part of any data processing system. The first database can therefore also be integrated into the same data processing system as the diagnostic module, the coding module, and / or the decoding module. However, the first database can also be provided on a separate data processing system. Such a separate data processing system can be, for example, a server or server network.

[0024] During the development of hardware and / or software components, new software versions are often created. This can be accommodated by versioning the coding instructions. Depending on the version of the respective hardware or software, the appropriately versioned coding instruction can then be used to encode and decode the log file. For example, the hardware of the data processing system monitored by the diagnostic module may change and / or the corresponding data processing system may process new parameters, such as new sensor data. This requires new protocol parameters to be monitored. This allows a new version of the coding instructions to be generated that also includes these new protocol parameters.

[0025] These differently versioned coding instructions can also be implemented diagnostic module-specifically.

[0026] A further advantageous embodiment of the method according to the invention further provides that a plurality of coding instruction modules are stored in a second database, wherein the data processing system reads and combines at least two coding instruction modules from the second database to generate a coding instruction. This facilitates the process for providing different coding instructions. Depending on the monitored data processing system or the design of the diagnostic module, various differently versioned coding instructions can comprise identical components. These can be stored in the second database, which allows for the particularly quick and easy generation of different coding instructions, in particular differently versioned coding instructions.The second database can also preferably be provided on a data processing system implemented as a server or server network. This server or server network can then be accessed, for example, by programmers who generate the respective coding instructions for the various data processing systems and diagnostic modules.

[0027] According to a further advantageous embodiment When a specific protocol parameter is mentioned for the first time by a protocol event, the coding module appends the value of the protocol parameter described by the diagnostic data to the encoded log file as a log entry; before each further append of a log entry relating to the respective protocol parameter, the coding module checks whether the current value of the protocol parameter or a difference between the current value and the previous value results in a smaller file size of the encoded log file; and the coding module appends the entry to the encoded log file as a log entry which results in the smaller file size.

[0028] This reduces the storage space required to store the encoded log file. An appropriately encoded log file can then be sent more quickly over a wireless or wired network. For example, if the log file is a text file, the encoding module can quickly and easily check whether the value or the difference between the current and previous value results in a smaller file size. This means that only the number of characters in the value or difference needs to be compared. The value or difference that results in the lower number of characters is then selected.

[0029] A further advantageous embodiment of the method according to the invention further provides that the coding module applies a calculation rule to a log entry to be appended to the encoded log file and appends the calculated log entry to the encoded log file. This further reduces the memory required to store the encoded log file. The calculation rule can be any formula. If the protocol parameter is, for example, an IP address, this can be converted into a hex format using the calculation rule. This allows an IP address without periods to be reduced from 15 characters to eight characters. This procedure can be compared to generating hash values ​​using a hash function. In general, any conceivable protocol parameter or log entry can be converted as desired.This also allows the encoded log file to be encrypted, so that an attacker cannot get from the encoded log file to a decoded log file without the appropriate encoding instructions.

[0030] According to a further advantageous embodiment of the method, the coding module checks the log events for the existence of a deletion event and, upon detection of a deletion event for the log parameters mentioned in the deletion event, removes all log entries relating to the respective log parameters from the encoded log file. The deletion event is a specific log event. The coding instructions can specify which log events are to be interpreted as deletion events and, accordingly, which log parameters are to be removed from the encoded log file. For example, a deletion event can be defined as the detection of a specific number of log events, or the detection of a specific log parameter returning a specific value. The expiration of a specified period of time can also represent a deletion event.In particular, a deletion event is specified when it can be ensured that the corresponding protocol parameters to be deleted are no longer required for further analysis purposes. This allows the file size of the encoded log file to be reduced.

[0031] A further advantageous embodiment of the method further provides that a maximum size for the coded log file is specified and, upon reaching the maximum size of the coded log file, the coding module begins generating a second coded log file after the last saving of the coded log file or does not generate another log file, at least for this system monitoring. The maximum size for the coded log file can be defined in various formats, for example as a file size, for example in kilobytes, megabytes, gigabytes, or the like, or as a string length, maximum number of entries, or the like. Once the maximum size for the coded log file is reached, another coded log file can be started from the beginning, or logging can be paused or terminated.

[0032] According to a further advantageous embodiment of the method according to the invention, a maximum number of log entries for a specific log parameter is specified for an encoded log file. Upon reaching the maximum number for this log parameter, the oldest log entry is overwritten by the most recent log entry of the log parameter when a respective log entry is re-appended to the encoded log file. This makes it possible to keep the file size of the encoded log file constant despite ongoing logging, while still capturing current log events. Older, particularly no longer relevant, log events and corresponding log entries are then removed from the encoded log file.

[0033] According to the invention, a data processing system is configured to carry out a method described above. As already mentioned, the data processing system can be any information technology system, such as an embedded system, a mobile device, a personal computer, a server, a quantum computer, or the like. Preferably, multiple data processing systems can be used to implement the method according to the invention. The data processing systems are, in particular, networked with one another. A data processing system can also be simulated or emulated, for example, on a virtual machine.

[0034] According to the invention, a vehicle comprises a data processing system as described above. This makes it possible, with the aid of the method according to the invention, to check the system behavior of computing systems used in vehicles, for example, a control unit of a vehicle subsystem.

[0035] Further advantageous embodiments of the method according to the invention for processing log files and of the data processing system also emerge from the exemplary embodiments which are described in more detail below with reference to the figures.

[0036] Showing: Fig. 1 shows a structure of one or more data processing systems used to carry out a method according to the invention; Fig. 2 shows a schematic representation of the operation of a coding module; and Fig. 3 shows a flowchart of the operation of the coding module.

[0037] A method according to the invention describes the processing of log files 1 by means of one or more data processing systems 2. In the Figure 1In the exemplary embodiment shown, a diagnostic module 8 is integrated into a first data processing system 2, a coding module 6 is integrated into a second data processing system 2, a decoding module 9 is integrated into a third data processing system 2, and a first database 13.1 and a second database 13.2 are integrated into a fourth data processing system 2. In general, it would also be possible for all modules 8, 6, 9 as well as the two databases 13.1 and 13.2 to be integrated into a common data processing system 2. Any conceivable distribution of the modules 8, 6, 9 as well as the databases 13.1 and 13.2 to any number of data processing systems 2 is possible. The diagnostic module 8 and the coding module 6 are particularly preferably integrated into a common data processing system 2 (not shown).

[0038] The diagnostic module 8 performs system monitoring of the data processing system 2 into which it is integrated. The diagnostic module 8 then supplies a stream 7 of diagnostic data for evaluation to the coding module 6. Taking into account coding instructions 5, the coding module 6 then generates an encoded log file 1.KOD from this stream 7 of diagnostic data. This encoded log file 1.KOD is read in by the decoding module 9 and converted into a decoded log file 1.DEK, also taking into account the coding instructions 5.

[0039] The first database 13.1 can contain a multitude of differently versioned coding instructions 5. The second database 13.2 can contain coding instruction modules that can be combined to form new coding instructions 5.

[0040] Figure 2symbolizes the operation of the coding module 6 in a schematic representation. The coding module 6 processes the stream 7 of diagnostic data received from the diagnostic module 8. This includes, for example, status values ​​that describe the system status of the data processing system 2 monitored by the diagnostic module 8 and / or (sensor) data acquired and / or otherwise processed by the corresponding data processing system 2. The stream 7 of diagnostic data can be understood as a sequence of protocol events 11, whereby each protocol event 11 comprises a respective value 14 of the protocol parameter 12 for a respective protocol parameter 12. Depending on the design of the hardware and / or software used, it can be provided that in a respective protocol event 11 in the stream 7 of diagnostic data, all existing protocol parameters 12 are always listed, or only those protocol parameters 12 that indicate a changed value 14.

[0041] Taking into account the coding instructions 5, the coding module 6 converts this information into the encoded log file 1.KOD. The encoded log file 1.KOD comprises a header 3, which describes a version identifier of the coding instructions 5 used, as well as a payload 4, which comprises a sequence of the respectively logged log entries 10. For example, the encoded log file 1.KOD is a text file and thus the header 3 and the payload 4 are a corresponding character string stored in the text file. This character string can include any combination of numbers, letters, special characters, mathematical symbols, and the like. To maintain clarity, only some of the log entries 10 are provided with reference symbols. The respective log entries 10 are listed in the Figure 2In the embodiment shown, they are separated from each other by vertical lines. However, other characters or symbols could generally be used, or a fixed character length could be used.

[0042] Figure 3 shows a flowchart of the operation of the coding module 6.

[0043] In a step 301, the method begins with the coding module 6 analyzing the stream 7 of diagnostic data and detecting the presence of protocol events 11 with respective protocol parameters 12 and respective values ​​14.

[0044] In step 302, the coding module 6 iteratively checks for each protocol event 11 whether a value 14 is present for the first time in the stream 7 of diagnostic data for a respective protocol parameter 12 or whether values ​​14 have already been logged. For this purpose, the coding module 6 can, for example, also read the coded log file 1.KOD.

[0045] If this is the case, step 303 checks whether the value 14 has changed compared to the previous value 14. If the value 14 is constant, no new log entry 10 is generated in step 304. If, however, the value 14 has changed, the difference between the two values ​​14 is calculated in step 305. In step 306, the coding module 6 checks whether the value 14 or the difference between the current and respective previous value 14 results in a smaller size of the encoded log file 1.KOD, for example, due to a smaller number of characters.

[0046] However, if stream 7 of diagnostic data contains a value of 14 for a specific protocol parameter 12 for the first time, or if the value of 14 results in a smaller file size, the value of 14 is appended as an appendix to a character string to be inserted into the encoded log file 1.KOD in step 307. However, if the difference between the current and previous value of 14 is smaller in step 308, i.e., results in a smaller file size, this difference is appended to the character string as an appendix.

[0047] In step 309, the character string, or information to be appended to the character string as log entry 10, is written to the encoded log file 1.KOD. The encoded log file 1.KOD is then saved. Additionally, a timer is running, and in step 310, it is checked whether the timer has expired. If this is the case, step 309 is executed again. Since no new characters were added to the character string of the encoded log file 1.KOD, only the encoded log file 1.KOD is saved again.

[0048] In step 311, the encoded log file 1.KOD can be accessed for analysis. For this purpose, the encoded log file 1.KOD is decoded by the decoding module 9.

Claims

1. Method for processing log files (1) using computer-aided data processing systems (2), a log file (1) being generated, processed and / or at least temporarily stored by a data processing system (2) in a coded format, having the following method steps: - performing system monitoring and generating a stream (7) of diagnostic data by means of a diagnostic module (8) during the system monitoring; and - reading a coded log file (1.KOD) into a decoding module (9); characterized in that the log file (1) comprises a header (3) and a payload (4) and the following method steps are carried out: - reading coding instructions (5) into a coding module (6); - reading the stream (7) of diagnostic data into the coding module (6); - generating the coded log file (1.KOD) by applying the coding instructions (5) to the stream (7) of diagnostic data by the coding module (6); - reading the coding instructions (5) into the decoding module (9); and - generating a decoded log file (1.DEK) taking into account the coding instructions (5) from the coded log file (1.KOD) by the decoding module (9); - the coding module (6) assigning a version identifier of the coding instructions (5) used to the header (3) of the coded log file (1.KOD); - the coding module (6) appending a protocol entry (10) as a payload (4) to the coded log file (1.KOD) if the coding module (6) detects a protocol event (11) in the diagnostic data which affects a specific protocol parameter (12) for the first time or which describes a change in the particular protocol parameter (12); and - the coding module (6) causing the coded log file (1.KOD) to be stored in its current state during the system monitoring if the coding module (6) has appended a protocol entry (10) to the coded log file (1.KOD) or if a specified period of time has elapsed without appending a further protocol entry (10).

2. Method according to claim 1, characterized in that - a first coded log file (1.KOD) is buffered at least temporarily in the data processing system (2) after its generation during a first system monitoring; and - the coding module (6), when a second coded log file (1.KOD) is generated during a second system monitoring, concatenates the first coded log file (1.KOD) with the second coded log file (1.KOD).

3. Method according to claim 1 or 2, characterized in that a plurality of differently versioned coding instructions (5) are kept in a first database (13.1).

4. Method according to claim 3, characterized in that a plurality of coding instruction modules are kept in a second database (13.2), the data processing system (2) reading out and combining at least two coding instruction modules from the second database (13.2) to generate a coding instruction (5).

5. Method according to any of claims 1 to 4, characterized in that - the coding module (6), when a specific protocol parameter (12) is mentioned for the first time by a protocol event (11), appends the value (14) of the protocol parameter (12) described by the diagnostic data to the coded log file (1.KOD) as a protocol entry (10); - the coding module (6) checks, before each further appending of a protocol entry (10) relating to the particular protocol parameter (12), whether the current value (14) of the protocol parameter (12) or a difference between the current value (14) and the previous value results in a smaller file size of the coded log file (1.KOD); and - the coding module (6) appends as a protocol entry (10) the entry to the coded log file (1.KOD) which leads to the smaller file size.

6. Method according to any of claims 1 to 5, characterized in that the coding module (6) applies a calculation rule to a protocol entry (10) to be appended to the coded log file (1.KOD) and appends the calculated protocol entry to the coded log file (1.KOD).

7. Method according to any of claims 1 to 6, characterized in that the coding module (6) checks the protocol events (11) for the existence of a deletion event and, upon detection of a deletion event for the protocol parameters (12) mentioned in the deletion event, removes all protocol entries (10) relating to the respective protocol parameters (12) from the coded log file (1.KOD).

8. Method according to any of claims 1 to 7, characterized in that a maximum size for the coded log file (1.KOD) is specified and when the maximum size of the coded log file (1.KOD) is reached, the coding module (6) starts generating a second coded log file after the last saving of the coded log file (1.KOD) or does not generate any further log file (1) at least for this system monitoring.

9. Method according to any of claims 1 to 8, characterized in that a maximum number of protocol entries (10) for a specific protocol parameter (12) is specified for a coded log file (1.KOD), the oldest protocol entry, when the maximum number for this protocol parameter (12) is reached when a particular protocol entry (10) is appended again to the coded log file (1.KOD), being overwritten by the most recent protocol entry of the protocol parameter (12).

10. Data processing system (2), characterized by a device for carrying out a method according to any of claims 1 to 9.

11. Vehicle, characterized by a data processing system (2) according to claim 10.