Method and device for analyzing and converting CAN bus log data and medium

Abstract

The analysis and conversion method for the CAN bus log data provided by the invention comprises the following steps: acquiring a specified CAN log file, a specified DBC database file and a specified timestamp file; analyzing the DBC database file, generating a tree structure containing the signal definition, displaying the tree structure to a user, and receiving a target signal set selected from the tree structure; reading a CAN log file, and analyzing original CAN frame data in the CAN log file line by line to extract a CAN ID and a data load; decoding the data load into a signal value based on the definition of the CAN ID in the DBC database file; generating an absolute timestamp corresponding to each row of CAN frame data; and combining the decoded signal values with the corresponding absolute timestamps, and generating an output data file only containing the target signal set according to a time sequence. According to the invention, automation of CAN data analysis, high-precision time synchronization and user customized output are realized, the problems of poor integration, inaccurate time processing and complex operation of the existing tool are solved, and the efficiency and accuracy of vehicle-mounted data processing are significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automotive electronics and data communication, in particular to a method for analyzing, converting and processing vehicle controller area network (CAN) bus communication data, and more particularly to a method for automatically converting original CAN log files into structured data with high-precision timestamps and user-customizable signals. BACKGROUND

[0002] CAN bus is the core network for communication between various electronic control units (ECUs) in modern vehicles. During vehicle development, testing and fault diagnosis, engineers usually use tools such as CANoe and PCAN to capture original communication data on the bus and store it as log files (such as.asc,.blf format). These log files are essentially raw binary or hexadecimal message frames recorded in chronological order, lacking direct physical meaning and cannot be directly used for data analysis, visualization or diagnosis.

[0003] In order to understand these data, the CAN database (DBC) file must be used. The DBC file defines the specific location, length, data type, scaling and unit of each signal in the CAN message, and is the "dictionary" for analyzing raw data. Currently, relying on large commercial software such as CANoe for analysis is the mainstream solution, but this solution has some significant defects as follows.

[0004] (1) Poor integration and automation: the software is heavy and difficult to integrate automatically with other data analysis platforms (such as MATLAB, Python data analysis stack or cloud platforms). Its processing process often requires a lot of manual interaction and cannot be embedded in an automated test pipeline.

[0005] (2) Insufficient time synchronization accuracy: the ASC log timestamp generated by CANoe is usually accurate to the second and is a relative time. When performing multi-source data synchronization analysis with high-precision video, sensor data or other system logs, it is difficult to achieve accurate alignment at the millisecond or even microsecond level, which seriously affects the accuracy of the analysis.

[0006] (3) Complex operation and weak customization: the software has a large number of functions and a complex interface, requiring high professional requirements for users. Its data processing and output logic is relatively fixed, and users cannot flexibly select signals, define output formats or processing rules (such as data timeout padding, sampling) according to specific needs.

[0007] (4) Insufficient output flexibility: the output format is limited and often cannot meet the needs of diversified downstream applications.

[0008] Therefore, there is an urgent need in the art for a lightweight, automated, high-precision and highly customizable CAN data parsing and conversion method to overcome the limitations of the above commercial software. SUMMARY

[0009] The present application aims to solve at least one of the technical problems existing in the prior art mentioned above, and proposes a CAN bus log data parsing and conversion method, device and medium to solve the problems of poor integration, inaccurate time synchronization, complex operation and lack of customization ability existing in the prior art.

[0010] In a first aspect, the embodiments of the present application provide a CAN bus log data parsing and conversion method, comprising:

[0011] obtaining a CAN log file, a DBC database file and an optional timestamp file specified by a user through a human-computer interaction interface;

[0012] parsing the DBC database file to generate a tree structure containing all signal definitions and display it to the user, and receiving a target signal set selected by the user from the tree structure;

[0013] reading the CAN log file, parsing the original CAN frame data in it line by line to extract CAN ID and data payload, and decoding the data payload into signal values with physical meaning based on the definition of the CAN ID in the DBC database file;

[0014] generating an absolute timestamp corresponding to each row of the CAN frame data;

[0015] combining the decoded signal values with the corresponding absolute timestamps to generate an output data file containing only the target signal set in time sequence.

[0016] Further, the generation of the absolute timestamp corresponding to each row of the CAN frame data comprises:

[0017] if the timestamp file exists, an initial absolute timestamp is extracted from the timestamp file, and is added to the relative timestamp parsed from the CAN log file to generate the absolute timestamp;

[0018] if the timestamp file does not exist, the timestamp information contained in the CAN log file is directly parsed to generate the absolute timestamp.

[0019] Further, the extraction of the initial absolute timestamp from the timestamp file comprises:

[0020] parsing the Unix timestamp format field in the timestamp file;

[0021] convert the Unix timestamp to a DateTime object of local time as the initial absolute timestamp.

[0022] Further, the method further comprises a skipping process, specifically including:

[0023] skipping lines starting with a predetermined keyword or containing a specific identifier when reading the CAN log file line by line;

[0024] wherein the predetermined keyword includes one or more of "base", "internal", "Begin";

[0025] the specific identifier includes "Start".

[0026] Further, the method further comprises a timeout process, specifically including:

[0027] maintaining a last update time for each signal in the target signal set;

[0028] if the difference between the absolute timestamp of the currently processed CAN frame data and the last update time of a signal exceeds a preset timeout threshold, outputting a preset value for the signal at the current time point.

[0029] Further, the method further comprises a sampling process, specifically including:

[0030] recording the time of the last batch of data writing;

[0031] only when the difference between the absolute timestamp of the currently processed CAN frame data and the time of the last batch of data writing reaches or exceeds a preset sampling interval, writing the current data to the output data file.

[0032] Further, the human-computer interaction interface is a graphical user interface;

[0033] The step of "obtaining the CAN log file, the DBC database file and the optional timestamp file specified by the user through the human-computer interaction interface" is implemented by calling a file selection dialog box;

[0034] The step of "parsing the DBC database file, generating a tree structure containing all signal definitions and displaying it to the user, and receiving the target signal set selected by the user from the tree structure" is implemented by displaying a tree list box control.

[0035] Further, the method is executed through a command line interface, specifically including:

[0036] the paths of the CAN log file, the DBC database file and the timestamp file are obtained by parsing command line parameters;

[0037] The user-defined target signal set is obtained by parsing the command line parameters.

[0038] In a second aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors;

[0039] a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors can implement the steps in the parsing and conversion method of any of the preceding aspects.

[0040] In a third aspect, an embodiment of the present application provides a computer readable medium, and the computer readable medium stores a computer program, and the computer program can implement the steps in the parsing and conversion method of any of the preceding aspects when executed by a processor.

[0041] The present application provides a CAN bus log data parsing and conversion method, which integrates the originally scattered, manual and error-prone CAN data parsing process into a highly automated, accurate and user-customizable complete solution. First, the present application connects "file acquisition", "signal selection", "parsing and decoding", "timestamp processing" and "data generation" into a complete process, which ensures end-to-end automation from raw data to available results, greatly improves processing efficiency and reliability, and avoids errors and inconsistencies caused by manual intervention. Secondly, the present application innovatively takes "user signal selection" as an independent and key step, and takes the result (target signal set) as the direct basis for the final output. This means that the final generated file only contains effective data that users are interested in, rather than all signals. This "on-demand output" mechanism fundamentally solves the problem of redundant and massive output data in the prior art, greatly reducing unnecessary data storage and transmission overhead, and improving the efficiency of subsequent data analysis. The present application not only protects an automated tool and method, but also protects a "user demand-driven" simplified data processing idea, making it have strong practicality and excellent economy in industrial applications. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A core process block diagram of a CAN bus log data parsing and conversion method provided by an embodiment of the present application;

[0043] Figure 2 A schematic diagram of the content of the ASC log file generated by the traditional CANoe in the prior art;

[0044] Figure 3This is a schematic diagram of a system graphical user interface (GUI) according to an embodiment of the present application, showing a file upload area, a signal tree list, and operation buttons;

[0045] Figure 4 This is a detailed schematic diagram of a signal selection tree list generated by parsing a DBC file in one embodiment of this application;

[0046] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0047] To enable those skilled in the art to better understand the technical solutions of this application, exemplary embodiments of this application are described below with reference to the accompanying drawings, including various details of the embodiments of this application to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. Unless otherwise specified, the various embodiments of this application and the features within those embodiments can be combined with each other.

[0048] As used herein, the term “and / or” includes any and all combinations of one or more of the associated enumerated entries. The terminology used herein is for describing particular embodiments only and is not intended to limit the application. As used herein, the singular forms “a” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated features, integrals, steps, operations, elements, and / or components is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0049] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It should also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0050] CANoe, short for CAN open environment, is a bus development environment developed by the German company Vector for automotive bus development. Compared to CanLog (an in-vehicle CAN bus data acquisition and analysis tool, mainly used for data recording and diagnosis of automotive electronic systems, supporting offline storage, real-time analysis, and other functions), CANoe has the following technical limitations.

[0051] (1) Limited integration capabilities.

[0052] Compared to the modular design of the CanLog tool, CANoe may be less flexible in integrating with other development tools and data processing platforms. The log files generated by CANoe (such as .asc files) are typically raw hexadecimal data, lacking crucial information such as timestamps. This complicates time alignment and synchronization when integrating with data visualization platforms (such as LightningChart), increasing the difficulty of manual processing.

[0053] (2) The user interface is not user-friendly.

[0054] While CANoe offers powerful features, its graphical user interface (GUI) and command-line interface (CLI) may not be intuitive enough for non-professional users. Compared to the simplified operation of CanLog tools, using CANoe requires users to have a certain level of professional knowledge and experience, increasing the learning curve and difficulty of use.

[0055] (3) Limited data export formats.

[0056] CANoe supports a relatively limited number of export formats, especially when processing and transforming data, which may not meet the needs of all users. Compared to the rich export formats offered by CanLog, CANoe's data export functionality is less flexible, limiting the convenience of subsequent data visualization and analysis.

[0057] (4) Insufficient customization capabilities.

[0058] CANoe's parsing and processing logic is relatively fixed, making it difficult to customize to meet different user needs. In contrast, CanLog tools offer greater flexibility and scalability, adapting to various application scenarios and data processing requirements, while CANoe may fall short in this regard.

[0059] In automotive networks, the CAN bus is a widely used communication protocol for transmitting data within the vehicle. CAN data is typically acquired by tools such as CANoe, generating log files (e.g., in .asc format). However, the data in these log files is raw frame data recorded in hexadecimal format, which cannot be directly understood or used. Therefore, a series of parsing and processing steps are required to transform this raw data into meaningful information that can be used for graphical representation.

[0060] The original ASCII log file is as follows Figure 2 As shown, to parse raw CAN frame data into understandable and meaningful information, a DBC (Database CAN) file is required. A DBC file is a database file format used to describe signals and messages in a CAN network. It is typically generated by CANoe or other tools of Vector and contains detailed definitions of all messages and signals in the CAN network. Through the DBC file, the data structure of each CAN frame can be defined, including information such as signal bit offsets, bit lengths, and data types.

[0061] This application designs and implements a professional solution for CanLog conversion tools, used for parsing and converting CAN bus log data. This tool aims to automatically convert CAN bus log files (such as .asc format) into signal value data that can be directly used for analysis and display, while retaining the timestamp information in the original data, ensuring that the parsed signal values ​​completely correspond to the time series.

[0062] refer to Figure 3 In one embodiment of this application, the main UI of the window primarily utilizes Windows Forms controls and DevExpress controls, providing users with richer interactive and data display functions, enhancing the interface's aesthetics and ease of operation. The main interactive controls in the interface can be divided into two categories: an upload button next to the file path of the user-selected file, and a start parsing button to initiate the parsing process. These two types of controls (buttons) work together to ensure that users can complete operations such as file upload, signal selection, and data parsing. The implementation principles of these two types of controls will be explained separately below. For specific implementation methods of constructing the system's functional modules, please refer to... Figure 1 .

[0063] 1. File path upload control (corresponding upload button)

[0064] The file path upload control is mainly used to upload CAN data files (CAN log files) that need to be parsed, upload XML files that record timestamp information, upload DBC files (DBC database files) needed for parsing CAN data, and export the parsed CSV files. Its main implementation principles include:

[0065] (1) Create a file dialog box object.

[0066] First, an OpenFileDialog object is created, which allows the user to browse and select files using the provided standard file selection dialog box.

[0067] Setting the Multiselect property to false means that the user cannot select multiple files at the same time, but can only select one file.

[0068] (2) Set the title of the file dialog box and the file filter.

[0069] Set the title of the file dialog box so that users know which type of file to select.

[0070] Configure a file filter to restrict users to selecting only files of specific formats. For example, CAN data includes files in .csv, .asc, and .blf formats.

[0071] By using file filters, users can only see and select specific file types, preventing the selection of incompatible file types.

[0072] (3) Display the file dialog box and process the user's selection.

[0073] Calling the dialog.ShowDialog() method displays a file selection dialog box and suspends the current thread to wait for user interaction.

[0074] If the user selects a file and clicks the "OK" button, ShowDialog() will return DialogResult.OK. The full path of the selected file (dialog.FileName) will be assigned to the text box on the screen. This way, the user can see the path of the selected file on the screen.

[0075] (4) Preview part of the file content.

[0076] After the system's ShowDialog() method returns DialogResult.OK, it calls the GetFilePreview method. In GetFilePreview, the system uses File.ReadLines(filePath) to read the file content line by line. This method is lazily executed, meaning it only reads file lines when needed, avoiding loading the entire file into memory at once, thus improving performance and resource utilization.

[0077] By using the Take(5) method in LINQ, the system reads only the first 5 lines of the file. Take(5) extracts the first 5 lines from the line enumeration of the file, forming a subset.

[0078] The system calls the ToArray() method to convert the subset into a string array, ensuring that the contents of these 5 lines can be accessed and manipulated independently. The first 5 lines of the file are read from the specified path, converted into an array, and then concatenated into a string for user preview.

[0079] (5) CAN signal navigation tree generation. The DBC database file is parsed to generate a tree structure containing all signal definitions and displayed to the user. The user selects the target signal set from the tree structure.

[0080] The information selected by the user from the DBC file needs to be displayed in a tree control. This allows users to easily view and select the CAN signals they need through the tree control. Its main implementation principles include:

[0081] Open the DBC file and obtain signal information.

[0082] The `DbcMethod.DbcFileOpen` method is called to open the selected DBC file and returns the file index and signal length via reference parameters. The `res` variable may be used to check whether the file opening was successful.

[0083] The DbcMethod.GetRealTime method retrieves all signal information from the current DBC file and stores it in an array.

[0084] Call the DbcMethod.GetDevTreeNode method, pass in the signal information array, and generate a tree-structured list of nodes, treeNode.

[0085] Create a tree node named "All", set its ID to -1 and ParentId to -2 (representing the root node), and then add it to the treeNode list.

[0086] Bind the data source and update the UI.

[0087] Bind the generated list of tree nodes to tree_dbclist (TreeList control).

[0088] Expand all tree nodes so that users can see all signals at a glance.

[0089] Set all nodes to the selected state so that users can quickly select all signals.

[0090] Update the number of selected signals.

[0091] A detailed diagram of the signal selection tree list generated from the DBC file parsing can be found here. Figure 4 .

[0092] 2. Analyze the CAN signal control (corresponding to the Start Analysis button).

[0093] Once the file is received, parsing can begin. The CAN log file is read, and the raw CAN frame data is parsed line by line to extract the CAN ID and data payload. Based on the CAN ID definition in the DBC database file, the data payload is decoded into physically meaningful signal values. An absolute timestamp corresponding to each line of CAN frame data is generated. The decoded signal values ​​are combined with the corresponding absolute timestamps to generate an output data file containing only the target signal set, arranged in a time sequence. Its main implementation principles include:

[0094] (1) Obtain the paths to the CAN log file, output file (the DBC file needed to upload and parse CAN data), and XML file (used to upload and record timestamp information) from the text box control on the human-computer interaction interface. Use the FileWriteAllText method to create a file in the path specified by outPath and write an empty string to the file.

[0095] The human-machine interface is a graphical user interface. The steps of obtaining the user-specified CAN log file, DBC database file, and optional timestamp file through the human-machine interface are implemented by calling a file selection dialog box. The steps of parsing the DBC database file, generating a tree structure containing all signal definitions and displaying it to the user, and receiving the target signal set selected by the user from the tree structure are implemented by displaying a tree list box control.

[0096] In some embodiments, when a user clicks the "Upload DBC" button, the system displays an OpenFileDialog dialog box with the filter set to *.dbc. After the user selects a file, its path is displayed in a text box, the file is parsed, and a signal tree is displayed in a TreeList control for the user to select. This provides an intuitive and user-friendly operation method, greatly reducing the user's learning curve and operating costs.

[0097] (2) Open the uploaded file through FileStream for reading and writing operations, and create a StreamWriter object to write text data into the file. At the same time, use the using statement to ensure automatic resource recycling and error management.

[0098] (3) Process CAN data timestamps. If a timestamp file exists, an initial absolute timestamp is extracted from the timestamp file and added to the relative timestamp parsed from the CAN log file to generate an absolute timestamp; if a timestamp file does not exist, the timestamp information in the CAN log file is directly parsed to generate an absolute timestamp.

[0099] The initial timestamp format in the CAN data .asc file is MMM d hh:mm:ss tt yyyy, which needs to be converted to yyyy / MM / dd HH:mm:ss:fff format. First, use Split to split the time string into a string array by spaces. Next, use the LINQ Where method to filter out empty strings or strings containing only whitespace characters. Then, use Skip(2) to skip the first two elements and keep only the rest. Next, convert the final result into an array by calling ToArray(). Finally, use the Join method to join the array elements into a new string by spaces.

[0100] The timestamps in the .asc file are only accurate to the second. To ensure accurate data alignment during playback, if the uploaded folder contains XML files, the Unix timestamp after `start_posix_host` in the XML needs to be extracted as the actual initial timestamp. An object representing the Unix start time (1970-01-01 00:00:00 UTC) is created using the `DateTimeOffset` constructor. The extracted Unix timestamp (in milliseconds) is added to the start time using the `AddMilliseconds` method to obtain the specific `DateTimeOffset` object. The `LocalDateTime` property converts the `DateTimeOffset` object to a local `DateTime` object.

[0101] In some embodiments, the `start_posix_host` field (a millisecond value representing Unix time) is parsed from an XML-formatted timestamp file and converted to local time as the initial time T0. The relative time (e.g., 0.123456) at the beginning of each line in the `.asc` file is read and added to T0 to obtain the absolute timestamp of that line (e.g., 2025 / 01 / 01 08:00:00:123). This provides a high-precision and flexible timestamp processing mechanism, enabling absolute time synchronization using external high-precision time sources while also being compatible with files without external time sources, thus enhancing the applicability and accuracy of the method.

[0102] Extract an initial absolute timestamp from the timestamp file, including: parsing the Unix timestamp format field in the timestamp file; converting the Unix timestamp into a DateTime object of local time as the initial absolute timestamp.

[0103] In some embodiments, reading from an XML file<start_posix_host> 1640995200000< / start_posix_host> Substituting this value (1640995200000 milliseconds) into DateTimeOffset.FromUnixTimeMilliseconds().LocalDateTime, we obtain Beijing time 2022 / 1 / 1 8:00:00. This setting enables accurate conversion between different time formats (Unix timestamps and local time), providing a reliable foundation for time synchronization.

[0104] (4) After determining the initial time, begin formally parsing the CAN data .asc file, reading the file content and processing each line using a file stream (fs) and a reader (sr) to read the log file content line by line. If the current line starts with "base", "internal", or "Begin", or contains "Start", skip the line and do not perform subsequent processing. If the current line starts with " / / ", it indicates a comment line. Extract the version information and update canid_index, dlc_index, and data_index according to the version number. Among them, canid_index represents the index of the identifier (CAN ID) in the CAN information; dlc_index represents the index of the data length code (DLC); and data_index represents the index of the data part. Define a long variable beforeWriteTime and initialize it to 0.

[0105] In some embodiments, when reading an .asc file, internal data lines starting with "base" are skipped without further frame parsing and decoding. This setting improves the efficiency and accuracy of data processing and avoids parsing errors caused by non-data lines.

[0106] (5) After reaching the formal CAN data frame record line, split the content of the current line by spaces, remove empty elements, and obtain the cols list. If the length of cols is less than or equal to 9, skip the line. Convert cols[0] to a timestamp timeSpan. If the beforeWriteTime value is 0, assign the value of timeSpan to it. Create a CanMsg_T structure and fill in its fields: canID, dlc, data, and timestamp. Among them, canID is converted from cols[canid_index] to a hexadecimal unsigned integer; dlc is converted from cols[dlc_index] to a byte integer; data is extracted from cols[data_index], the length of canMsg_T.dlc, and converted to a hexadecimal byte array. Call the DbcMethod.DbcCanDecode method to decode the CAN message.

[0107] (6) Sampling Processing. Record the time when the previous batch of data was written; only when the difference between the absolute timestamp of the currently processed CAN frame data and the time when the previous batch of data was written reaches or exceeds a preset sampling interval will the current data be written to the output data file. Maintain a last update time for each signal in the target signal set; if the difference between the absolute timestamp of the currently processed CAN frame data and the last update time of a certain signal exceeds a preset timeout threshold, then the output data at the current time point will output a preset value for that signal.

[0108] Specifically, if the difference between the current timestamp and `beforeWriteTime` (the time the previous batch of data was written) is greater than or equal to 50 milliseconds, sampling is performed. `DbcMethod.GetRealTime` is called to retrieve real-time signal information. A single output line, `one_line`, is constructed, with the timestamp as the first column. `signal_TagList` is iterated; if the signal name is in `selectSignalNameList`, a timeout is determined based on the timestamp difference (e.g., exceeding 200 ms). If a timeout occurs, 0 is output; otherwise, the signal value is output. The constructed line is written to the output stream `sw`. `beforeWriteTime` is updated to the current timestamp.

[0109] In some embodiments, the preset sampling interval is 50 ms. After the system processes a frame of data with a time of 100.050 s, the next frame of data arrives at 100.090 s. Since the time difference (40 ms) is less than 50 ms, this frame of data is not output. Only when the data at 100.110 s arrives, and the time difference (60 ms) exceeds 50 ms, is the data from 100.110 s (or the previously accumulated unoutputted data) written to the file. This setting effectively reduces the total amount of output data while maintaining the accuracy of the time series, thus reducing storage pressure and the computational overhead of subsequent processing.

[0110] In some embodiments, the preset timeout threshold is 200 ms. If the "vehicle speed" signal was last updated at 1000 ms, and the current processing reaches a CAN frame at 1205 ms, and this frame does not contain the "vehicle speed" signal, then the "vehicle speed" column in the output data of the 1205 ms row is filled with 0. This setting ensures the continuity of the output data stream, prevents curve interruptions during data visualization due to temporary signal delays, and improves the quality of data analysis.

[0111] Preferably, the CAN bus log data parsing and conversion method provided in this application is executed through a command-line interface (CLI), specifically including:

[0112] The paths to the CAN log file, DBC database file, and timestamp file are obtained by parsing command-line arguments.

[0113] The user-defined set of target signals is obtained by parsing command-line arguments.

[0114] In some embodiments, the user uses the command: `CanLogParser -asc can_data.asc -dbcmodel.dbc -xml timestamp.xml -signals VehicleSpeed ​​EngineRpm -outputresult.csv`. After receiving the parameters, the tool automatically selects the specified signals (VehicleSpeed, EngineRpm) for parsing, without needing to launch a graphical interface. It supports batch and automated script processing, meeting the needs of advanced application scenarios such as continuous integration (CI) / automated testing, and improving the tool's efficiency and application scope.

[0115] One embodiment of this application uses a desktop application developed on a Windows system as an example, but its core methods are equally applicable to other platforms or embedded implementations.

[0116] Implementation Environment and Tools: This embodiment uses the C# programming language, is developed based on the .NET Framework and Windows Forms framework, and uses the DevExpress control library to enhance the aesthetics and functionality of the user interface.

[0117] The implementation process is as follows:

[0118] (1) File Acquisition: After the user runs the program, such as Figure 3 The main interface is shown. Users can select the three required files by clicking the "Browse" button: logdata.asc (CAN log), model.dbc (DBC database), and timestamp.xml (timestamp). Clicking the button triggers the OpenFileDialog control, whose Filter property is set to the corresponding file format.

[0119] (2) Signal Selection: After the DBC file is loaded, the system calls methods such as DbcMethod.DbcFileOpen and DbcMethod.GetRealTime to parse the file and obtain all signal information. Then, DbcMethod.GetDevTreeNode is called to generate tree nodes and bind them to the TreeList control on the interface, forming a structure like... Figure 4 The signal tree shown. Users can select the desired signals, such as "vehicle speed" and "engine speed," by checking the checkboxes in front of the tree nodes.

[0120] (3) Data parsing and transformation (core processing flow): After the user clicks the "Start parsing" button, the program starts the core processing thread.

[0121] Initialization: Obtain all file paths from the text boxes in the interface and create an output file stream.

[0122] Processing time references: The program parses the timestamp.xml file and finds...<start_posix_host> 1640995200000< / start_posix_host> The tag reads the Unix timestamp (1640995200000 milliseconds). It is then converted to local time 2022 / 1 / 1 8:00:00 using DateTimeOffset.FromUnixTimeMilliseconds(1640995200000).LocalDateTime, which is used as the absolute time base T0.

[0123] Processing .asc files line by line: Use StreamReader to read logdata.asc line by line.

[0124] Skip non-data rows: Check the string at the beginning of the row; if it starts with "base", "internal", or "Begin", skip the row.

[0125] Parse data rows: For valid data rows, use the string.Split method to split by spaces, filter out null values, and obtain the field array cols. Extract the relative time string (e.g., 0.001234), CANID (hexadecimal string), DLC, and data segment (hexadecimal string) from the array according to the predefined index positions.

[0126] Decoding the signal: Convert the CAN ID and data segment into a byte array, call the DbcMethod.DbcCanDecode method, pass in the DBC file index and CAN message structure, and decode all physical signal values ​​contained in the frame data.

[0127] Generate absolute timestamp: Convert the extracted relative time (0.001234 seconds) into a TimeSpan object, and then add it to the absolute time base T0 to obtain the absolute timestamp of this frame data 2022 / 1 / 1 8:00:00.001234.

[0128] Data generation and output:

[0129] Timeout Handling: The system maintains a last update time for each selected signal. It checks the difference between the current timestamp and the last update time for each signal. If the difference exceeds 200 ms (e.g., the "vehicle speed" signal has not appeared for 200 ms), it fills the current row of output data with 0 for that signal.

[0130] Sampling Processing: The system records the timestamp of the last data write, lastWriteTime. The difference between the current frame timestamp and lastWriteTime is calculated. If the difference is less than 50 ms, no output is performed; if it is greater than or equal to 50 ms, output is executed.

[0131] Write to file: Construct a single CSV data row, with the first column being the absolute timestamp, and subsequent columns outputting the current value (or timeout value) of all user-selected signals in sequence. Write this row of data to the output file stream.

[0132] (4) Completion: After processing all rows, close the file stream and prompt the user that parsing is complete. The final generated CSV file can be directly used in various analysis tools.

[0133] Overall, the advantages of this application compared to the prior art include:

[0134] (1) High degree of automation and integration: This application realizes full-process automation from file input to result output, and can be seamlessly integrated into automated testing and data analysis pipelines through CLI, greatly reducing manual intervention.

[0135] (2) High-precision time synchronization: By introducing an external high-precision time source (such as Unix timestamps in XML), millisecond-level absolute time synchronization is achieved, providing a solid foundation for multi-system data fusion analysis.

[0136] (3) User-friendly and customizable: The graphical signal tree and file dialog box reduce the difficulty of operation. Users can freely select the signals to be output, meeting the customization needs of different scenarios.

[0137] (4) Flexible output and optimized data: Supports multiple common output formats to adapt to different analysis tools. The built-in timeout and sampling processing mechanisms optimize data quality and volume while ensuring data validity.

[0138] (5) Lightweight and efficient: Compared with large commercial software, this application focuses on the core parsing and conversion functions, with a more efficient processing flow and less resource consumption.

[0139] Based on the same inventive concept, embodiments of this application also provide an electronic device. Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application. Figure 5 As shown in the embodiments of this application, an electronic device includes: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement any of the CAN bus log data parsing and conversion methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0140] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0141] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0142] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0143] This application also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps in any of the CAN bus log data parsing and conversion methods described in the above embodiments. The computer-readable storage medium can be volatile or non-volatile.

[0144] This application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described method for parsing and converting CAN bus log data.

[0145] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0146] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0147] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0148] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing the status information of the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0149] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0150] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0151] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0152] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0154] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for general illustrative purposes only and should not be construed as limiting. In some embodiments, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this application as set forth by the appended claims.

Claims

1. A method for parsing and converting CAN bus log data, characterized in that, include: The system obtains the user-specified CAN log file, DBC database file, and optional timestamp file through the human-computer interaction interface. The DBC database file is parsed to generate a tree structure containing all signal definitions and displayed to the user. The user then selects a set of target signals from the tree structure. The CAN log file is read, and the raw CAN frame data is parsed line by line to extract the CAN ID and data payload; based on the definition of the CAN ID in the DBC database file, the data payload is decoded into a signal value with physical meaning. Generate an absolute timestamp corresponding to each line of CAN frame data; The decoded signal values ​​are combined with the corresponding absolute timestamps to generate an output data file containing only the target signal set in a time series.

2. The parsing and conversion method according to claim 1, characterized in that, The generation of the absolute timestamp corresponding to each line of the CAN frame data includes: If the timestamp file exists, an initial absolute timestamp is extracted from the timestamp file and added to the relative timestamp parsed from the CAN log file to generate the absolute timestamp; If the timestamp file does not exist, the timestamp information contained in the CAN log file is directly parsed to generate the absolute timestamp.

3. The parsing and conversion method according to claim 2, characterized in that, Extracting an initial absolute timestamp from the timestamp file includes: Parse the Unix timestamp format fields in the timestamp file; The Unix timestamp is converted into a DateTime object with local time and used as the initial absolute timestamp.

4. The parsing and conversion method according to claim 1, characterized in that, This also includes skipping processing steps, specifically: When reading the CAN log file line by line, skip lines that begin with a predetermined keyword or contain a specific identifier; The predetermined keywords include one or more of "base", "internal", and "Begin"; The specific identifier includes "Start".

5. The parsing and conversion method according to claim 1, characterized in that, It also includes a timeout handling process, specifically including: Maintain a last update time for each signal in the target signal set; If the difference between the absolute timestamp of the currently processed CAN frame data and the last update time of a certain signal exceeds a preset timeout threshold, then the output data behavior at the current time point will output a preset value for that signal.

6. The parsing and conversion method according to claim 1, characterized in that, It also includes the sampling process, specifically including: Record the time when the previous batch of data was written; The current data is written to the output data file only when the difference between the absolute timestamp of the currently processed CAN frame data and the time when the previous batch of data was written reaches or exceeds a preset sampling interval.

7. The parsing and conversion method according to claim 1, characterized in that, The human-computer interaction interface is a graphical user interface; The step "obtain the user-specified CAN log file, DBC database file, and optional timestamp file through the human-computer interaction interface" is achieved by calling the file selection dialog box; The step "parse the DBC database file, generate a tree structure containing all signal definitions and display it to the user, and receive the target signal set selected by the user from the tree structure" is implemented by displaying a tree list box control.

8. The parsing and conversion method according to claim 1, characterized in that, The method is executed via a command-line interface and specifically includes: The paths to the CAN log file, DBC database file, and timestamp file are obtained by parsing command-line parameters; The user-defined set of target signals is obtained by parsing command-line arguments.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the steps of the parsing and conversion method as described in any one of claims 1 to 8.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it can perform the steps of the parsing and conversion method as described in any one of claims 1 to 8.

Images