CAN signal data dynamic backtracking and hierarchical drawing processing system and method based on LOD and LTTB

Abstract

The invention discloses a CAN signal data dynamic backtracking and hierarchical drawing processing system based on LOD and LTTB, which comprises a message acquisition unit, a message analysis unit, a message storage unit, a signal processing unit and a display rendering unit, and also discloses a processing method of the CAN signal data dynamic backtracking and hierarchical drawing processing system. Comprising the following steps: starting a system and starting to receive message data, constructing 0-level data, calculating the level of LOD, and drawing an image. Full data life cycle traceability can be realized, and a complete data basis is provided for deep fault diagnosis; repeated calculation delay is eliminated; maintaining visual features of the signals; full-cycle data backtracking is achieved, limitation of a traditional tool is broken through, dynamic signal adding and complete historical data loading in the analysis stage are supported, fault diagnosis flexibility and analysis efficiency are improved, and test repetition caused by signal missing is avoided; and super-large-scale data processing is supported, memory-level access performance is realized with hard disk-level storage cost, and a common computer is supported to process 10 billion-level data points.

Description

Technical Field

[0001] This invention belongs to the field of application publishing technology, specifically relating to a processing system and method for dynamic backtracking and hierarchical plotting of CAN signal data based on LOD and LTTB. Background Technology

[0002] In automotive electronics development and testing, the CAN (Controller Area Network) bus is the primary in-vehicle network protocol. The DBC (Database CAN) file is a standard file format used to describe messages, signals, and their physical values ​​on the CAN bus. Through DBC files, raw, hexadecimal CAN messages can be parsed into engineering-meaning physical signals (such as vehicle speed, engine speed, battery voltage, etc.). Visualizing and analyzing these signals is a crucial tool for engineers in fault diagnosis, performance evaluation, and system calibration. In scenarios such as vehicle road tests and durability tests, it is necessary to continuously record CAN signals for hours or even days, generating data volumes reaching hundreds of millions or even billions of points. Currently, mainstream CAN analysis tools (such as Vector's CANoe and Tongxing's TSMaster) generally employ memory-based layered construction and downsampling techniques to handle the visualization needs of massive CAN signal data.

[0003] These existing technologies share a common feature: their memory-resident architecture. Both raw data and display data, after hierarchical downsampling, are stored in a limited memory buffer. CANoe allows users to configure the maximum number of data points in a single signal window, while TSMaster manages real-time data by maintaining a fixed number of points. During graphics rendering, these tools dynamically perform downsampling calculations from the raw data in memory based on the current display time range, generating a data point set suitable for the current view to achieve smooth display at different scaling levels.

[0004] 1. Low data processing efficiency and interaction delay Existing technologies employ a memory-based temporary caching architecture, requiring re-execution of downsampling calculations every time the view changes (zoom / pan). This "one interaction, one calculation" model leads to unavoidable redundant computation overhead. For example, CANoe exhibits significant interaction latency every time the time range is adjusted, and even repeated operations within the same time range produce the same latency, indicating that the system lacks an effective caching mechanism and cannot avoid unnecessary redundant computations.

[0005] 2. Lack of data persistence capabilities Because it relies entirely on in-memory storage, existing solutions have inherent data capacity limitations. When the amount of signal data being analyzed exceeds the physical memory capacity, the system cannot process it effectively. More importantly, this architecture cannot support dynamic backtracking analysis—newly added signals during the analysis process cannot access historical data prior to their addition time, severely limiting the flexibility and completeness of the analysis.

[0006] 3. Visual distortion problem of downsampling algorithm Existing tools employ downsampling algorithms that have significant shortcomings in preserving the visual characteristics of signals. For example, TSMaster exhibits visual feature shifts at different scaling levels, and key signal features (such as peaks and transients) are easily lost or distorted during downsampling, affecting the accuracy and reliability of the analysis.

[0007] With the rapid development of automotive electronics, industrial control, and other fields, the CAN (Controller Area Network) bus has become the core communication bus for connecting various sensors, actuators, and control units due to its advantages such as high reliability, real-time performance, and low cost. In practical applications, the CAN bus generates massive amounts of signal data, including key information such as vehicle operating status, equipment conditions, and fault information. This data not only needs to be transmitted and stored in real time but also needs to support subsequent data analysis, fault diagnosis, and trend prediction.

[0008] Currently, CAN signal data processing technologies mainly revolve around data storage, data backtracking, and data visualization. Figure 3 The core components are as follows: ① Data storage technology: Mainstream solutions use local databases (such as SQLite, MySQL) or distributed storage systems (such as Hadoop HDFS) to store CAN signal data. Data is usually partitioned and managed according to timestamps to improve the efficiency of data writing and basic querying; ② Data backtracking technology: Existing backtracking methods are mostly "full loading + fixed precision retrieval", that is, after the user specifies a time range, the system loads all the original CAN signal data within that range, or extracts data for backtracking display according to a preset fixed sampling interval (such as 100ms / time); ③ Data visualization and plotting technology: Commonly used plotting tools (such as MATLAB, Python Matplotlib, industrial configuration software) adopt a "single resolution plotting" mode, either directly plotting the full amount of original data (suitable for small data volume scenarios), or reducing the data volume through simple uniform sampling (such as taking one sample every N data points) before plotting to avoid screen lag.

[0009] While existing CAN signal data processing technologies can meet basic application needs, they have significant shortcomings when dealing with massive amounts of data (such as GB-level CAN data generated by a car driving continuously for 24 hours), high real-time backtracking requirements (such as retrieving historical data within seconds after a fault occurs), and detailed plotting requirements (such as viewing global trends and local details simultaneously). Specifically, these shortcomings include: ① Low data backtracking efficiency and excessive resource consumption: Existing backtracking technologies use a "full loading" mode. When a user needs to backtrack CAN data over a certain time range (such as 1 hour), the system needs to load all the original data within that time period, resulting in long data transmission times (e.g., loading GB-level data takes minutes) and high memory consumption (easily causing system lag or crashes); ② Fixed-precision retrieval cannot adapt to the user's dynamic needs: If a user first views the global trend (requiring low-precision data) and then focuses on the details of a specific fault period (requiring high-precision data), existing technologies need to reload data of the corresponding precision, failing to achieve "one-time backtracking, dynamic precision adjustment," making the operation cumbersome and time-consuming. ③ Difficulty in balancing plotting quality and efficiency: Single-resolution plotting presents a contradiction: When plotting full raw data, if the data volume is too large (e.g., 1000 CAN signals per second), it will lead to slow plotting rendering, screen stuttering, or even failure to display normally; when using simple uniform sampling plotting, it is easy to lose key data features (e.g., peaks, valleys, abrupt changes), resulting in distorted plotting results and affecting the accuracy of scenarios such as fault diagnosis (e.g., missing key data points of sudden changes in engine speed); ④ Lack of a layered plotting mechanism: Existing plotting technology cannot automatically adjust data precision according to the user's view zooming needs. For example, when the user zooms in on the plotting area to view local details, the system cannot automatically load the high-precision raw data of that area; when zooming out to view the global trend, it cannot automatically switch to low-precision downsampled data, resulting in a poor user experience; ⑤ Lack of a dynamic balance mechanism between data precision and processing efficiency: Existing technology does not incorporate the concept of LOD (Level of Detail) and cannot dynamically adjust data precision according to application scenarios (e.g., backtracking range, plotting zoom ratio); Based on the shortcomings of the aforementioned background technologies, the following core technical issues urgently need to be addressed in the current CAN signal data processing field: ① How to achieve dynamic precision backtracking of CAN signal data: Avoid loading the full amount of data and achieve "automatic matching of data precision according to the backtracking range". For example, when backtracking 1 day of data, use low-precision downsampled data, and when backtracking 10 minutes of data, automatically switch to high-precision raw data to improve backtracking efficiency and reduce resource consumption; ② How to achieve layered plotting of CAN signal data: Construct a "view zooming - data precision" linkage mechanism. When the user zooms the plotting interface, the system automatically loads data of the corresponding precision for plotting, ensuring the smoothness of the global trend display without losing key features of local details. ③ How to retain key features of CAN signal data during downsampling: It is necessary to introduce efficient downsampling algorithms (such as LTTB) to solve the feature loss problem caused by simple uniform sampling, and ensure that the downsampled data can still accurately reflect the trend, peak, and abrupt change points of the original data, so as to meet the accuracy requirements of scenarios such as fault diagnosis and working condition analysis. ④ How to integrate dynamic backtracking and hierarchical plotting functions to achieve efficient end-to-end processing: It is necessary to build a unified processing framework, integrate the LOD concept into the entire process of data storage, backtracking and plotting, and realize the collaborative work of "data storage hierarchically according to precision, backtracking dynamically adjusts precision according to needs, and plotting automatically hierarchically according to view", so as to improve the overall processing efficiency and user experience. Summary of the Invention

[0010] To address the technical problems existing in the prior art, this invention provides a real-time visualization and full-cycle analysis solution for massive signal data from vehicle CAN bus. By introducing a level-of-detail model combined with the LTTB downsampling algorithm, a persistent multi-resolution signal data pyramid is constructed. This enables dynamic adaptive loading of different levels of sampled data based on the plotting time range, fundamentally solving the plotting lag problem during long-term browsing. Furthermore, through a full data persistence architecture, it supports the immediate and complete loading of all historical data from the recording point to the current moment when any signal curve is dynamically added during the analysis process, breaking the limitation of traditional tools that can only display newly added data. This solution supports second-level scaling, smooth plotting, and full-cycle traceable in-depth analysis of hundreds of millions of data points.

[0011] The technical solution adopted in this invention is as follows: A processing system for dynamic backtracking and hierarchical plotting of CAN signal data based on LOD and LTTB includes a message acquisition unit, a message parsing unit, a message storage unit, a signal processing unit, and a display rendering unit; wherein, the signal processing unit includes a signal backtracking unit, a signal storage unit, and a signal sampling unit; The message acquisition unit is used to acquire messages and input them into the message storage unit; The message parsing unit is used to parse messages into signal data and input them into the signal storage unit for storage. The message storage unit is used to store message data. When a signal needs to be queried, the message is queried and input to the signal backtracking unit, and the signal is parsed through the message storage unit. The signal backtracking unit is used to backtrack data by inputting from the message storage unit when it is necessary to query signal data that has not passed through the signal storage unit. The signal storage unit is used to persist the signal data generated by the message parsing unit and the signal backtracking unit, as well as the signal data sampled by the signal sampling unit. The signal sampling unit is used to optimize the sampling of the signal data to be displayed input from the signal storage unit, and generate sampled signal data that can be displayed smoothly. The display rendering unit is used to display the sampled signal data from the signal sampling unit as shown in the above figure.

[0012] The processing method of the above-mentioned processing system includes the following steps: Step S1: The system starts up and begins receiving message data; Step S2: Construct Level 0 data; Step S3: Calculate the level of LOD based on the range of data to be displayed. When LOD is not level 0, obtain the level data corresponding to the cached LOD, obtain the level 0 data corresponding to the invalid data range, and then generate the valid data of the corresponding level according to the LTTB algorithm. This data is then added to the invalid data of the original corresponding level until all data is valid. After this process, submit the data and perform image rendering.

[0013] Furthermore, step S1 specifically includes the following steps: Step S1.1: Transfer the message data to the database and save it to the database folder; Step S1.2: Pass the message data to the parsing thread for parsing; Step S1.3: Pass the parsed signal into the LOD construction thread, and perform level 0 data persistence according to the channel and number corresponding to the signal. Since the timestamp of each signal value and the space occupied by the data as a whole are the same, calculate the current data position to be written based on the number of signals already written, perform address offset, and write the signal data.

[0014] Furthermore, step S2 specifically includes the following steps: Step S2.1: An external thread continuously inserts the latest signal data into the signal data queue; Step S2.2: Check if there is any unsaved signal data in the signal queue. If it exists, proceed to step S2.3; if it does not exist, proceed to step S2.8. Step S2.3: Check if there is a level 0 data file currently open. If there is, proceed to step S2.4; otherwise, proceed to step S2.6. Step S2.4: Check whether the currently opened level 0 data file has reached the storage threshold. If it has not reached the threshold, it is in a writable state and proceed to step S2.5. If it has reached the threshold, it is in a non-writable state and proceed to step S2.7. Step S2.5: Write the signal data to the currently opened data file and clear the data that has been saved and exists in the signal queue. Repeat step S2.2. Step S2.6: Create the next level 0 data file and pre-allocate threshold space, then execute step S2.4; Step S2.7: Close and save the currently open level 0 data file, and then proceed to step S2.3; Step S2.8: End data construction.

[0015] Furthermore, step S2.1 specifically includes the following steps: Step S2.1.1: Level 0 data construction is carried out according to the corresponding order of the signals after the software starts. When the system starts and receives some messages, if the display function of a new signal is added at this time, the new signal processing flow will be triggered. Step S2.1.2: The interface thread triggers an add signal, processes the interface display range, and then notifies the LOD construction thread to build the data. Step S2.1.3: The LOD construction thread calculates the LOD parameters and obtains the range of received but unparsed message data from the message receiving thread; Step S2.1.4: The LOD thread requests message data from the message receiving thread; Step S2.1.5: After receiving the request, the message receiving thread starts to query historical messages from the database and sends them to the parsing thread for signal parsing; Step S2.1.6: The parsing thread transmits the parsed signal data to the message receiving thread; Step S2.1.7: The message receiving thread continues to process until the data is complete, and then sends the missing level 0 data to the LOD construction thread. The LOD construction thread processes and splices the data to ensure the integrity of the curve data.

[0016] Furthermore, step S3 specifically includes the following steps: Step S3.1: When the user slides or zooms the graphical interface, create a query request; Step S3.2: Determine the range of data to be queried and displayed by dragging the X-axis time range of the interface; Step S3.3: Calculate the number of signal points to be sampled for each pixel based on the time range, ensuring that the actual number of drawn points on the Y-axis for each pixel does not exceed 5, and that at least one data point is available. Starting from the lowest X-axis scale value, each pixel displays one signal value. As the user zooms out of the curve interface, the X-axis range expands, and the number of signal points drawn for each pixel increases. When the number of signal points drawn for each pixel reaches 5, LOD is triggered, switching the data level to the next level, denoted as LODn. At this point, the number of points drawn for each pixel is the data in LOD1, meaning each data point in LOD1 can represent 5 original signal data points. As the user continues to zoom out, it will switch to LOD2, LOD3, LOD4, ... until LODn is 0, ensuring that each pixel draws points. The formula for calculating LODn is...

[0017] In the formula, T represents the current display time range, in seconds. t is the minimum time interval of the signal, measured in seconds (s). LODn is the nth LOD level, where n is a natural number; Step S3.4: Determine if it is level 0. If yes, proceed to step S3.12; otherwise, proceed to step S3.5. Step S3.5: Take the LOD level obtained in step S3.3 raised to the power of 5 as the LTTB sampling bucket size. The LTTB sampling bucket size represents the number of original signal points sampled for each pixel. Step S3.6: Query the data index range of the LOD level from the start time to the end time of the signal; Step S3.7: Query whether there is invalid data in the existing hierarchical data; Step S3.8: If invalid data exists, proceed to step S3.9; if invalid data does not exist, proceed to step S3.12. Step S3.9: From the level 0 data, query the data corresponding to the data range required for the currently displayed level; Step S3.10: Using the bucket size calculated in step S3.5, LOD0 is divided into several data buckets. Each bucket contains a corresponding number of raw signal data. For the first bucket, the average signal value of all data within the bucket is calculated first. Then, the signal value point closest to the average value is found, and the corresponding time scale and signal value are used as the first sampling data point. When calculating the second bucket, the average value of the second bucket is calculated, and the average value algorithm of the first bucket is used to obtain the starting point A(X). a ,Y a ) and endpoint C(X)c ,Y c ), and the data points B(X) in the bucket that actually need to be calculated. b ,Y b The set of triangles, using the formula for calculating the area of ​​a triangle,

[0018] Calculate all data in the bucket, and take the point that forms the largest triangle area with the starting point and the ending point as the sampling data. The sampling data point of the current bucket is calculated by using the average of the previous sampling data point and the next bucket. When calculating the last bucket, since the average data of the next bucket is not available, the point with the largest difference between it and the previous sampling data point is selected as the last downsampled data. Step S3.11: Store the sampled data in the data file corresponding to the level and overwrite the original invalid data; Step S3.12: Create a valid dataset; Step S3.13: Display the interface rendering curve.

[0019] Furthermore, in step S2.1.1, the process of triggering the new signal processing is to obtain the timestamp of the message containing the current new signal, retrieve all message data with the same identifier and channel before this data from the historical message data, perform signal parsing on all matching message data, extract the signal value, save it to level 0 data, and then store the data to ensure data integrity.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention achieves full data lifecycle traceability, supports the re-parsing of historical data after DBC file updates, and provides a complete data foundation for in-depth fault diagnosis; 2. This invention eliminates the delay caused by repeated calculations, and will not produce continuous delays in repeated operations within the same time range, achieving millisecond-level response after the first calculation; 3. This invention can completely preserve the visual characteristics of the signal. Even when the data volume is reduced to 0.0002%, it can still accurately retain all key peak points. In signal testing containing periodic fault spikes, the accuracy retention rate of this invention is 100%. 4. This invention can achieve true full-cycle data backtracking, breaking through the limitation of traditional tools that can only display newly added data. It supports the dynamic addition of signals at any stage of analysis and the loading of complete historical data through the original message database. At the same time, it greatly improves the flexibility and efficiency of fault diagnosis and avoids repeated testing due to signal omission. 5. The technical solution of the present invention supports ultra-large-scale data processing, achieves memory-level access performance at hard disk-level storage cost, and supports processing tens of billions of data points on ordinary computers. Attached Figure Description

[0021] Figure 1 This is a system structure diagram of this embodiment; Figure 2 This is a flowchart of the message database recording process in this embodiment; Figure 3 This is a flowchart of the level 0 data construction process in this embodiment; Figure 4 This is a flowchart of the process of combining LOD and LTTB to generate and persist display data in this embodiment; Figure 5 This is the thread timing diagram of the combination of LOD thread and LTTB thread in this embodiment; Figure 6 This is an example diagram of the message data saved to the database in this embodiment; Figure 7 This is an example diagram of some of the parsing results in this embodiment; Figure 8 This is an example diagram of the data file structure of some of the saved signal data and directory index in this embodiment; Figure 9 This is an example diagram of creating an empty data file in this embodiment; Figure 10 This is a diagram of the original data results retrieved from level 0 data in this embodiment; Figure 11 This is a graph showing the data results after LOD1 sampling in this embodiment; Figure 12 shows the rendering curves of the display interface for some levels in this embodiment. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] Example 1 like Figure 1 As shown, this embodiment employs a processing system for dynamic backtracking and hierarchical plotting of CAN signal data based on LOD and LTTB, including a message acquisition unit, a message parsing unit, a message storage unit, a signal processing unit, and a display rendering unit; wherein, the signal processing unit includes a signal backtracking unit, a signal storage unit, and a signal sampling unit; The message acquisition unit is used to acquire messages and input them into the message storage unit; The message parsing unit is used to parse the message into signal data and input it into the signal storage unit for storage; The message storage unit is used to store message data. When a signal needs to be queried, the message is queried and input into the signal backtracking unit, and the signal is parsed through the message storage unit. The signal backtracking unit is used to perform data backtracking through the input of the message storage unit when it is necessary to query signal data that has not passed through the signal storage unit. The signal storage unit is used to persist the signal data generated by the message parsing unit and the signal backtracking unit, as well as the signal data sampled by the signal sampling unit. The signal sampling unit is used to optimize the sampling of the signal data to be displayed input from the signal storage unit to generate sampled signal data that can be displayed smoothly. The display rendering unit is used to display the sampled signal data from the signal sampling unit as shown in the above figure.

[0024] The processing method used by the above system includes the following steps: Step S1: The system starts up and begins receiving message data. That is, when the system starts, the system opens a message receiving thread to receive message information. The message information is received by the message receiving thread. The message database recording flowchart in this embodiment is as follows: Figure 2 The image shown is an example of some message data saved to the database in this embodiment. Figure 6 As shown; Step S1.1: Transfer the message data to the database and save it to the database folder; Step S1.2: Pass the message data to the parsing thread for parsing (e.g., Figure 5 As shown, the message receiving thread sends the message data to the parsing thread, which parses the original message data into signal data. The message receiving thread then continuously loops through receiving and sending actions. Figure 7 The image shown is an example of some of the parsing results in this embodiment; Step S1.3: The parsed signal is passed to the LOD construction thread (i.e., the data is continuously received and organized by the parsing thread from the message receiving thread, and then sent to the LOD construction thread). Based on the channel and number corresponding to the signal, level 0 data is persistently saved (after receiving different signal information from each channel and identifier, the LOD construction thread filters and categorizes them, storing them in the corresponding level 0 data). Since the timestamp and overall space occupied by each signal value are the same, the current data position to be written is calculated based on the number of signals already written, an address offset is applied, and the signal data is written, such as... Figure 8The image shows a partial view of the saved signal data and directory index in this embodiment, along with an example of the data file structure. Step S2: Construct Level 0 data. The specific process is as follows: Figure 3 As shown: Step S2.1: An external thread continuously inserts the latest signal data into the signal data queue. The specific process is as follows: Step S2.1.1: Level 0 data construction is carried out according to the order of the messages after the software starts. When the system starts and receives some messages, if the display function of a new signal is added at this time, the new signal processing flow will be triggered. The new signal processing flow is triggered by obtaining the timestamp of the message where the new signal is located, obtaining all message data with the same identifier and channel before this data from the historical message data, parsing all matching message data, extracting the signal value, saving it to the level 0 data, and then storing the data to ensure data integrity.

[0025] Step S2.1.2: The interface thread triggers an add signal, processes the interface display range, and then notifies the LOD construction thread to build the data. Step S2.1.3: The LOD construction thread calculates the LOD parameters and obtains the range of received but unparsed message data from the message receiving thread; Step S2.1.4: The LOD thread requests message data from the message receiving thread; Step S2.1.5: After receiving the request, the message receiving thread starts to query historical messages from the database and sends them to the parsing thread for signal parsing; Step S2.1.6: The parsing thread transmits the parsed signal data to the message receiving thread; Step S2.1.7: The message receiving thread continues to process until the data is complete, and then sends the missing level 0 data to the LOD construction thread. The LOD construction thread processes and splices the data to ensure the integrity of the curve data. Step S2.2: Check if there is any unsaved signal data in the signal queue. If it exists, proceed to step S2.3; if it does not exist, proceed to step S2.8. Step S2.3: Check if there is a level 0 data file currently open. If there is, proceed to step S2.4; otherwise, proceed to step S2.6. Step S2.4: Check whether the currently opened level 0 data file has reached the storage threshold. If it has not reached the threshold, it is in a writable state and proceed to step S2.5. If it has reached the threshold, it is in a non-writable state and proceed to step S2.7. Step S2.5: Write the signal data to the currently opened data file and clear the data that has been saved and exists in the signal queue. Repeat step S2.2. Step S2.6: Create the next level 0 data file and pre-allocate threshold space, then execute step S2.4; Step S2.7: Close and save the currently open level 0 data file, and then proceed to step S2.3; Step S2.8: End data construction. In this embodiment, the process of combining LOD and LTTB to generate and persist display data is as follows: Figure 4 As shown, in this embodiment, the LOD thread and LTTB thread are combined in terms of thread timing, as follows: Figure 5 As shown; Step S3: Calculate the level of LOD based on the range of data to be displayed. When LOD is not level 0, obtain the level data corresponding to the cached LOD, obtain the level 0 data corresponding to the invalid data range, and then generate the valid data of the corresponding level according to the LTTB algorithm. This data is then added to the invalid data of the original corresponding level until all data is valid. After this process, submit the data and perform image rendering.

[0026] Step S3.1: When the user swipes or zooms the graphical interface, create a query request. Step S3.2: Determine the range of data to be queried and displayed by dragging the X-axis time range of the interface; Step S3.3: Calculate the number of signal points to be sampled for each pixel based on the time range, ensuring that the actual number of drawn points on the Y-axis for each pixel does not exceed 5, and that at least one data point is available. Starting from the lowest X-axis scale value, each pixel displays one signal value. As the user zooms out of the curve interface, the X-axis range expands, and the number of signal points drawn for each pixel increases. When the number of signal points drawn for each pixel reaches 5, LOD is triggered, switching the data level to the next level, denoted as LODn. At this point, the number of points drawn for each pixel is the data in LOD1, meaning each data point in LOD1 can represent 5 original signal data points. As the user continues to zoom out, it will switch to LOD2, LOD3, LOD4, ... until LODn is 0, ensuring that each pixel draws points. The formula for calculating LODn is...

[0027] In the formula, T represents the current display time range, in seconds. t is the minimum time interval of the signal, measured in seconds (s). LODn is the nth LOD level, where n is a natural number; Step S3.4: Determine if it is level 0. If yes, proceed to step S3.12; otherwise, proceed to step S3.5. Step S3.5: Take the LOD level obtained in step S3.3 raised to the power of 5 as the LTTB sampling bucket size. The LTTB sampling bucket size represents the number of original signal points sampled for each pixel. Step S3.6: Query the data index range of the LOD level from the start time to the end time of the signal; Step S3.7: Query whether there is invalid data in the existing hierarchical data; Since this is the first time data is created at LOD1 level, the data file will be an empty file. Figure 9 As shown; Step S3.8: If invalid data exists, proceed to step S3.9; if invalid data does not exist, proceed to step S3.12. Step S3.9: From the level 0 data, query the data corresponding to the required data range for the currently displayed level. like Figure 10 The image shows the original data retrieved. Step S3.10: Using the bucket size calculated in step S3.5, LOD0 is divided into several data buckets. Each bucket contains a corresponding number of raw signal data. For the first bucket, the average signal value of all data within the bucket is calculated first. Then, the signal value point closest to the average value is found, and the corresponding time scale and signal value are used as the first sampling data point. When calculating the second bucket, the average value of the second bucket is calculated, and the average value algorithm of the first bucket is used to obtain the starting point A(X). a ,Y a ) and endpoint C(X) c ,Y c ), and the data points B(X) in the bucket that actually need to be calculated. b ,Y b The set of triangles, using the formula for calculating the area of ​​a triangle,

[0028] For each data point in a bucket, the point that forms the largest area of ​​the triangle with the starting and ending points is selected as the sampled data. The sampled data points for the current bucket are calculated using the average of the previous and next sampled data points. When calculating the last bucket, since the average data from the next bucket is lacking, the point with the largest difference between its average and the previous sampled data point is selected as the last downsampled data point. The resulting data after LOD1 sampling is as follows: Figure 11 As shown, the first data point is the point with the maximum average value in the first bucket, and the second data point is the point with the maximum area obtained by calculating the area of ​​the triangle obtained by the average of the 5 data points in the first bucket, the 5 data points in the next bucket, and the 10th data point in the original data. Step S3.11: Store the sampled data in the data file corresponding to the level and overwrite the original invalid data; Step S3.12: Create a valid dataset; Step S3.13: Display the interface rendering curves. Here, we show the curves obtained at some levels. As shown in Figure 12(a), the curve drawn at the LOD0 level is shown in Figure 12(b), the curve drawn at the LOD1 level is shown in Figure 12(c), and the curve drawn at the LOD3 level is shown in Figure 12(c).

Claims

1. A processing system for dynamic backtracking and hierarchical plotting of CAN signal data based on LOD and LTTB, characterized in that, It includes a message acquisition unit, a message parsing unit, a message storage unit, a signal processing unit, and a display rendering unit; wherein, the signal processing unit includes a signal backtracking unit, a signal storage unit, and a signal sampling unit; The message acquisition unit is used to acquire messages and input them into the message storage unit; The message parsing unit is used to parse the message into signal data and input it into the signal storage unit for storage; The message storage unit is used to store message data. When a signal needs to be queried, the message is queried and input into the signal backtracking unit, and the signal is parsed through the message storage unit. The signal backtracking unit is used to perform data backtracking through the input of the message storage unit when it is necessary to query signal data that has not passed through the signal storage unit. The signal storage unit is used to persist the signal data generated by the message parsing unit and the signal backtracking unit, as well as the signal data sampled by the signal sampling unit. The signal sampling unit is used to optimize the sampling of the signal data to be displayed input from the signal storage unit to generate sampled signal data that can be displayed smoothly. The display rendering unit is used to display the sampled signal data from the signal sampling unit as shown in the above figure.

2. A processing method for a CAN signal data dynamic backtracking and hierarchical plotting processing system based on LOD and LTTB, characterized in that, Includes the following steps: Step S1: The system starts up and begins receiving message data; Step S2: Construct Level 0 data; Step S3: Calculate the level of LOD based on the range of data to be displayed. When LOD is not level 0, obtain the level data corresponding to the cached LOD, obtain the level 0 data corresponding to the invalid data range, and then generate the valid data of the corresponding level according to the LTTB algorithm. This data is then added to the invalid data of the original corresponding level until all data is valid. After this process, submit the data and perform image rendering.

3. The processing method of the CAN signal data dynamic backtracking and hierarchical plotting processing system based on LOD and LTTB according to claim 2, characterized in that, Step S1 specifically includes the following steps: Step S1.1: Transfer the message data to the database and save it to the database folder; Step S1.2: Pass the message data to the parsing thread for parsing; Step S1.3: Pass the parsed signal into the LOD construction thread, and perform level 0 data persistence according to the channel and number corresponding to the signal. Since the timestamp of each signal value and the space occupied by the data as a whole are the same, calculate the current data position to be written based on the number of signals already written, perform address offset, and write the signal data.

4. The processing method of the CAN signal data dynamic backtracking and hierarchical plotting processing system based on LOD and LTTB according to claim 3, characterized in that, Step S2 specifically includes the following steps: Step S2.1: An external thread continuously inserts the latest signal data into the signal data queue; Step S2.2: Check if there is any unsaved signal data in the signal queue. If it exists, proceed to step S2.3; if it does not exist, proceed to step S2.

8. Step S2.3: Check if there is a level 0 data file currently open. If there is, proceed to step S2.4; otherwise, proceed to step S2.

6. Step S2.4: Check whether the currently opened level 0 data file has reached the storage threshold. If it has not reached the threshold, it is in a writable state and proceed to step S2.

5. If it has reached the threshold, it is in a non-writable state and proceed to step S2.

7. Step S2.5: Write the signal data to the currently opened data file and clear the data that has been saved and exists in the signal queue. Repeat step S2.

2. Step S2.6: Create the next level 0 data file and pre-allocate threshold space, then execute step S2.4; Step S2.7: Close and save the currently open level 0 data file, and then proceed to step S2.3; Step S2.8: End data construction.

5. The processing method of the CAN signal data dynamic backtracking and hierarchical plotting processing system based on LOD and LTTB according to claim 4, characterized in that, Step S2.1 specifically includes the following steps: Step S2.1.1: Level 0 data construction is carried out according to the corresponding order of the signals after the software starts. When the system starts and receives some messages, if the display function of a new signal is added at this time, the new signal processing flow will be triggered. Step S2.1.2: The interface thread triggers an add signal, processes the interface display range, and then notifies the LOD construction thread to build the data. Step S2.1.3: The LOD construction thread calculates the LOD parameters and obtains the range of received but unparsed message data from the message receiving thread; Step S2.1.4: The LOD thread requests message data from the message receiving thread; Step S2.1.5: After receiving the request, the message receiving thread starts to query historical messages from the database and sends them to the parsing thread for signal parsing; Step S2.1.6: The parsing thread sends the parsed signal data back to the message receiving thread; Step S2.1.7: The message receiving thread continues to process until the data is complete, and then sends the missing level 0 data to the LOD construction thread. The LOD construction thread processes and splices the data to ensure the integrity of the curve data.

6. The processing method of the CAN signal data dynamic backtracking and hierarchical plotting processing system based on LOD and LTTB according to claim 2, characterized in that, Step S3 specifically includes the following steps: Step S3.1: When the user slides or zooms the graphical interface, create a query request; Step S3.2: Determine the range of data to be queried and displayed by dragging the X-axis time range of the interface; Step S3.3: Calculate the number of signal points to be sampled for each pixel based on the time range, ensuring that the actual number of drawn points on the Y-axis for each pixel does not exceed 5, and that at least one data point is available. Starting from the lowest X-axis scale value, each pixel displays one signal value. As the user zooms out of the curve interface, the X-axis range expands, and the number of signal points drawn for each pixel increases. When the number of signal points drawn for each pixel reaches 5, LOD is triggered, switching the data level to the next level, denoted as LODn. At this point, the number of points drawn for each pixel is the data in LOD1, meaning each data point in LOD1 can represent 5 original signal data points. As the user continues to zoom out, it will switch to LOD2, LOD3, LOD4, ... until LODn is 0, ensuring that each pixel draws points. The formula for calculating LODn is... ； In the formula, T represents the current display time range, in seconds. t is the minimum time interval of the signal, measured in seconds (s). LODn is the nth LOD level, where n is a natural number; Step S3.4: Determine if it is level 0. If yes, proceed to step S3.12; otherwise, proceed to step S3.

5. Step S3.5: The LOD level obtained in step S3.3 is raised to the power of 5 and used as the LTTB sampling bucket size, where the LTTB sampling bucket size represents the number of original signal points sampled for each pixel. Step S3.6: Query the data index range of the LOD level from the start time to the end time of the signal; Step S3.7: Query whether there is invalid data in the existing hierarchical data; Step S3.8: If invalid data exists, proceed to step S3.9; if invalid data does not exist, proceed to step S3.

12. Step S3.9: From the level 0 data, query the data corresponding to the data range required for the currently displayed level; Step S3.10: Using the bucket size calculated in step S3.5, LOD0 is divided into several data buckets. Each bucket contains a corresponding number of raw signal data. For the first bucket, the average signal value of all data within the bucket is calculated first. Then, the signal value point closest to the average value is found, and the corresponding time scale and signal value are used as the first sampling data point. When calculating the second bucket, the average value of the second bucket is calculated, and the average value algorithm of the first bucket is used to obtain the starting point A(X). a ,Y a ) and endpoint C(X) c ,Y c ), and the data points B(X) in the bucket that actually need to be calculated. b ,Y b The set of triangles, using the formula for calculating the area of ​​a triangle, ； Calculate all data in the bucket, and take the point that forms the largest triangle area with the starting point and the ending point as the sampling data. The sampling data point of the current bucket is calculated by using the average of the previous sampling data point and the next bucket. When calculating the last bucket, since the average data of the next bucket is not available, the point with the largest difference between it and the previous sampling data point is selected as the last downsampled data. Step S3.11: Store the sampled data in the data file corresponding to the level and overwrite the original invalid data; Step S3.12: Create a valid dataset; Step S3.13: Display the interface rendering curve.

7. The processing method of the CAN signal data dynamic backtracking and hierarchical plotting processing system based on LOD and LTTB according to claim 5, characterized in that, In step S2.1.1, the process of triggering the new signal processing is to obtain the timestamp of the message containing the current new signal, obtain all message data with the same identifier and channel before this data from the historical message data, perform signal parsing on all matching message data, extract the signal value, save it to level 0 data, and then store the data to ensure data integrity.

Images