Vehicle electronic Trace data storage method and system based on multi-stage compression strategy and readable storage medium

Abstract

The invention provides a vehicle electronic Trace data storage method and system based on a multistage compression strategy and a readable storage medium, and the method comprises the steps: S1, collecting Trace data from a vehicle bus in real time, and analyzing a protocol header to extract metadata, the Trace data comprising a diagnosis log and tracking data; s2, analyzing the repeatability, entropy, periodicity, burstiness and data type of the Trace data on line by adopting a sliding window to form a feature vector; s3, matching and selecting a compression algorithm and a parameter combination according to the feature vector and a rule base; s4, performing lossless or lossy compression according to the compression algorithm selected in the S3 to obtain compressed data; s5, storing the compressed data in a hierarchical manner according to the importance of the data, and establishing a multi-dimensional index; and S6, monitoring the compression delay and the compression ratio in real time and triggering strategy adjustment. The invention aims to realize efficient, real-time and intelligent compression and storage of the automobile electronic Trace data.

Description

Technical Field

[0001] This invention relates to the field of automotive electronic data storage technology, and based on automotive bus data feature technology, particularly to a vehicle electronic trace data storage method, system, and readable storage medium with a multi-level compression strategy. Background Technology

[0002] With the rapid development of automotive intelligence, connectivity, and electrification, the number of electronic control units (ECUs) and the volume of data interaction are exploding, resulting in a dramatic increase in diagnostic, log, and tracking data. This data is crucial for vehicle fault diagnosis, performance optimization, functional verification, and the training of autonomous driving algorithms. However, the storage and transmission of massive amounts of data place enormous pressure on onboard storage systems and network bandwidth. Therefore, efficient data compression technology has become key to solving this problem.

[0003] Current mainstream automotive electronic data recording and analysis tools have significant limitations in compression: Vector's CANoe / CANalyzer uses a "recording filtering + container compression" process, based on offline compression in MDF4 format (compression rate of approximately 74.92%, processing time of approximately 2.04μs), but lacks adaptive capabilities, requires manual parameter configuration, and is difficult to cope with dynamically changing data characteristics; the domestic tool TSMaster supports multi-bus recording, but the details of its online compression and integration with standard container formats are not disclosed, and its compression performance and adaptive capabilities are still unclear.

[0004] Traditional general-purpose compression algorithms such as LZ77, LZW, and Deflate are the foundation of data compression, but they lack targeted optimization for automotive electronics scenarios and suffer from high computational and memory overhead, making it difficult to meet the stringent requirements of in-vehicle systems for real-time performance and low power consumption.

[0005] Therefore, it is necessary to study a multi-level compression strategy for vehicle electronic trace data storage method, system, and readable storage medium to address the shortcomings of existing technologies and solve or mitigate one or more of the aforementioned problems. Summary of the Invention

[0006] In view of this, the present invention provides a vehicle electronic trace data storage method, system and readable storage medium with a multi-level compression strategy, aiming to achieve efficient, real-time and intelligent compression and storage of automotive electronic trace data.

[0007] On one hand, the present invention provides a vehicle electronic trace data storage method with a multi-level compression strategy, the vehicle electronic trace data storage method with a multi-level compression strategy includes the following steps: S1: Collect Trace data from the vehicle bus in real time and parse the protocol header to extract metadata. The Trace data includes diagnostic logs and trace data. S2: Use a sliding window to analyze the repeatability, entropy, periodicity, burstiness, and data type of the Trace data online to form a feature vector; S3: Select the compression algorithm and parameter combination based on the matching of feature vectors and rule base; S4: Perform lossless or lossy compression based on the compression algorithm selected in S3 to obtain compressed data; S5: Store compressed data in hierarchical order based on data importance and create multi-dimensional indexes; S6: Monitor compression latency and compression ratio in real time and trigger strategy adjustments.

[0008] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein S1 specifically includes: acquiring data via CAN, LIN, FlexRay, or an in-vehicle Ethernet bus; using a DMA transfer mechanism and a timestamp to synchronously parse the DLT protocol to extract the ECU ID, session ID, and timestamp to obtain parsed Trace data; determining the data stream integrity and timing consistency based on the parsed Trace data, wherein the data stream is used for subsequent feature analysis.

[0009] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein S2 specifically includes: S21: The repeatability analyzer obtains the repeatability value by calculating the proportion of repeated substrings in the Trace data using a hash table; S22: The entropy calculator evaluates the byte distribution complexity of the Trace data based on the Shannon entropy formula to obtain the entropy value; S23: The periodic detector obtains the periodic value by evaluating the periodic intensity of the Trace data through an autocorrelation function; S24: The burst analyzer obtains the burst value by statistically analyzing the burst length of the Trace data through a sliding window; S25: The data type classifier identifies the Trace data type based on the DLT protocol header information and obtains the data type label; S26: The feature vector is formed by using repetition value, entropy value, periodic value, burst value and data type label.

[0010] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein S3 specifically includes: the decision-maker performs matching in the rule base according to the feature vector, wherein the rule base predefines the feature conditions and the algorithm mapping relationship; if the repetition is greater than the threshold and the entropy is less than the threshold, then a dictionary pattern compression algorithm and parameter combination are selected; if the periodicity is strong, then a differential coding compression algorithm and parameter combination are selected; when the feedback controller monitors the performance indicators and triggers the strategy adjustment when they exceed the threshold, it outputs the optimal compression algorithm and parameter combination for compression execution.

[0011] In addition to the aspects and any possible implementations described above, an implementation is further provided, wherein S4 specifically includes: S41: Performs lossless compression by integrating dictionary classes, lightweight LZ77 classes, and differential compression algorithms; S42: After performing lossy compression on non-critical data, a residual layer is used to perform lossless recompression on the error; S43: Determine the compression path based on the importance of the data; S44: Obtain the compressed Trace data; The data importance classification includes lossless compression for critical levels, limited lossy compression for important levels, a combination of lossless and lightweight lossy compression for general levels, and lossy compression with priority given to non-critical levels.

[0012] In addition to the aspects and any possible implementations described above, an implementation is further provided, wherein S5 specifically includes: S51: The compressed data is stored according to its importance and access frequency using hot storage, warm storage, and cold storage strategies; S52: Create time indexes based on timestamps, type indexes based on data types, importance indexes based on importance levels, and compressed indexes that record compression algorithms and parameter metadata; S53: The multi-dimensional index supports the retrieval and decompression of the compressed data.

[0013] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein S6 specifically includes: an adaptive learner optimizing rule base parameters based on historical data; switching a lightweight compression algorithm when the monitoring queue length and back pressure state exceed thresholds; and adjusting the policy engine decision according to real-time performance indicators to achieve closed-loop adaptive control.

[0014] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein S25 specifically includes: defining dictionary size and compression level parameter templates for DLT Log data; defining differential window and quantization step size parameter templates for CAN periodic data; and selecting the corresponding parameter template according to the data type label and applying it to the feature vector matching.

[0015] As described above and in any possible implementation, a vehicle electronic trace data storage system with a multi-level compression strategy is further provided, for use with the vehicle electronic trace data storage method with the multi-level compression strategy, wherein the vehicle electronic trace data storage system with the multi-level compression strategy includes: The data extraction module collects Trace data from the vehicle bus in real time and parses the protocol header to extract metadata. The Trace data includes diagnostic logs and tracking data. The feature measurement module uses a sliding window to analyze the repeatability, entropy, periodicity, burstiness, and data type of the Trace data online to form a feature vector; The strategy engine module selects a compression algorithm and parameter combination based on the matching of the feature vector with the rule base; The compression execution module performs lossless or lossy compression according to the compression algorithm selected by the strategy engine. The storage management module classifies compressed data into hierarchical storage based on data importance and establishes multi-dimensional indexes. The feedback control module monitors compression latency and compression ratio in real time and triggers strategy adjustments.

[0016] A computer-readable storage medium is provided, on which a vehicle electronic trace data storage program with a multi-level compression strategy is stored. When the program is executed by a processor, it implements the vehicle electronic trace data storage method with the multi-level compression strategy as described above.

[0017] Compared with the prior art, the present invention can achieve the following technical effects: 1. Significantly improved compression performance: Compared with traditional general compression algorithms and existing compression schemes in automotive electronic tools, this invention improves the compression rate by more than 30% by adaptively matching the optimal compression strategy based on data features, greatly reducing the occupation of in-vehicle storage resources; the end-to-end processing latency is controlled within 1.8ms, meeting the real-time requirements of automotive electronic systems; the algorithm maintains stable performance through feedback control and adaptive learning, adapting to dynamic changes in different types of data.

[0018] 2. Balancing data security and storage efficiency: Based on a differentiated compression strategy with a four-level data importance classification, critical security data is stored without loss, while non-critical data achieves a high compression rate, balancing data integrity and storage efficiency; the fine-grained control of the fault tolerance budget keeps data loss within a controllable range, without affecting subsequent diagnostic analysis and algorithm training.

[0019] 3. Strong compatibility and engineering feasibility: Strictly adheres to the AUTOSAR DLT protocol and related diagnostic standards, and can seamlessly integrate with mainstream toolchains such as CANoe and CANalyzer without modifying the existing vehicle system architecture; modular design and hardware acceleration solutions reduce the difficulty of system deployment, and three-level storage strategy and multi-dimensional indexing improve data management efficiency, meeting the needs of engineering implementation.

[0020] 4. Adaptable to complex in-vehicle environments: Through strict delay budget control, back pressure management, and anomaly handling mechanisms, the system maintains stable operation under high load conditions; hardware acceleration and low power consumption design meet the requirements of the in-vehicle environment for computing resources and power consumption, and are suitable for various intelligent and connected automotive electronic systems.

[0021] Of course, any product implementing this invention does not necessarily need to achieve all of the technical effects described above at the same time. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the steps of a vehicle electronic trace data storage method with a multi-level compression strategy provided in an embodiment of the present invention. Figure 2 This is an overall architecture diagram of a vehicle electronic trace data storage system with a multi-level compression strategy provided in one embodiment of the present invention; Figure 3 This is a decision flowchart of a multi-level compression strategy provided in one embodiment of the present invention. Detailed Implementation

[0024] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0025] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0026] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0027] like Figure 1 As shown, this invention provides a vehicle electronic trace data storage method with a multi-level compression strategy, aiming to achieve efficient data storage and fast retrieval through feature analysis of vehicle bus data and dynamic compression strategies. The implementation process of this invention will be described in detail below with reference to specific embodiments to make the objectives, technical solutions, and advantages of this invention clearer.

[0028] In one embodiment, the method provided by this invention primarily processes trace data generated by vehicle electronic systems. Trace data typically includes diagnostic logs and tracking data, and is characterized by its large volume and diverse features. Through steps such as real-time acquisition, feature analysis, dynamic compression, and hierarchical storage, the method addresses the problems of insufficient data storage space and low access efficiency in vehicle electronic systems. The following will describe each step of this method in the logical order of its technical implementation.

[0029] Step S1 is changed to real-time acquisition of communication messages from the vehicle communication bus (such as CAN or LIN), parsing its protocol structure (such as CAN ID, UDS service identifier, or SOME / IP header), extracting metadata such as timestamps, message identifiers, and source / destination nodes, and distinguishing between diagnostic logs (such as UDS responses) and runtime trace data. As the core communication network of the vehicle's electronic system, the bus carries key data reflecting operating status, fault information, and system logs. The acquisition device obtains data through the bus interface, ensuring integrity and real-time performance, and avoiding loss or delay that could affect subsequent analysis; the extracted metadata (such as source, time, and category) provides the basis for feature analysis and compression processing.

[0030] Specifically, in step S11, the data acquisition module supports the acquisition of various vehicle bus protocols, including but not limited to CAN, LIN, FlexRay bus, and in-vehicle Ethernet bus. To improve data acquisition efficiency, a DMA (Direct Memory Access) transfer mechanism is adopted to ensure high-speed data transmission from the bus interface to the processing unit. Simultaneously, a timestamp synchronization mechanism is used to mark data packets to ensure the temporal consistency of the data stream. During the parsing process, for DLT protocol data packets, the ECU ID, session ID, and timestamp information are extracted. This parsed metadata will serve as an important basis for subsequent analysis. For example, in a specific vehicle diagnostic scenario, the data acquisition module obtains a diagnostic log data containing fault codes from the CAN bus. By parsing the DLT protocol header, it extracts the corresponding ECU ID as "ECU_001", session ID as "Session_2023_01", and timestamp as "2023-10-15 14:30:22.123". This information not only identifies the source and time of the data but also provides support for subsequent data integrity verification.

[0031] Further, in step S12, the integrity and timing consistency of the data stream are determined based on the parsed Trace data. Integrity verification primarily checks the continuity of data packets and the checksum field in the protocol header to ensure that data has not been lost or corrupted during transmission. Timing consistency is verified using timestamp information to ensure that data packets are arranged in the order they were collected. For example, during a 5-minute data collection process of vehicle operation, if the system detects a jump in the timestamp of a data packet, possibly due to data loss caused by a bus communication interruption, the system will record this anomaly and mark that data stream as incomplete to avoid interference with subsequent analysis. This processing method ensures the data quality for subsequent feature analysis.

[0032] In one possible implementation, the data acquisition module can also dynamically adjust the acquisition frequency and priority based on different vehicle operating scenarios. For example, when the vehicle is traveling at high speed, priority is given to acquiring trace data related to the powertrain, while when the vehicle is parked, more attention is paid to diagnostic log data. This dynamic adjustment can effectively reduce unnecessary data acquisition, easing the burden on subsequent compression and storage.

[0033] Step S2: The feature measurement module uses a sliding window to analyze the repeatability, entropy, periodicity, burstiness, and data type of the trace data online, forming a feature vector. In vehicle electronic systems, trace data exhibits significant characteristic differences; for example, diagnostic logs may have high repeatability, while real-time tracking data may show obvious periodicity. Quantifying these characteristics through the feature measurement module provides a basis for subsequent compression strategy selection. This step uses a sliding window approach for online analysis, ensuring the analysis process can adapt to dynamic changes in the data flow while reducing computational resource consumption.

[0034] Specifically, in step S21, the repeatability analyzer uses a hash table to count the proportion of duplicate substrings in the trace data to obtain a repeatability value. The repeatability value reflects the frequency of identical content appearing in the data and is usually directly related to the data's compression potential. For example, in a diagnostic log containing a large number of duplicate fault codes, the repeatability analyzer constructs a hash table and counts that a specific byte sequence "0xFF 0x01 0xAA" appears in more than 50% of the data stream, thus calculating a repeatability value of 0.52. This value indicates that the data segment has high compression potential and is suitable for dictionary-based compression methods. It should be noted that the size of the sliding window can be dynamically adjusted according to the characteristics of the data stream. For example, when the data volume is large, the window size can be set to 1000 bytes to balance analysis accuracy and computational efficiency.

[0035] In step S22, the entropy calculator uses the entropy calculation method from information theory to evaluate the complexity of the byte distribution of the trace data and obtain the entropy value. The entropy value measures the randomness or disorder of data; the higher the entropy value, the more random the data and the more difficult it is to compress. For example, in a piece of real-time tracking data, the byte distribution is relatively uniform, and the calculated entropy value is 7.8, close to the maximum value of 8.0, indicating that the data has high randomness and the compression effect may be limited. In another piece of diagnostic log data, the entropy value is only 3.2, indicating that the data distribution is relatively concentrated and suitable for efficient compression. These analysis results provide important reference for subsequent strategy selection.

[0036] Further, in step S23, the periodicity detector evaluates the periodicity of the trace data using an autocorrelation function and outputs a periodicity value, reflecting whether there are regular recurring patterns in the data. For example, vehicle sensors often transmit data at a fixed frequency, exhibiting strong periodicity. In one embodiment, for the tracking data of the engine speed sensor, the detector identifies a pattern that repeats every 500 milliseconds through autocorrelation analysis, calculating a periodicity value of 0.85, indicating high periodicity and suitability for targeted compression methods such as differential coding. This analysis helps to uncover data patterns and improve compression efficiency.

[0037] In step S24, the burst analyzer uses a sliding window to count the burst length of the trace data to obtain a burst value. The burst value describes the characteristic of high-density data occurring within a short period in the data stream. For example, when a vehicle malfunctions, the diagnostic log may generate a large amount of data in a short time. For instance, in a trace data segment after a malfunction, if the burst analyzer detects more than 1000 data packets generated within one second, the calculated burst value is 0.92, indicating that the data has strong burst characteristics. For this type of data, the compression strategy needs to consider real-time performance and low latency to avoid processing bottlenecks.

[0038] In step S25, the data type classifier identifies the type of trace data based on the DLT protocol header information, obtaining data type labels. These labels distinguish the business attributes of the data, such as diagnostic logs, real-time tracking data, or system event records. Different types of data may have different compression requirements and importance levels. For example, in a segment of trace data, the data type classifier, by parsing the DLT protocol header, identifies 60% of the data as diagnostic logs, 30% as CAN periodic data, and the remaining 10% as system event records, labeled as "Log," "CAN_Periodic," and "Event," respectively. These labels, along with repetition values, entropy values, periodicity values, and burstiness values, constitute a feature vector for subsequent compression strategy matching.

[0039] In one embodiment, the feature measurement module can also dynamically adjust the analysis parameters according to the vehicle's operating status. For example, when the vehicle is running at low speed, the data stream changes slowly, and the sliding window can be set to a larger time span to capture long-term trends; while at high speed or under fault conditions, the data stream changes drastically, and the window size should be reduced to improve the real-time performance of the analysis. This adaptive adjustment can better adapt to the data characteristics under different scenarios.

[0040] In step S3, the strategy engine module selects a compression algorithm and parameter combination based on the matching of feature vectors and the rule base. Feature vectors, as a quantitative description of trace data, contain various characteristics of the data, while the rule base predefines the mapping relationship between feature conditions and compression algorithms. Through the matching process of the strategy engine module, the system can select the most suitable compression algorithm and parameter combination based on the specific characteristics of the data, thereby achieving a balance between compression ratio and processing latency.

[0041] Specifically, in step S31, the decision-maker matches the feature vectors against a rule base. The rule base stores various feature conditions and algorithm mappings. For example, for data with high repetition and low entropy, a dictionary-pattern compression algorithm is preferred; for strongly periodic data, a differential coding compression algorithm is preferred. For instance, in a segment of Trace data with feature vectors of "repetition 0.6, entropy 3.5, periodicity 0.2, burstiness 0.1, data type Log", the decision-maker matches the data against the rule base and finds that it meets the conditions of high repetition and low entropy. Therefore, it selects the dictionary-pattern compression algorithm and sets the dictionary size to 4KB as the parameter combination. This matching process ensures a high degree of compatibility between the compression algorithm and the data characteristics.

[0042] In step S32, if the repetition rate is greater than a preset threshold and the entropy value is less than a preset threshold, then the dictionary pattern compression algorithm and parameter combination are selected. The dictionary pattern compression algorithm achieves efficient data compression by constructing a dictionary table of repeating content, making it particularly suitable for processing data with a large number of repeating substrings. For example, in a piece of diagnostic log data, the repetition rate is 0.65, greater than the threshold of 0.5, and the entropy value is 3.0, less than the threshold of 4.0. The decision-maker selects the dictionary pattern compression algorithm and sets the dictionary table capacity to 8KB based on the data size, while also setting the maximum matching length to 32 bytes. This parameter combination can control computational complexity while ensuring the compression ratio.

[0043] Furthermore, in step S33, if the periodicity is strong, a differential coding compression algorithm and parameter combination are selected. Differential coding reduces redundant information by recording the differences between data, making it particularly suitable for processing periodic data. For example, in a segment of tracking data from a vehicle speed sensor, the periodicity value is 0.88, which is greater than the threshold of 0.7. The decision-maker selects the differential coding compression algorithm and sets the differential window size to 10 data points and the quantization step size to 0.1. This parameter setting can fully utilize the periodicity of the data to achieve high compression efficiency.

[0044] In step S34, the feedback controller triggers strategy adjustment when the monitored performance metrics exceed a threshold, outputting the optimal compression algorithm and parameter combination for compression execution. Performance metrics include compression latency and compression ratio. If any metric exceeds a preset range, the system re-evaluates the current strategy and adjusts the algorithm or parameters. For example, during data compression, if the feedback controller detects a compression latency exceeding 100 milliseconds, exceeding the threshold of 50 milliseconds, the system automatically switches to a lightweight compression algorithm with lower computational complexity and adjusts parameters to reduce processing time. This dynamic adjustment mechanism ensures stable system operation under different loads.

[0045] In one possible implementation, the rule base can also be optimized based on historical data. For example, the system periodically analyzes the feature distribution and compression effect of various trace data over the past month, updating the thresholds and mapping relationships in the rule base to adapt to changes in vehicle usage patterns. This self-learning capability can further improve the accuracy of strategy selection.

[0046] In one embodiment, the policy engine module can also preload different subsets of the rule base based on the vehicle's operating scenario. For example, in urban road driving scenarios, the data stream mainly consists of diagnostic logs, and the rule base prioritizes loading policies related to highly repetitive data; while in highway driving scenarios, the data stream mainly consists of periodic tracking data, and the rule base prioritizes loading policies related to periodic data. This scenario-based rule loading method can further improve the efficiency and targeting of policy matching.

[0047] It should be noted that when selecting compression algorithms and parameters, the strategy engine takes into account the usage of hardware resources. For example, when the processing unit load is high, it will prioritize algorithms with lower computational complexity to avoid system overload and thus ensure the stable operation of the system under various working conditions.

[0048] In another embodiment, the strategy engine module can also support user-defined rules. For example, vehicle manufacturers can set specific feature thresholds and algorithm mapping relationships based on the data characteristics of specific vehicle models to achieve a more accurate compression strategy. This customization feature provides greater flexibility for different application scenarios.

[0049] Step S4: The compression execution module performs lossless or lossy compression on the Trace data according to the algorithm and parameters selected by the strategy engine. This strategy is based on the matching results of the data feature vectors and the rule base in the previous steps, which have already determined the optimal compression scheme. This step implements the actual compression accordingly to reduce storage overhead while retaining as much critical information as possible; the specific lossless or lossy method used depends on the importance of the data and business needs, achieving a balance between compression efficiency and data quality.

[0050] Specifically, in step S41, the compression execution module integrates multiple lossless compression algorithms, including dictionary-based compression algorithms, lightweight LZ77-based compression algorithms, and differential coding compression algorithms, to perform lossless compression on the data. The goal of lossless compression is to reduce data volume as much as possible without losing any original information. For example, for a highly repetitive diagnostic log data, the compression execution module, based on the strategy engine's selection, uses a dictionary-based compression algorithm. By constructing a dictionary table containing repeating byte sequences, it replaces the repeating content with dictionary indices, thereby significantly reducing the data volume. In a specific vehicle diagnostic scenario, a trace data set containing a large number of repeating fault codes, originally 10MB in size, was reduced to 3.2MB after dictionary-based compression, achieving a compression ratio of 3.1, while maintaining data integrity. This lossless compression method is particularly suitable for scenarios with high requirements for data accuracy.

[0051] Furthermore, in step S42, for non-critical data, the compression execution module employs lossy compression to improve the compression ratio, and then performs lossless recompression on the compression error through a residual layer. Lossy compression, by discarding secondary information (such as reducing sampling accuracy), is suitable for scenarios where high accuracy is not required; the residual layer records the differences between the original and compressed data and performs lightweight lossless compression on it, supporting partial data recovery. For example, a piece of non-critical tracking data from a vehicle environment sensor, after lossy compression, has its size reduced from 5 MB to 1 MB; its residual data is further losslessly compressed to 0.3 MB. This layered strategy significantly saves storage space while preserving limited but effective data recoverability.

[0052] In step S43, the compression execution module selects the appropriate compression path based on the data importance level (critical, important, general, non-critical): Lossless compression is used at the critical level to ensure complete data fidelity; The importance level allows for limited lossy compression, with errors controlled within an acceptable range; The general-purpose compression combines lossless and lightweight lossy compression, balancing compression ratio and mass. For non-critical applications, lossy compression should be used first to maximize compression efficiency.

[0053] For example, diagnostic logs from the engine control unit are marked as critical and compressed using dictionary-based lossless compression; while vehicle ambient temperature data, classified as non-critical, is compressed using lossy compression. This hierarchical strategy efficiently utilizes storage resources and flexibly adapts to different business needs while ensuring the integrity of critical data.

[0054] In one possible implementation, the compression execution module also supports dynamically switching compression paths. For example, during vehicle operation, if the system detects that storage space is about to run out, the compression execution module will automatically adjust some critical data from limited lossy compression to more aggressive lossy compression to free up more storage space. This dynamic adjustment mechanism can adapt to different storage pressure scenarios.

[0055] In one embodiment, the compression execution module can dynamically select the compression implementation method based on hardware performance: on high-performance processors, dictionary-based algorithms with higher computational complexity are preferred to improve the compression ratio; on low-power embedded devices, lightweight algorithms are selected to reduce resource overhead. This hardware adaptation mechanism ensures the stable and efficient operation of the system on different platforms.

[0056] In step S5, the storage management module hierarchically stores the compressed data according to its importance and builds a multi-dimensional index. The trace data, processed by the compression execution module, has already formed different compression results based on its importance. In this step, the storage management module stores this compressed data in different storage media according to the hierarchical strategy, while simultaneously building a multi-dimensional index for subsequent fast retrieval and access. This hierarchical storage and indexing mechanism effectively improves data management efficiency and meets the diverse data storage and query needs of vehicle electronic systems.

[0057] Specifically, in step S51, the storage management module adopts a three-tiered storage strategy of hot, warm, and cold storage, classifying and storing data according to its importance and access frequency: Hot storage: For highly important, frequently accessed data (such as real-time diagnostic logs), it is stored on high-speed solid-state drives to ensure fast read and write speeds; Warm storage: Data of moderate importance and low frequency of access (such as historical diagnostic records) is stored on mechanical hard drives; Cold storage: Used for non-critical, rarely accessed archived data (such as long-term trace logs), stored on tape or in the cloud.

[0058] For example, engine fault logs from the most recent 24 hours are stored in hot storage, data from one week to one month is stored in warm storage, and data older than one year is archived in cold storage. This strategy effectively optimizes storage resource allocation and improves data access efficiency.

[0059] In step S52, the storage management module establishes multi-dimensional indexes, including time indexes, type indexes, importance indexes, and compression indexes. The time index, based on the timestamp information of data packets, is used to retrieve data in chronological order; the type index, based on data type tags, is used to filter data by business category; the importance index, based on the importance level of the data, is used to manage data by priority; and the compression index records the data compression algorithm and parameter metadata, used for decompression and data recovery. For example, after storing a segment of Trace data, the storage management module generates a set of index information for that data. The time index records the data acquisition time as "2023-10-15 14:30:22.123", the type index is marked as "diagnostic log", the importance index is marked as "critical", and the compression index records the compression algorithm as "dictionary-type compression" and the dictionary size as 4KB. This index information provides multi-faceted query entry points for subsequent data retrieval.

[0060] Furthermore, in step S53, the storage management module supports efficient retrieval and decompression of compressed data through multi-dimensional indexes (such as time range, data type, and importance level). Users or the system can quickly locate target data and use the corresponding decompression algorithm to restore the original content based on the compression index. For example, in fault analysis, when engineers retrieve engine-related diagnostic logs from the past week, the system first filters using the type index, then precisely locates the data using the time index, and finally calls the corresponding decompression algorithm to restore the data. This mechanism significantly improves retrieval accuracy and efficiency.

[0061] In one implementation, the storage management module supports dynamic index updates: when data is migrated between hot, warm, and cold storage, the system automatically updates the index to ensure it is consistent with the actual storage location and meets the data lifecycle management requirements.

[0062] In one embodiment, the module can also optimize storage strategies based on vehicle usage scenarios—for example, for long-haul vehicles (with low data access frequency), data is prioritized for archiving to cold storage to reduce local load; for urban commuter vehicles (with frequent access), hot storage is prioritized to ensure rapid response. This scenario-based strategy more accurately adapts to the usage characteristics of different vehicle types.

[0063] Step S6: The feedback controller monitors the compression delay and compression ratio in real time and triggers strategy adjustments. In vehicle electronic systems, the generation and processing of trace data is a dynamic process, and data characteristics and system load may change over time. In this step, the feedback controller dynamically adjusts the compression strategy and parameters by monitoring key performance indicators of the compression process in real time, ensuring that the system maintains efficient operation under different conditions. This closed-loop control mechanism is one of the important innovations of this invention, effectively addressing complex scenarios in vehicle operation.

[0064] Specifically, in step S61, the adaptive learner optimizes the rule base parameters based on historical data. As the core basis of the policy engine module, the rule base's parameters and mapping relationships need to continuously adapt to changes in data characteristics and the system environment. The adaptive learner periodically updates the thresholds and policies in the rule base by analyzing historical compression records and performance metrics. For example, in the data analysis of the past month, the adaptive learner found that the compression ratio of highly repetitive data did not meet expectations; the system automatically adjusted the repetition threshold from 0.5 to 0.6 and updated the relevant algorithm mapping relationships. This optimization method based on historical data can continuously improve the accuracy of policy selection.

[0065] In step S62, the feedback controller monitors the queue length and backpressure status in real time. If either indicator exceeds a preset threshold, it switches to a lightweight compression algorithm. These two indicators reflect the system load: an excessively long queue or excessively high backpressure indicates that the computational complexity of the current compression algorithm exceeds the processing capacity, requiring timely load reduction. For example, in high-traffic scenarios, when the processing queue exceeds 1000 data packets (threshold 500), the system automatically switches from a dictionary-based algorithm to a lightweight algorithm to reduce computational overhead and alleviate pressure. This mechanism effectively avoids data processing bottlenecks.

[0066] Furthermore, in step S63, the feedback controller adjusts the strategy engine's decisions based on real-time performance metrics to achieve closed-loop adaptive control. Performance metrics include compression latency and compression ratio. If the compression latency is too high, the system will prioritize reducing algorithm complexity; if the compression ratio is insufficient, the system will attempt to adjust parameters or change the algorithm. For example, in a high-speed vehicle operation scenario, if the feedback controller detects a compression latency of 120 milliseconds, exceeding the threshold of 50 milliseconds, the system automatically adjusts the compression parameters, reducing the dictionary size from 8KB to 4KB and increasing the maximum matching length limit to reduce computation time. This closed-loop control method ensures that the system maintains stable performance in dynamic environments.

[0067] In one possible implementation, the feedback controller also supports a multi-level warning mechanism. For example, when the compression latency first exceeds a threshold of 10%, the system issues a low-level warning, indicating a potential performance risk; when the latency exceeds the threshold by 50%, the system issues a high-level warning and immediately triggers policy adjustments. This tiered warning mechanism can detect potential problems early and reduce the probability of system failures.

[0068] In one embodiment, the feedback controller can also dynamically adjust the monitoring frequency based on the vehicle's operating status. For example, when the vehicle is running at low speed or stopped, the data traffic is low, and the monitoring frequency can be set to once per minute; while during high-speed operation or a fault, the data traffic surges, and the monitoring frequency increases to once per second. This adaptive monitoring method can reduce unnecessary resource consumption while ensuring system responsiveness.

[0069] The closed-loop adjustment of the feedback controller can be optimized by incorporating user feedback: for example, vehicle manufacturers can inject custom rules (such as prioritizing the compression ratio of specific data types or limiting compression latency in certain scenarios) based on actual compression performance and storage requirements to improve system applicability.

[0070] In another embodiment, the controller supports cross-vehicle collaborative optimization: in the fleet management system, it aggregates compression performance data of multiple vehicles, extracts common features, and uniformly updates rule base parameters to achieve fleet-level strategy optimization and improve overall compression and storage efficiency.

[0071] In addition, the controller can record detailed logs for each policy adjustment, including the reasons for the adjustment, comparison of performance indicators before and after, and results, providing data support for subsequent system diagnosis and algorithm optimization.

[0072] Regarding the data type classifier's identification of data type labels in step S25, step S251 defines dictionary size and compression level parameter templates for DLT log data. DLT log data typically has high repetition, making dictionary-based compression algorithms suitable. For example, the system presets a parameter template with a dictionary size of 4KB and a medium compression level for DLT log data to achieve a balance between compression ratio and processing speed. In a specific scenario, after a segment of DLT log data is identified, the system directly calls this parameter template for compression, reducing the size of the resulting compressed file from 2MB to 0.6MB, demonstrating a significant compression effect.

[0073] In step S252, a differential window and quantization step size parameter template are defined for the CAN periodic data. CAN periodic data is typically transmitted at a fixed frequency and has strong periodicity, making it suitable for differential encoding compression. For example, the system presets a parameter template for CAN periodic data with a differential window size of 10 data points and a quantization step size of 0.1. In a vehicle speed sensor data processing scenario, the system performs differential encoding compression on the data according to this template, reducing the data volume from 1.5MB to 0.4MB while preserving the periodicity of the data.

[0074] Furthermore, in step S253, a corresponding parameter template is selected based on the data type label and applied to feature vector matching. The data type label, as part of the feature vector, participates in the matching process of the strategy engine along with other feature values. For example, in a segment of mixed trace data, the system identifies one part as DLT log data and the other as CAN periodic data, respectively, and calls the corresponding parameter templates for feature matching, ultimately generating a more targeted compression strategy. This data type-based parameter template selection method can further improve the targeting and efficiency of compression.

[0075] In one embodiment, the data type classifier can also adjust the parameter template based on version differences of the vehicle bus protocol. For example, for older versions of CAN protocol data, due to the simpler data format, the system will choose a smaller differential window and a lower quantization step size; while for newer versions of automotive Ethernet data, due to the higher data complexity, the system will increase the differential window size and quantization step size to adapt to more complex data characteristics.

[0076] In another embodiment, the data type classifier also supports handling unknown data types. For example, when the system encounters an unrecognized Trace data type, it will be marked as "unknown type" by default, and a general lightweight compression parameter template will be applied, while relevant information will be recorded for subsequent analysis. This fault-tolerance mechanism ensures that the system can still function normally when faced with non-standard data.

[0077] It should be noted that the parameter templates for the data type classifier can be extended according to actual application scenarios. For example, in certain special vehicles (such as construction machinery vehicles), trace data may contain more custom types. The system allows users to manually add new data type labels and corresponding parameter templates to meet specific needs.

[0078] In one possible implementation, the system also supports dynamic optimization of parameter templates. For example, the system periodically analyzes the compression effect of various data types. If it finds that the compression ratio of a certain type of data is consistently lower than expected, it will automatically adjust parameters such as the dictionary size or the difference window in the corresponding parameter template. This adaptive optimization method can continuously improve the applicability of the compression strategy.

[0079] Through the above steps, the method provided by this invention achieves end-to-end processing from data acquisition, feature analysis, compression execution to storage management and feedback control. Addressing the diversity and dynamism of vehicle electronic trace data, the system employs a multi-level compression strategy and adaptive adjustment mechanism to minimize storage space usage and improve data retrieval efficiency while ensuring data quality. Particularly in the compression execution and storage management stages, the system flexibly selects compression paths and storage media based on data importance and access frequency, ensuring efficient resource utilization. Furthermore, the closed-loop control mechanism of the feedback controller can respond in real-time to changes in data characteristics and system load, maintaining system stability.

[0080] In one embodiment, the system also supports comprehensive monitoring and logging of the compression and storage processes. For example, the system records information such as the compression algorithm, compression ratio, storage location, and retrieval frequency for each trace data segment, and generates visual reports for users or developers to analyze system performance. This monitoring and logging capability helps users better understand the system's operating status and provides data support for subsequent optimization.

[0081] In another embodiment, the system can also adjust compression and storage strategies according to different stages of the vehicle's lifecycle. For example, during the vehicle's factory testing phase, data importance is generally high, and the system will prioritize lossless compression and hot storage strategies; while in the later stages of vehicle use, data importance gradually decreases, and the system will use more lossy compression and cold storage strategies to conserve resources. This lifecycle management approach can better adapt to the long-term needs of vehicle data management.

[0082] It should be noted that the system fully considers hardware resource limitations when implementing the above functions. For example, in embedded devices, the system limits the number of compression tasks running simultaneously to avoid resource contention; in high-performance server environments, the system fully utilizes the parallel processing capabilities of multi-core processors to accelerate the compression and storage process. This hardware adaptability ensures the system's applicability across different hardware platforms.

[0083] In one possible implementation, the system also supports periodic verification of stored data. For example, the storage management module periodically scans data files in hot and warm storage areas to check for data corruption or missing indexes, and automatically repairs or regenerates the indexes. This data integrity protection mechanism effectively reduces the risk of data loss.

[0084] In one embodiment, the system also supports encrypted storage of compressed data. For example, for critical data, encryption algorithms are further applied after compression to ensure data security during storage and transmission. This security measure meets the high data privacy requirements of vehicle electronic systems.

[0085] Furthermore, the system can restrict access to data in different storage areas based on user permissions. For example, ordinary users can only access general and non-critical data, while senior users or engineers can access critical data. This access control mechanism can further protect sensitive data and prevent unauthorized access.

[0086] In another embodiment, the system also supports periodic migration of cold-stored data. For example, when cold-stored data reaches a certain storage period, the system automatically migrates it to a lower-cost storage medium, such as from tape to cloud backup, while updating the relevant index information. This data migration mechanism can effectively reduce long-term storage costs.

[0087] It should be noted that the system fully considers the real-time requirements of data processing when implementing the above functions. For example, during vehicle operation, the system prioritizes processing data generated in real time, ensuring that the compression and storage processes do not affect the normal operation of the vehicle's electronic systems. This real-time guarantee meets the data management needs in dynamic vehicle environments.

[0088] In one possible implementation, the system also supports optimization of multi-dimensional indexes. For example, the system dynamically adjusts the index structure based on the frequency and pattern of data retrieval, prioritizing the optimization of frequently queried index fields, thereby further improving retrieval speed. This index optimization method can adapt to the usage habits of different users.

[0089] In one embodiment, the system also supports simulation testing of the adjustment strategy for the feedback controller. For example, before updating the rule base parameters or adjusting the strategy, the effect of the new strategy can be tested in a simulation environment to evaluate its impact on compression latency and compression ratio. Only after confirming its accuracy can it be applied to the actual system. This simulation testing mechanism can reduce the potential risks associated with strategy adjustments.

[0090] This invention directly applies to data storage and compression technology in automotive electronic and electrical (E / E) architectures, particularly real-time compression and storage technology for diagnostic logs and trace data (DLT) generated by buses such as Controller Area Networks (CAN / CAN-FD), Local Interconnect Networks (LIN), FlexRay, and in-vehicle Ethernet. This invention does not involve higher-level technical fields such as general data compression algorithm theory, nor does it involve adjacent technical fields such as automotive network communication protocols themselves. Instead, it focuses on the specific technical field of intelligent compression and storage of trace data in automotive electronic systems. The core of this invention lies in dynamically selecting and adjusting compression strategies based on the characteristics of automotive bus data (such as periodicity, event triggering, and data importance) to achieve the optimal balance between real-time performance and compression rate. This invention does not focus on the implementation details of underlying communication protocols but utilizes existing communication protocols (such as CAN, FlexRay, and Ethernet) for data acquisition. Furthermore, this invention does not involve general data compression algorithm theory but applies these algorithms (such as Zstd, FastLZ, and LZW) to the specific scenario of automotive electronics and optimizes and improves them according to the characteristics of the scenario.

[0091] Example 1: This invention aims to address the shortcomings of existing automotive electronic trace data storage technologies in terms of real-time performance, compression ratio, and adaptability, proposing an adaptive compression architecture based on the characteristics of in-vehicle bus data. This architecture intelligently compresses and stores data by online sensing of data periodicity, event triggering, repetition, entropy, and other characteristics, and intelligently selecting the optimal compression algorithm and parameter combination through a strategy engine. While strictly meeting the real-time requirements of automotive electronic systems, it maximizes compression efficiency and effectively addresses the storage and transmission challenges brought about by the explosive growth of in-vehicle data.

[0092] 1. Constructing a multi-level compression strategy based on data importance: This invention aims to establish a four-level data importance classification system that matches the functional safety level of ISO 26262, and on this basis, construct a lossless / lossy hybrid compression system. By employing differentiated compression methods for different types and levels of data importance, this invention aims to balance the integrity of critical data with the high compression ratio requirements of non-critical data, achieving the optimal balance between data security and storage efficiency.

[0093] 2. Ensuring compatibility with existing toolchains and diagnostic standards: This invention aims to strictly adhere to the AUTOSAR DLT protocol, ensuring that the compressed data remains searchable and interpretable, and seamlessly integrates with existing automotive electronics toolchains (such as CANoe and CANalyzer). Simultaneously, this invention also considers compatibility with diagnostic standards such as ODX / UDS / OBD, aiming to provide a complete, engineerable product solution that meets functional safety and compliance requirements.

[0094] like Figure 3 As shown, the technical solution of this invention adopts an adaptive compression architecture of "feature perception - strategy selection - linear compression - feedback calibration", which consists of the following core modules: 1. Data Acquisition Module: This module is responsible for acquiring trace data in real time from various automotive buses such as CAN / CAN-FD, LIN, FlexRay, and automotive Ethernet. To ensure data integrity and timing consistency, this module adopts a DMA (Direct Memory Access) transfer mechanism and supports timestamp synchronization based on hardware clock or Network Time Protocol (NTP). Furthermore, this module integrates a DLT protocol parser, capable of parsing message headers, extension headers, and payloads according to the AUTOSAR standard to extract key metadata such as ECU ID, session ID, and timestamps.

[0095] 2. Feature Measurement Module: This module is the core of this invention for achieving adaptive compression. It employs a sliding window mechanism to analyze multiple dimensions of data characteristics online, such as repetition, entropy, periodicity, and burstiness, and establishes a multi-dimensional feature vector. These features can comprehensively and accurately describe the inherent patterns of automotive bus data, providing a basis for subsequent strategy selection.

[0096] 3. Strategy Engine Module: This module dynamically selects the optimal compression algorithm and parameter combination based on the feature vectors output by the feature measurement module and a pre-defined rule base. The strategy engine combines rule matching with machine learning, enabling rapid decision-making using expert knowledge and continuous optimization of decision accuracy through machine learning models. Furthermore, this module integrates a feedback controller and an adaptive learner, allowing for dynamic adjustment of the strategy based on real-time performance metrics (such as latency and compression ratio) and optimization of rule parameters based on historical data, achieving true intelligence and adaptability.

[0097] 4. Compression Execution Module: This module integrates multiple compression algorithms, including Zstd, FastLZ, LZW, Deflate, etc., and supports hardware acceleration. The algorithm library adopts a layered design, including dictionary / statistical classes, lightweight LZ77 classes, difference classes, and lossy encoding classes, to meet different data types and different compression requirements.

[0098] 5. Storage Management Module: This module is responsible for the hierarchical storage and indexing management of compressed data. It employs a three-tiered storage strategy of hot, warm, and cold storage, storing data on different performance and cost-effective storage media based on the importance and access frequency of the data. Simultaneously, this module establishes multi-dimensional indexes based on time, type, importance, compression algorithm, etc., supporting fast retrieval and efficient decompression of compressed data.

[0099] Adaptive compression algorithm architecture The adaptive compression algorithm architecture of this invention mainly consists of three parts: feature measurement and classifier, policy engine and feedback control, and algorithm library and hardware collaboration.

[0100] Feature Measurement and Classifier: This module includes the following five sub-modules, which work in parallel to perform a comprehensive analysis of data features: Repeatability analyzer: This analyzer counts the proportion of repeated substrings in the data using a hash table. Its threshold range is set between 0.1 and 0.9, with an optimal value of 0.6. High repeatability usually indicates good data compression potential, making it suitable for dictionary-based compression algorithms.

[0101] Entropy Calculator: This calculator, based on the Shannon entropy formula, analyzes the complexity of data byte / symbol distribution. Its threshold range is set between 0.1 and 1.0, with an optimal value of 0.4. Low entropy values ​​indicate uneven data distribution and the presence of obvious statistical regularities, making entropy coding or differential coding suitable.

[0102] Periodicity Detector: This detector evaluates the periodicity and stability of the data stream using methods such as autocorrelation functions. Its threshold range is set between 0.1 and 1.0, with an optimal value of 0.7. Strongly periodic data is suitable for differential coding or compression algorithms specifically designed for periodic signals.

[0103] Burst Analyzer: This analyzer uses a sliding window to statistically measure the burst length and peak value of the data stream. Its threshold range is set from 1 to 1000 frames, with an optimal value of 50 frames. Data with strong bursts requires compression algorithms that can quickly adapt to data changes.

[0104] Data Type Classifier: This classifier identifies the data type based on DLT protocol header information, CAN ID, etc., such as DLT Log / Trace (Diagnostic Log and Trace), CAN cycles / events, GNSS, sensor streams, etc. Different types of data have different characteristics and require different compression strategies.

[0105] Strategy Engine and Feedback Control: This module employs a rule-driven decision-making mechanism and mainly includes the following parts: Rule base: This base predefines a series of feature conditions and algorithm mapping relationships, such as "If the repetition is > 0.6 and the entropy is < 0.4, then select the Zstd dictionary pattern". These rules are derived based on expert knowledge and historical data analysis.

[0106] Decision Maker: This decision maker matches the feature vectors output by the feature measurement module against a rule base, outputting the optimal algorithm selection and parameter configuration. The decision-making process can be simple rule matching or classification prediction based on a machine learning model.

[0107] Feedback Controller: This controller monitors system performance metrics in real time, such as compression latency, queue length, and compression ratio. When these metrics exceed preset thresholds, the controller triggers policy adjustments; for example, it automatically switches to a lightweight algorithm with lower computational requirements when latency is too high.

[0108] Adaptive learner: This learner continuously optimizes the parameters in the rule base based on historical data. For example, by analyzing the compression effect under different data features, it adjusts the thresholds in the rules to improve the accuracy of decision-making.

[0109] like Figure 2 As shown, the multi-level compression strategy of this invention aims to achieve the best balance between data security and storage efficiency by employing differentiated compression methods based on the importance of the data. This strategy mainly includes three parts: data importance classification, hybrid compression architecture, and parameter templates.

[0110] Data Importance Classification: This invention establishes a four-level importance classification system, which corresponds to ISO functional safety levels. Critical level: This includes DTC (Diagnostic Trouble Code), fault-related logs, critical control traces, and other data directly related to safety. This type of data must be lossless compressed to ensure its integrity and reliability, and no information loss is permissible.

[0111] Importance Level: This includes data that is of significant value to diagnosis and analysis, such as security-related events and diagnostic context. This type of data is primarily compressed without loss, but for certain non-critical fields, minimal and controllable information loss (fault tolerance budget ≤ 0.5%) is permissible.

[0112] General level: Includes application logs, routine traces, and other data used for performance analysis and troubleshooting. This type of data uses a combination of lossless and lightweight lossy compression methods, with a fault tolerance budget relaxed to ≤2%.

[0113] Non-critical level: This includes redundant descriptions, test visualization data, etc. This type of data uses a lossy compression strategy, with a fault tolerance budget of up to ≤5%.

[0114] Hybrid compression architecture: This invention adopts a three-layer design of "layer-domain-stream": Lossless layer: Processes critical / important data, with algorithms such as Zstd (level=1-3), LZW, and Deflate selected.

[0115] Lossy layer: processes non-critical data, and algorithms can be selected from video encoding (H.264 / H.265), point cloud geometric encoding, etc.

[0116] Residual layer: Lossless recompression is performed on the errors generated by lossy compression, using LZ77 or arithmetic coding to further explore the compression potential.

[0117] Parameter template: This invention defines specific compression parameters for different data types: DLT Log data: dictionary size 8KB-64KB, compression level 1-5, trigger threshold 1MB.

[0118] CAN periodic data: differential window 2-16 frames, quantization step size 0.1-1.0, refresh cycle 100-1000ms.

[0119] Video data: PSNR≥30dB, GOP size 8-32, ROI priority 1-5.

[0120] Point cloud data: point error ≤ 0.1m, compression ratio target 5:1-20:1.

[0121] Real-time processing flow This invention designs an end-to-end real-time processing pipeline and sets strict delay budgets and backpressure thresholds for each stage.

[0122] End-to-end production line: Acquisition phase: latency budget ≤ 100μs, back pressure threshold queue < 1000 frames.

[0123] Feature metrics: latency budget ≤200μs, backpressure threshold queue <500 frames.

[0124] Strategy selection: latency budget ≤ 50μs, backpressure threshold queue < 100 frames.

[0125] Compressed execution: latency budget ≤ 500μs, backpressure threshold queue < 200 frames.

[0126] Buffer queue: latency budget ≤ 200μs, backpressure threshold queue < 1000 frames.

[0127] Transmission and storage: latency budget ≤ 1ms, backpressure threshold queue < 5000 frames.

[0128] Monitoring and feedback: Delay monitoring: Real-time tracking of P50 / P95 / P99 delay and threshold alarms.

[0129] Queue monitoring: Real-time monitoring of queue length and back pressure status to prevent data overflow.

[0130] Compression ratio monitoring: Real-time statistics on compression ratio and trend analysis.

[0131] Anomaly Handling: When a performance degradation or anomaly is detected, a degradation strategy (such as switching to a lighter algorithm) is automatically triggered and fault recovery is performed.

[0132] Storage Management and Indexing To achieve efficient data storage and retrieval, this invention employs a hierarchical storage strategy and multi-dimensional index management.

[0133] Tiered storage strategy: Hot storage: Used to store recent critical data, using SSD storage, with a retention period of 7-30 days.

[0134] Warm storage: Used to store historical general data, using mechanical hard disk storage, with a retention period of 90-365 days.

[0135] Cold storage: Used for long-term archived data, using magnetic tape or cloud storage, with a retention period of ≥3 years.

[0136] Index Management Time Index: Fast retrieval based on timestamps, supporting queries by time range.

[0137] Type Index: A categorized index based on data type (such as CAN, DLT, video).

[0138] Importance index: A priority index based on importance level, used to quickly locate key data.

[0139] Compression Index: Records metadata such as the compression algorithm and parameters used for each piece of data, which is used for fast decompression.

[0140] The working principle of this invention is based on a closed-loop adaptive control system. First, the data acquisition module obtains trace data from the vehicle's bus in real time via a hardware interface, and utilizes DMA (Direct Memory Access) technology to ensure the integrity and efficiency of data transmission. The timestamp synchronization mechanism, based on a hardware clock or Network Time Protocol (NTP), ensures the consistency of data timing from different buses. The DLT protocol parser strictly follows the AUTOSAR standard to parse the message header, extension header, and payload, extracting key information such as ECU ID, session ID, and timestamp.

[0141] Subsequently, the data stream enters the feature measurement module. This module employs a sliding window mechanism to perform online analysis of the data stream. Repeatability analysis uses a hash table to count repeating substrings, calculating the repeatability as the number of repeating bytes divided by the total number of bytes. Entropy calculation is based on the Shannon entropy formula (H = -Σp(x)log2p(x)) to assess the distributional complexity of the data. Periodicity detection uses the autocorrelation function (ACF(τ) = Σx(t)x(t+τ)) to detect the intensity of data periodicity. Burst analysis uses a sliding window to statistically analyze the distribution of burst lengths to identify anomalous peaks. These analytical results constitute a multi-dimensional feature vector, which is then fed into the strategy engine module.

[0142] The strategy engine module makes decisions based on a combination of rule matching and machine learning. Rule matching maps predefined feature conditions to algorithms; for example, if "repetition > 0.6 and entropy < 0.4", the "Zstd dictionary mode" is selected. Machine learning trains classification models (such as random forests or neural networks) on historical data to achieve more intelligent decision-making. A feedback control mechanism monitors performance metrics such as compression latency and compression ratio in real time, automatically switching compression algorithms when metrics exceed preset thresholds. The adaptive learning module updates rule parameters online to adapt to changes in data distribution. Finally, the compression execution module performs compression using a multi-algorithm parallel processing approach based on the strategy engine's decisions. For example, Zstd compression is based on dictionary matching and fast entropy encoding, supporting negative levels (--fast) for speed; FastLZ compression is a lightweight LZ77 implementation suitable for low-complexity data; Deflate compression is a hardware-friendly combination of LZ77 and Huffman coding, suitable for FPGA acceleration; and LZW compression is a classic dictionary compression algorithm that eliminates the need to transmit a dictionary.

[0143] The foregoing has provided a detailed description of a vehicle electronic trace data storage method, system, and readable storage medium using a multi-level compression strategy, as provided in the embodiments of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas; furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

[0144] Certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that hardware manufacturers may use different names to refer to the same component. This specification and claims do not distinguish components based on differences in name, but rather on differences in function. The terms "comprising" and "including" used throughout the specification and claims are open-ended and should be interpreted as "comprising / including but not limited to". "Approximately" means that within an acceptable margin of error, those skilled in the art can solve the technical problem and substantially achieve the technical effect within a certain margin of error. The following descriptions in the specification are preferred embodiments for carrying out this application; however, these descriptions are for the purpose of illustrating the general principles of this application and are not intended to limit the scope of this application. The scope of protection of this application shall be determined by the appended claims.

[0145] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0146] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0147] The foregoing description illustrates and describes several preferred embodiments of this application. However, as previously stated, it should be understood that this application is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the application concept described herein through the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of this application should be within the protection scope of the appended claims.

Claims

1. A method for storing vehicle electronic trace data using a multi-level compression strategy, characterized in that, The vehicle electronic trace data storage method with the multi-level compression strategy includes the following steps: S1: Collect Trace data from the vehicle bus in real time and parse the protocol header to extract metadata. The Trace data includes diagnostic logs and trace data. S2: Use a sliding window to analyze the repeatability, entropy, periodicity, burstiness, and data type of the Trace data online to form a feature vector; S3: Select the compression algorithm and parameter combination based on the matching of feature vectors and rule base; S4: Perform lossless or lossy compression based on the compression algorithm selected in S3 to obtain compressed data; S5: Store compressed data in hierarchical order based on data importance and create multi-dimensional indexes; S6: Monitor compression latency and compression ratio in real time and trigger strategy adjustments.

2. The vehicle electronic trace data storage method with a multi-level compression strategy according to claim 1, characterized in that, S1 specifically includes: acquiring data via CAN, LIN, FlexRay, or vehicle Ethernet bus; using DMA transfer mechanism and timestamp synchronization to parse DLT protocol to extract ECU ID, session ID, and timestamp to obtain parsed Trace data; determining data stream integrity and timing consistency based on the parsed Trace data, and using the data stream for subsequent feature analysis.

3. The vehicle electronic trace data storage method with a multi-level compression strategy according to claim 1, characterized in that, S2 specifically includes: S21: The repeatability analyzer obtains the repeatability value by calculating the proportion of repeated substrings in the Trace data using a hash table; S22: The entropy calculator evaluates the byte distribution complexity of the Trace data based on the Shannon entropy formula to obtain the entropy value; S23: The periodic detector obtains the periodic value by evaluating the periodic intensity of the Trace data through an autocorrelation function; S24: The burst analyzer obtains the burst value by statistically analyzing the burst length of the Trace data through a sliding window; S25: The data type classifier identifies the Trace data type based on the DLT protocol header information and obtains the data type label; S26: The feature vector is formed by using repetition value, entropy value, periodic value, burst value and data type label.

4. The vehicle electronic trace data storage method with a multi-level compression strategy according to claim 1, characterized in that, S3 specifically includes: the decision-maker performs matching in the rule base according to the feature vector, wherein the rule base predefines the feature conditions and the algorithm mapping relationship; if the repetition is greater than the threshold and the entropy is less than the threshold, then the dictionary pattern compression algorithm and parameter combination are selected; if the periodicity is strong, then the differential coding compression algorithm and parameter combination are selected.

5. The vehicle electronic trace data storage method with a multi-level compression strategy according to claim 1, characterized in that, S4 specifically includes: S41: Performs lossless compression by integrating dictionary classes, lightweight LZ77 classes, and differential compression algorithms; S42: After performing lossy compression on non-critical data, a residual layer is used to perform lossless recompression on the error; S43: Determine the compression path based on the importance of the data; S44: Obtain the compressed Trace data; The data importance classification includes lossless compression for critical levels, limited lossy compression for important levels, a combination of lossless and lightweight lossy compression for general levels, and lossy compression with priority given to non-critical levels.

6. The vehicle electronic trace data storage method with a multi-level compression strategy according to claim 1, characterized in that, S5 specifically includes: S51: The compressed data is stored according to its importance and access frequency using hot storage, warm storage, and cold storage strategies; S52: Create time indexes based on timestamps, type indexes based on data types, importance indexes based on importance levels, and compressed indexes that record compression algorithms and parameter metadata; S53: The multi-dimensional index supports the retrieval and decompression of the compressed data.

7. The vehicle electronic trace data storage method with a multi-level compression strategy according to claim 1, characterized in that, Specifically, S6 includes: an adaptive learner optimizing rule base parameters based on historical data; switching a lightweight compression algorithm when the monitoring queue length and back pressure state exceed thresholds; and adjusting the strategy engine decision based on real-time performance indicators to achieve closed-loop adaptive control.

8. The vehicle electronic trace data storage method with a multi-level compression strategy according to claim 3, characterized in that, S25 specifically includes: defining dictionary size and compression level parameter templates for DLT Log data; defining differential window and quantization step size parameter templates for CAN periodic data; and selecting the corresponding parameter template according to the data type label and applying it to the feature vector matching.

9. A vehicle electronic trace data storage system with a multi-level compression strategy, used in the vehicle electronic trace data storage method with a multi-level compression strategy as described in any one of claims 1-8, characterized in that, The vehicle electronic trace data storage system with the multi-level compression strategy includes: The data extraction module collects Trace data from the vehicle bus in real time and parses the protocol header to extract metadata. The Trace data includes diagnostic logs and tracking data. The feature measurement module uses a sliding window to analyze the repeatability, entropy, periodicity, burstiness, and data type of the Trace data online to form a feature vector; The strategy engine module selects a compression algorithm and parameter combination based on the matching of the feature vector with the rule base; The compression execution module performs lossless or lossy compression according to the compression algorithm selected by the strategy engine. The storage management module classifies compressed data into hierarchical storage based on data importance and establishes multi-dimensional indexes. The feedback control module monitors compression latency and compression ratio in real time and triggers strategy adjustments.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a vehicle electronic trace data storage method program with a multi-level compression strategy. When the vehicle electronic trace data storage method program with a multi-level compression strategy is executed by a processor, it implements the steps of the vehicle electronic trace data storage method with a multi-level compression strategy as described in any one of claims 1 to 8.

Images