Commercial vehicle fleet cooperative communication data intelligent compression method

Abstract

This invention belongs to the field of compression technology, specifically relating to an intelligent compression method for collaborative communication data of commercial vehicle fleets. The method includes the following steps: constructing and maintaining a three-layer heterogeneous data dictionary, comprising a pre-set global basic dictionary containing a general high-frequency dataset; a fleet semantic dictionary generated periodically by extracting high-frequency data sequences from historical fleet communication data using a time-series correlation and information entropy weighted model; and a spatiotemporal proximity dictionary that aggregates confirmed successful interactions between the vehicle and neighboring nodes based on the vehicle's real-time location and communication topology, and updates it using a first-in-first-out (FIFO) strategy. This invention simplifies the decoding process by uniformly encapsulating encoding results from different sources, achieving deep compression of commercial vehicle fleet communication data streams overall, and reducing communication bandwidth usage and transmission costs.

Description

Technical Field

[0001] This invention belongs to the field of compression technology, specifically relating to an intelligent compression method for collaborative communication data of commercial vehicle fleets. Background Technology

[0002] Commercial vehicle fleets are an integral part of modern logistics and public transportation. The application of collaborative driving within and between fleets requires real-time information exchange between vehicles. This exchange data includes vehicle location, speed, acceleration, driving intentions, sensor data, and control commands. This intensive data exchange puts pressure on limited wireless communication channel resources. In areas with high vehicle density or poor network conditions, it can easily lead to network congestion, packet loss, and transmission delays, affecting the safety and efficiency of collaborative fleet control.

[0003] Dictionary-based compression algorithms can leverage the high repetition and pattern recognition inherent in vehicular data streams. However, when applying general-purpose algorithms to commercial fleet cooperative communication scenarios, existing methods mostly employ a single, static, or slowly updated dictionary structure, failing to simultaneously adapt to and utilize the multi-layered relational characteristics coexisting in the data: these include both long-term, global semantic patterns determined by shared fleet tasks and driving habits, and short-term, local spatiotemporal data similarities determined by real-time proximity relationships between vehicles. In terms of matching strategies, a greedy approach is often adopted, selecting the longest match at the current position. This strategy is prone to getting trapped in local optima, potentially missing encoding paths that could achieve higher overall compression ratios. For data that cannot be matched by the dictionary, the data is typically encoded as raw literals, failing to fully explore and utilize the predictability of the data based on its own historical state or the states of neighboring vehicles. This results in missed opportunities to compress data through prediction and residual encoding. These shortcomings lead to the compression efficiency of existing technologies not reaching optimal levels in heterogeneous, multimodal commercial fleet communication environments. Summary of the Invention

[0004] This invention provides a commercial vehicle fleet cooperative communication data intelligent compression method to solve the technical problem that existing technologies fail to fully utilize the multi-level correlation characteristics of data and the predictability of data based on its own historical state or the state of neighboring vehicles.

[0005] This invention provides an intelligent compression method for cooperative communication data in commercial vehicle fleets, comprising the following steps: Construct and maintain a three-layer heterogeneous data dictionary, which includes a pre-built global basic dictionary containing a general high-frequency dataset, a fleet semantic dictionary generated periodically by extracting high-frequency data sequences by applying a time-series correlation and information entropy weighted model to the fleet's historical communication data, and a spatiotemporal proximity dictionary that aggregates confirmed successful interactions between the vehicle and its neighboring nodes based on the vehicle's real-time location and communication topology and is updated using a first-in-first-out strategy. For compressed data streams, all matching items in the three-layer heterogeneous data dictionary are searched in parallel. A directed graph with data stream positions as nodes is constructed based on the encoding cost function. The optimal matching item is determined and output through the minimum cost path. If no valid match is found in any dictionary, for continuous numerical data, based on the recent historical volatility of the data stream, a prediction model based on the historical value of this vehicle or a prediction model based on the average change of data of nearby vehicles in time and space is selected, the residual between the predicted value and the actual value is calculated, and the residual is output as a quantized code after non-uniform quantization. For discrete state variables and non-numerical data, they are output as text variables. The generated dictionary matches, residual quantization codes, or text values ​​are sequentially encoded into uniform format data units containing type identifiers and then combined in order into a compressed data stream.

[0006] Its effect is that by constructing a three-layer heterogeneous dictionary with a global foundation, fleet semantics, and spatiotemporal proximity, it overcomes the problem that a single static dictionary in the existing technology cannot take into account multi-level data association. In the scenario of cooperative driving of commercial fleets, it can simultaneously capture redundant information generated by industry standards, specific fleet habits, and real-time neighbor vehicle interactions, and explore compression potential from multiple time and space dimensions, which greatly reduces the occupation of limited wireless bandwidth by high-frequency V2X communication.

[0007] Furthermore, a fleet semantic dictionary, generated periodically by applying a time-series correlation and information entropy weighted model to extract high-frequency data sequences from historical fleet communication data, includes: Historical communication data of vehicles within the fleet is periodically collected as input dataset; By applying a frequent sequence mining algorithm and setting a minimum support threshold, frequent data sequences exceeding a predetermined length can be extracted from the input dataset. For each frequently mined data sequence, the normalized support and normalized information entropy of the frequently mined data sequence are comprehensively evaluated, and a comprehensive score is calculated. Select the most frequent data sequences with the highest overall scores and update the fleet semantic dictionary as entries.

[0008] Its effects are as follows: by using an information entropy weighted model to dynamically maintain the fleet semantic dictionary, it can more accurately identify high-frequency communication patterns with strong determinism compared to the simple frequency statistics method. In long-term fleet operations, this method can automatically filter meaningless random noise data, ensuring that dictionary entries always maintain high compression value and improving the compression stability of fleet-specific business flows.

[0009] Furthermore, a spatiotemporal proximity dictionary is created by aggregating communication data segments of confirmed successful interactions between the vehicle and neighboring nodes based on the vehicle's real-time location and communication topology, and updating the dictionary using a first-in-first-out (FIFO) strategy. This dictionary includes: The vehicle periodically broadcasts a heartbeat packet containing its own location information; Other vehicles within a predetermined communication range are identified as neighboring nodes; Allocate a pre-defined buffer for the spatiotemporal proximity dictionary, managed using a first-in-first-out (FIFO) strategy; Extract and aggregate the communication data segments that have been confirmed to have been successfully interacted between this vehicle and neighboring nodes, and store the communication data segments and source identifiers as an entry in the cache area; When the buffer is full, the insertion of a new entry will cause the oldest entry to be removed.

[0010] Its effects are as follows: The introduction of a spatiotemporal proximity dictionary can capture the instantaneous communication data redundancy between fleet members due to their proximity in real time. Compared with the preset dictionary, this dictionary is dynamically updated with the vehicle topology. Especially in complex traffic scenarios where vehicles frequently cut in or change formation, it can quickly match and compress sensor synchronization fragments that are repeatedly interacting between neighboring nodes, effectively alleviating local network congestion.

[0011] Furthermore, the encoding cost function includes: Encoding cost function Where L is the matching length, For dictionary level weights, α represents the number of bits after encoding, and α, β, and γ are preset weighting coefficients. Setting function To match a monotonically decreasing function of length L; Set dictionary hierarchy weights Among them, the global basic dictionary has the lowest weight, the fleet semantic dictionary has a medium weight, and the spatiotemporal proximity dictionary has the highest weight. Set the number of bits after encoding The total number of bits required to encode the dictionary index, hierarchy identifier, and match length of the match.

[0012] Its effect is that by defining a multi-dimensional encoding cost function, the matching length, dictionary level weight and bit overhead are quantitatively balanced. Compared with the existing strategy that only pursues the matching length, the present invention can prioritize the encoding of global or semantic entries with higher stability and lower parsing cost in the heterogeneous communication environment of vehicle networking, thereby achieving a better balance between compression rate and computational robustness.

[0013] Furthermore, for the compressed data stream, all matching items in the three-layer heterogeneous data dictionary are searched in parallel. A directed graph with data stream positions as nodes is constructed based on the encoding cost function. The optimal matching item is determined and output through the minimum cost path, including: Starting from the current position in the data stream to be compressed, search in parallel for all possible matching strings in the three-layer heterogeneous data dictionary; For each found match, calculate the matching item encoding cost based on the cost function; Construct a directed acyclic graph with data stream positions as nodes and possible matching items as weighted edges, where the weight of the edge is the encoding cost of the corresponding matching item; The shortest path algorithm is applied to determine the minimum cost path from the current node to the end node of the data stream, and the first match of the minimum cost path is selected as the optimal match output for the current position.

[0014] Its effects are as follows: It introduces a directed graph path search based on a cost function, which solves the problem that the greedy algorithm used in the existing technology is prone to getting trapped in local optima. When processing complex commercial vehicle bus data streams, this method finds the optimal dictionary matching combination from a global perspective through the shortest path algorithm, ensuring that the encoding path of the entire frame data stream has the minimum bit overhead, and significantly improving the overall compression limit.

[0015] Furthermore, for continuous numerical data, based on the recent historical volatility of the data stream, a prediction model based on the vehicle's historical values ​​or a prediction model based on the average change of data from vehicles in the same time and space is selected to calculate the predicted values, including: Obtain recent historical data points for the current continuous numerical data and calculate the normalized variance; Set corresponding normalized variance thresholds for different types of data. If the calculated normalized variance is less than the normalized variance threshold, the data flow is determined to be stable, and a linear extrapolation prediction model based on the historical values ​​of this vehicle is selected to calculate the predicted value. If the calculated normalized variance is greater than or equal to the normalized variance threshold, then data stream fluctuation is determined. It is then determined whether the continuous numerical data belongs to a preset physical quantity with fleet spatial correlation. If so, a prediction model based on the mean change of data of spatiotemporally neighboring vehicles is selected to obtain the latest change of a predetermined number of neighboring vehicles with respect to the data item within the communication range. Based on the positive and negative correlation of physical quantities in the fleet spatial topology and spatial phase difference, the change of neighboring vehicles is compensated for time delay and weighted to obtain a comprehensive change. The comprehensive change is then superimposed on the latest value of the current vehicle as the predicted value. If not, a higher-order extrapolation prediction model based on the historical data of the current vehicle is selected to calculate the predicted value.

[0016] Furthermore, the residual is non-uniformly quantized and then output as a residual quantization code, including: The residual is calculated by subtracting the predicted value from the actual value; The quantization interval to which the residual belongs is determined based on the dedicated non-uniform quantization boundary preset for continuous numerical data; The residual is mapped to the preset quantization encoding value corresponding to the quantization interval and then output.

[0017] Furthermore, the generated dictionary matches, residual quantization codes, or literal values ​​are sequentially encoded into uniform format data units containing type identifiers, including: Each data unit is assigned a type identifier prefix to distinguish its data type; For dictionary matches, the data unit format is a type identifier, a hierarchy identifier, a dictionary index field, and a match length field; For residual quantization coding, the data unit format is a type identifier, a data item ID, and a residual quantization coding value; For text, the data unit is formatted as a type identifier, a length field indicating the length of the original data, and one or more bytes of the original data.

[0018] Furthermore, the global basic dictionary is constructed by offline analysis of historical CAN bus data and SAE J1939 standard messages of commercial vehicles. The fixed and high-frequency communication identifiers, control commands, and standard fault codes are extracted using the word frequency statistics TF-IDF algorithm and stored in a hash table structure.

[0019] Furthermore, the Aho-Corasick algorithm is used to search all matching items in the three-layer heterogeneous data dictionary in parallel.

[0020] The beneficial effects are as follows: This invention constructs a three-layer heterogeneous data structure consisting of a global basic dictionary, a fleet semantic dictionary, and a spatiotemporal proximity dictionary. This enables the detection and utilization of multi-level redundant information in commercial fleet communication data from three dimensions: general, fleet-specific, and local real-time. The cost function-based matching strategy employed finds the optimal encoding path for the data stream through global optimization, achieving a higher compression ratio. For data that fails to match, continuous numerical values ​​and discrete state variables are distinguished, and predictive coding is applied to compress the numerical residuals, handling new or abrupt data that cannot be covered by the dictionary. By unifying and encapsulating encoding results from different sources, the decoding process is simplified, achieving deep compression of commercial fleet communication data streams overall, reducing communication bandwidth usage and transmission costs. Attached Figure Description

[0021] Figure 1 A flowchart for an intelligent compression method for collaborative communication data in commercial vehicle fleets. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] An embodiment of the intelligent compression method for cooperative communication data of commercial vehicle fleets provided by the present invention: like Figure 1 As shown, the intelligent compression method for collaborative communication data in commercial vehicle fleets includes the following steps: S1 constructs and maintains a three-layer heterogeneous data dictionary, which includes a pre-built global basic dictionary containing a general high-frequency dataset, a fleet semantic dictionary generated periodically by extracting high-frequency data sequences from fleet historical communication data through the application of a time-series correlation and information entropy weighted model, and a spatiotemporal proximity dictionary that aggregates confirmed successful interactions between the vehicle and neighboring nodes based on the vehicle's real-time location and communication topology and is updated using a first-in-first-out strategy.

[0024] The global basic dictionary is constructed by offline analysis of historical CAN bus data and SAE J1939 standard messages from commercial vehicles. The TF-IDF algorithm is used to extract fixed and high-frequency communication identifiers, control commands, and standard fault codes, which are then stored as a hash table structure. The fleet semantic dictionary is generated locally on the vehicle terminal. The terminal caches the communication logs of all vehicles in the fleet over the past 24 hours, and an update process is triggered daily at midnight. The PrefixSpan sequence pattern mining algorithm is used to find frequent itemsets longer than 4 bytes, and the Shannon information entropy formula is then used to calculate the information entropy of each itemset. and divide by the maximum possible entropy of the sequence. We obtain the normalized information entropy, where, The symbol index within the sequence, with a value range of 1. , The total number of distinct symbols within the sequence. Let i be the i-th symbol in the sequence. For symbols The probability of appearing in the sequence. The sequence length in bytes. Sequences with high frequency and low information entropy—that is, deterministic and frequently occurring patterns—are selected and stored in a finite-size hash table as the fleet semantic dictionary. The spatiotemporal proximity dictionary is a fixed-size, for example, double-ended queue data structure `collections.deque` with 256 entries. When the vehicle successfully receives and verifies a neighboring vehicle broadcast data packet with a correct CRC checksum via C-V2X or DSRC communication, the data payload of that packet is pushed onto the right side of the queue as an entry. If the queue is full, the oldest entry is popped from the left, implementing a first-in, first-out (FIFO) update.

[0025] In some implementations, a fleet semantic dictionary is generated periodically by applying a time-series correlation and information entropy weighted model to extract high-frequency data sequences from historical fleet communication data. This dictionary includes: Historical communication data of vehicles within the fleet is periodically collected as input dataset; By applying a frequent sequence mining algorithm and setting a minimum support threshold, frequent data sequences exceeding a predetermined length can be extracted from the input dataset. For each frequently mined data sequence, the normalized support and normalized information entropy of the frequently mined data sequence are comprehensively evaluated, and a comprehensive score is calculated. Select the most frequent data sequences with the highest overall scores and update the fleet semantic dictionary as entries.

[0026] The fleet management center server or a designated lead vehicle node acts as the dictionary generation unit. Every preset period, such as 1 hour or 200 kilometers traveled, it actively collects and integrates communication data recorded by all vehicles in the fleet during this period. This data mainly includes CAN bus messages and V2X interaction messages, resulting in a raw dataset indexed by timestamps and vehicle IDs. A frequent sequence mining algorithm, such as PrefixSpan, is used to process the dataset. During this process, a minimum support threshold is set, such as 0.05, to filter out rare data sequences; a minimum sequence length is also set, such as no less than 5 bytes, to ensure that the mined sequences have practical compression value. A series of frequent data sequences that meet the above conditions are output.

[0027] For each frequently mined data sequence S, a comprehensive score Score(S) is calculated. This score is composed of a weighted average of normalized support and information entropy, and the calculation formula is as follows: ,in and For weighting coefficients, the preferred value is... , N is the total number of sequences in the dataset, and H(S) is the information entropy of sequence S. The length of the sequence in bytes. The original support level, Normalized support represents the frequency of a sequence's occurrence, while normalized information entropy measures the uncertainty of a sequence. Sequences with high scores exhibit both high frequency and low uncertainty, indicating strong determinism. All sequences are ranked in descending order based on their combined scores, and a predetermined number of the highest-ranking sequences, such as the top 256, are selected as entries for the new semantic dictionary of the fleet, replacing the old dictionary from the previous period, thus completing the update.

[0028] In some implementations, a spatiotemporal proximity dictionary is updated using a first-in-first-out (FIFO) strategy, which aggregates communication data segments of confirmed successful interactions between the vehicle and neighboring nodes based on the vehicle's real-time location and communication topology. This includes: The vehicle periodically broadcasts a heartbeat packet containing its own location information; Other vehicles within a predetermined communication range are identified as neighboring nodes; Allocate a pre-defined buffer for the spatiotemporal proximity dictionary, managed using a first-in-first-out (FIFO) strategy; Extract and aggregate the communication data segments that have been confirmed to have been successfully interacted between this vehicle and neighboring nodes, and store the communication data segments and source identifiers as an entry in the cache area; When the buffer is full, the insertion of a new entry will cause the oldest entry to be removed.

[0029] Each vehicle broadcasts a heartbeat packet via a V2V channel at a fixed high frequency, e.g., once per second, containing its vehicle ID, GPS latitude and longitude coordinates, timestamp, and current speed. Upon receiving the heartbeat packets from other vehicles, the vehicle terminal calculates the geographical distance using the Haversine formula. The Haversine formula, based on the semi-sine function, calculates the great circle distance between two points on the Earth's surface using their latitude and longitude coordinates. It is a spherical distance calculation method in geographic computing and navigation that balances accuracy and computational efficiency. If this distance is less than a preset communication threshold, e.g., 500 meters, the other vehicle is considered a spatiotemporally neighboring node. A fixed-size buffer, e.g., 4KB, is allocated in memory for the spatiotemporally neighboring dictionary, and a circular queue or doubly linked list data structure is used to implement a first-in, first-out (FIFO) management strategy, ensuring that insertion and deletion operations have a time complexity of O(1).

[0030] When a vehicle communicates with any neighboring node, and the communication is confirmed as a successful interaction via an application-layer acknowledgment message, the data payload of that interaction is extracted as a communication data fragment. This data fragment, along with the source vehicle's ID, is encapsulated into a dictionary entry in the format [Source_Vehicle_ID, Data_Fragment]. This entry is then stored at the tail of the first-in, first-out (FIFO) buffer. If the buffer is full at this point, i.e., the stored data volume has reached the 4KB limit, the oldest entry at the head of the queue is removed to make room for the new entry. Through this mechanism, the spatiotemporal proximity dictionary always maintains the latest and most relevant local communication data, addressing sudden or highly repetitive communication scenarios within the region.

[0031] S2, for the compressed data stream, searches all matching items in the three-layer heterogeneous data dictionary in parallel, constructs a directed graph with the data stream position as the node based on the encoding cost function, and determines the optimal matching item through the minimum cost path and outputs it.

[0032] Construct a Directed Acyclic Graph (DAG), where the nodes are the position indices of each byte in the data stream to be compressed, ranging from 0 to N-1. Starting from the current position index i, search in parallel across three levels of dictionaries for all matching terms prefixed with the data sequence starting at position i. The search employs the Aho-Corasick algorithm to implement multi-pattern parallel matching. For each found matching term, the matching length is L, and the matching term corresponds to a directed edge from node i to node i+L. The weight of the edge is determined by the encoding cost function. The calculation yielded the following. Assuming dictionary hierarchy weights, the weight of the spatiotemporal proximity dictionary is set to 5, the weight of the vehicle semantic dictionary is 3, and the weight of the global basic dictionary is 1. This corresponds to the size of the dictionary. Let be the matching length. After all possible matching edges are constructed, Dijkstra's algorithm is used to calculate the shortest weighted path from the current node i to the end node N of the data flow. To implement this calculation, the minimum cost from each node to the destination is calculated and cached in advance using dynamic programming from back to front. ,in, This represents the minimum encoding cost from the i-th byte position in the data stream to the end node of the data stream. This represents the encoding cost of the current match. Indicates the length of the current match. Indicates the position from the end of the current match. The minimum encoding cost to the end of the data stream. Based on the calculated shortest path, the matching item corresponding to the first edge starting from node i is selected as the optimal result of this compression, and the dictionary index, level identifier, and matching length of the encoding information are output.

[0033] In an alternative embodiment, the encoded cost function includes: Encoding cost function Where L is the matching length, For dictionary level weights, α represents the number of bits after encoding, and α, β, and γ are preset weighting coefficients. Setting function To match a monotonically decreasing function of length L; Set dictionary hierarchy weights Among them, the global basic dictionary has the lowest weight, the fleet semantic dictionary has a medium weight, and the spatiotemporal proximity dictionary has the highest weight. Set the number of bits after encoding The total number of bits required to encode the dictionary index, hierarchy identifier, and match length of the match.

[0034] The complete expression for the encoding cost function C(m) is: The weighting coefficients are optimized based on empirical data, mapping different terms to a unified cost value space. Typical preset values ​​are α=0.5, β=1.0, and γ=0.8, which balance the relationship between matching length, dictionary priority, and encoding overhead. The function f(L) is 1 / L, ensuring that the longer the matching length L, the smaller the contribution of the cost term α / L, thus selecting longer matches. Dictionary-level weights. Assigning specific numerical values ​​to reflect priority: weights of the spatiotemporal proximity dictionary The weight of the vehicle semantic dictionary is set to 5. Set the weight to the median value of 3, and the weight of the global basic dictionary. Setting it to 1 prioritizes matches from a more stable and universal global base dictionary when other conditions are similar, thereby minimizing encoding overhead.

[0035] Number of bits after encoding The calculation method is as follows: the hierarchical identifier uses a fixed 2 bits, for example, 11 represents spatiotemporal proximity, 10 represents fleet semantics, and 01 represents global basis; the dictionary index uses 2 bits. ,in This corresponds to the number of entries in the dictionary. For example, if the spatiotemporal proximity dictionary has 512 entries, then the index requires 9 bits. The encoding range for the match length is 3 to 258 bytes, represented using 8 bits. Therefore, for a match of 16 bytes found in the spatiotemporal proximity dictionary, Bits. The cost function is calculated as follows:

[0036] In some implementations, for the data stream to be compressed, all matching items in a three-layer heterogeneous data dictionary are searched in parallel. A directed graph with data stream positions as nodes is constructed based on the encoding cost function. The optimal matching item is determined and output through the minimum cost path, including: Starting from the current position in the data stream to be compressed, search in parallel for all possible matching strings in the three-layer heterogeneous data dictionary; For each found match, calculate the matching item encoding cost based on the cost function; Construct a directed acyclic graph with data stream positions as nodes and possible matching items as weighted edges, where the weight of the edge is the encoding cost of the corresponding matching item; The shortest path algorithm is applied to determine the minimum cost path from the current node to the end node of the data stream, and the first match of the minimum cost path is selected as the optimal match output for the current position.

[0037] The substring of the data stream, starting from position i, is matched in parallel with all entries in the global base dictionary, the fleet semantic dictionary, and the spatiotemporal proximity dictionary. This parallel search can be implemented based on a hash table or an optimized Aho-Corasick algorithm to quickly find all possible matches and their lengths L. For each successfully found match m, the encoding cost is calculated using the aforementioned cost function C(m).

[0038] Construct a Directed Acyclic Graph (DAG) with byte positions in the data stream as nodes. The node set of the graph is {i, i+1, ..., N}, where N is the total length of the data stream. For each match of length L found at position j, create a directed edge from node j to node j+L in the graph, and set the weight of this edge to the encoding cost C of the match. If no match is found at position j, create a literal edge from j to j+1, with a weight equal to the cost of encoding a single raw byte, such as the type identifier overhead + 8 bits. After construction, apply Dijkstra's algorithm or a DAG planning algorithm to calculate the minimum cumulative cost path from the current node i to the destination node N, determine the first edge constituting the optimal path, and the match corresponding to this edge is determined as the optimal match for the current position i. Output the dictionary index, level identifier, and match length of the encoding information. The compression position of the data stream will be moved forward to after this match, and the above process will be repeated.

[0039] S3. If no valid match is found in any dictionary, for continuous numerical data, based on the recent historical volatility of the data stream, select a prediction model based on the historical value of this vehicle or a prediction model based on the average change of data of nearby vehicles in time and space, calculate the residual between the predicted value and the actual value, and output the residual quantization code after non-uniform quantization. For discrete state variables and non-numerical data, output them as text variables.

[0040] When the first byte of the data stream cannot be found in any dictionary, check the metadata tag of the data to determine its type. For continuous numerical data such as vehicle speed or engine speed, calculate the variance of that data item over the past 10 sampling points for this vehicle. Simultaneously, it calculates the change of this data item for all neighboring vehicles within the communication range during the previous sampling period. The overall compensation amount and variance .like Below If the data for this vehicle is stable, a linear predictive coding (LPC) model based on least squares is used, employing the function `scipy.signal.lpc` to predict the current value based on the vehicle's historical values. Conversely, if the data is unstable, neighbor-based collaborative prediction is used, where the predicted value equals the vehicle's value from the previous moment plus a comprehensive compensation amount. The difference between the actual and predicted values, i.e., the residual, is calculated. The residual is then subjected to non-uniform quantization using a μ-law quantization algorithm. First, the maximum possible absolute value of the residual is determined. , residual Normalization Then substitute into the formula ,in, This represents the output value after μ-law compression mapping. This represents the normalized residual. This represents the compression factor, where μ is set to 255. The floating-point residual is mapped to an 8-bit quantization index value. The output data item's ID is used to distinguish different sensors, along with the 8-bit quantization index. For discrete data such as gear position or switch status, the raw binary value is treated as a literal, preceded by a literal identifier.

[0041] In some implementations, for continuous numerical data, based on the recent historical volatility of the data stream, a prediction model based on the vehicle's historical values ​​or a prediction model based on the average change of data from vehicles in the same time and space is selected to calculate the predicted values, including: Obtain recent historical data points for the current continuous numerical data and calculate the normalized variance; Set corresponding normalized variance thresholds for different types of data. If the calculated normalized variance is less than the normalized variance threshold, the data flow is determined to be stable, and a linear extrapolation prediction model based on the historical values ​​of this vehicle is selected to calculate the predicted value. If the calculated normalized variance is greater than or equal to the normalized variance threshold, then data stream fluctuation is determined. It is then determined whether the continuous numerical data belongs to a preset physical quantity with fleet spatial correlation. If so, a prediction model based on the mean change of data of spatiotemporally neighboring vehicles is selected to obtain the latest change of a predetermined number of neighboring vehicles with respect to the data item within the communication range. Based on the positive and negative correlation of physical quantities in the fleet spatial topology and spatial phase difference, the change of neighboring vehicles is compensated for time delay and weighted to obtain a comprehensive change. The comprehensive change is then superimposed on the latest value of the current vehicle as the predicted value. If not, a higher-order extrapolation prediction model based on the historical data of the current vehicle is selected to calculate the predicted value.

[0042] When processing continuous numerical data, retrieve the 10 most recent data points from the historical data cache. Based on this set of data, the normalized variance is calculated. This value represents the relative volatility of the data. A pre-defined parameter table sets specific normalized variance thresholds for each data type, such as engine speed, vehicle speed, and coolant temperature. For example, the threshold for engine speed is 0.02, while the threshold for coolant temperature is 0.005. If the calculated... If the value is less than the corresponding threshold, the data stream is considered stationary, and a simple linear extrapolation prediction model is used: predicted value .

[0043] like If the data stream is greater than or equal to a threshold, it is considered fluctuating. A pre-defined list of physical quantities is queried to determine if the current data type has strong spatial correlation, such as the brake pedal opening when a convoy is densely packed. If so, the spatiotemporal proximity prediction model is activated: the latest changes of the three nearest neighboring vehicles within the communication range for this data item are obtained. And based on positive and negative correlation (by correlation coefficient) (represented by) the time delay caused by spatial phase difference The weighted summation is used to obtain the overall change. ,in, This represents the spatial weight coefficient of the j-th vehicle. This represents the actual effective change of the j-th vehicle after time delay compensation. The predicted value is... , This indicates that the vehicle is in The actual measured value at any given time. If the data is not spatially correlated, such as engine oil pressure, then a second-order polynomial extrapolation model based on the vehicle's historical data is used for prediction: .

[0044] In an alternative embodiment, the residual is non-uniformly quantized before outputting the residual quantization code, including: The residual is calculated by subtracting the predicted value from the actual value; The quantization interval to which the residual belongs is determined based on the dedicated non-uniform quantization boundary preset for continuous numerical data; The residual is mapped to the preset quantization encoding value corresponding to the quantization interval and then output.

[0045] In obtaining predicted values Then, calculate the predicted residuals. ,in This is the actual measurement value at the current moment. Based on the current data type, such as vehicle speed, a dedicated non-uniform quantization table is located and loaded. This table is pre-planned based on the historical residual statistical distribution of this data type. It is a set of non-equidistant quantization boundaries, which makes the areas near which residuals occur with high probability more finely divided, while the areas near which residuals occur with low probability are divided more coarsely.

[0046] For example, for a vehicle speed residual in km / h, a non-uniform quantization boundary with 4 bits and 16 quantization levels might be set as: {...,-10,-5,-2,-1,-0.5,-0.1,0,0.1,0.5,1,2,5,10,...}. When the calculated residual r is -0.8 km / h, a binary search algorithm determines that the residual falls within the interval [-1,-0.5), which corresponds to a preset 4-bit quantization encoding value, such as 0101. The output encoding value 0101 is used as the compression result for this data point. The non-uniform quantization strategy can represent the vast majority of residuals with a very small number of bits while maintaining accuracy with a small prediction error, thus achieving a high compression ratio.

[0047] S4 encodes the generated dictionary matching items, residual quantization codes, or text quantities sequentially into uniform format data units containing type identifiers, and combines them in order into a compressed data stream.

[0048] A unified data unit format is established, with each unit starting with a 2-bit type identifier: 00 for a dictionary match, 01 for residual quantization encoding, and 10 for a textual quantity. If the type is 00, it is followed by 16 bits of dictionary matching information, including level, index, and length. If the type is 01, it is followed by the ID of the data item and an 8-bit residual index after μ-law quantization. If the type is 10, it is followed by a 4-bit length field indicating the number of bytes occupied by the original textual quantity, followed by the corresponding length of the original data bytes. The compression program repeatedly executes the above matching or non-matching processing flow, concatenating each generated data unit according to the order of the units in the original data stream to obtain the compressed data frame, until the original data stream is completely processed.

[0049] In an alternative embodiment, the generated dictionary matches, residual quantization codes, or literals are sequentially encoded into uniform format data units containing type identifiers, including: Each data unit is assigned a type identifier prefix to distinguish its data type; For dictionary matches, the data unit format is a type identifier, a hierarchy identifier, a dictionary index field, and a match length field; For residual quantization coding, the data unit format is a type identifier, a data item ID, and a residual quantization coding value; For text, the data unit is formatted as a type identifier, a length field indicating the length of the original data, and one or more bytes of the original data.

[0050] All compressed data is organized into a series of consecutive data units, each starting with a 2-bit type identifier prefix for the decoder to recognize the content. This identifier is defined as follows: 00 represents a dictionary match, 01 represents residual quantization encoding, 10 represents a single-byte literal, and 11 represents a multi-byte literal.

[0051] For dictionary match type 00, the data structure is: a 2-bit hierarchical identifier, where 01 is global, 10 is semantic, and 11 is spatiotemporal; a variable-length dictionary index, such as a global dictionary with 2048 entries, requires 11 bits; and an 8-bit match length field can represent lengths from 3 to 258. A complete example is: 00101001011000001001, representing a match from the fleet semantic dictionary with an index of 150 and a length of 12 bytes. For residual quantization encoding type 01, it is followed by a data item ID, for example, 4 bits, and then the fixed-length quantization encoding value generated above, for example, a 4-bit encoding: 0101. For unmatched literals, if it is a single byte, use type 10 followed by 8 bits of raw byte data, for example, 1001000001 represents the character A; if it is multiple consecutive unmatched bytes, use type 11 followed by an 8-bit length field indicating the number of bytes of subsequent raw data, ranging from 1 to 255, followed by a raw data byte stream of the corresponding length, for example, 1100000101[48454C4C4F] represents the string HELLO.

[0052] A 10GB mixed dataset was used, consisting of data from 20 vehicles continuously driving for one week. CAN bus messages accounted for 55%, V2X interaction messages for 30%, and other sensor data for 15%. The test platform was a server equipped with an Intel Xeon E5 processor and 128GB of memory. The general compression algorithm Gzip was used as the benchmark control group, and the complete present invention was used as the experimental group. The average compression ratio of the Gzip algorithm was 3.52:1, with a compression throughput of 35.4MB / s and a decompression throughput of 180.1MB / s. The present invention achieved an average compression ratio of 6.81:1, a compression throughput of 28.6MB / s, and a decompression throughput of 215.5MB / s. Compared to Gzip, the compression ratio of the present invention is improved by 93.5%, and the decompression speed is improved by 19.7%, demonstrating the effectiveness of optimizing the characteristics of vehicle network data.

[0053] With other modules remaining unchanged, removing the spatiotemporal proximity dictionary, and relying solely on the fleet semantic dictionary and the global basic dictionary, reduces the average compression ratio to 5.23:1. Removing the fleet semantic dictionary, and retaining only the spatiotemporal proximity dictionary and the global basic dictionary, further reduces the average compression ratio to 5.95:1. The spatiotemporal proximity dictionary contributes the most to the compression ratio, with a 23.2% performance decrease after its removal. The removal of the fleet semantic dictionary resulted in a 12.6% performance decrease, validating the value of periodically mining common communication patterns within fleets.

[0054] The special processing module for continuous numerical data was ablated. In this experiment, the prediction model selection based on variance judgment, residual calculation, and non-uniform quantization processes were eliminated. All continuous numerical data was treated as ordinary byte streams and processed by the dictionary matching module. The average compression ratio under this configuration was reduced to 4.48:1, a decrease of 34.2% compared to the complete solution. Numerical data in CAN bus and sensor data has strong temporal continuity and local stationarity. Traditional dictionary matching cannot compress small variations, while predictive coding combined with non-uniform quantization can represent the vast majority of data points at a low bit cost, thereby improving the compression efficiency of this type of data. This demonstrates that this module makes a decisive contribution to the overall performance.

[0055] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent compression of collaborative communication data in commercial vehicle fleets, characterized in that, Includes the following steps: A three-layer heterogeneous data dictionary is constructed and maintained, which includes a pre-built global basic dictionary containing a general high-frequency dataset, a fleet semantic dictionary generated periodically by extracting high-frequency data sequences from historical fleet communication data using a time-series correlation and information entropy weighted model, and a spatiotemporal proximity dictionary that aggregates confirmed successful communication data segments between the vehicle and neighboring nodes based on the vehicle's real-time location and communication topology and updates them using a first-in-first-out (FIFO) strategy. The spatiotemporal proximity dictionary is a fixed-size double-ended queue data structure. When the vehicle successfully receives and verifies a neighboring vehicle broadcast data packet with a correct CRC through C-V2X or DSRC communication, the data payload of the data packet is pushed into the queue as an entry from the right side. If the queue is full, the oldest entry is popped from the left side to achieve FIFO updates. A fleet semantic dictionary, generated periodically by applying a time-series correlation and information entropy weighted model to extract high-frequency data sequences from historical fleet communication data, includes: Historical communication data of vehicles within the fleet is periodically collected as input dataset; By applying a frequent sequence mining algorithm and setting a minimum support threshold, frequent data sequences exceeding a predetermined length can be extracted from the input dataset. For each frequently mined data sequence, the normalized support and normalized information entropy of the frequently mined data sequence are comprehensively evaluated, and a comprehensive score is calculated. Select the most frequent data sequences with the highest overall scores and update the fleet semantic dictionary as entries; For compressed data streams, a parallel search is performed on all matching terms in a three-layer heterogeneous data dictionary. A directed graph with data stream positions as nodes is constructed based on the encoding cost function. The optimal matching term is determined and output through the minimum cost path, including: Starting from the current position in the data stream to be compressed, search in parallel for all possible matching strings in the three-layer heterogeneous data dictionary; For each found match, calculate the matching item encoding cost based on the cost function; Construct a directed acyclic graph with data stream positions as nodes and possible matching items as weighted edges, where the weight of the edge is the encoding cost of the corresponding matching item; The shortest path algorithm is applied to determine the minimum cost path from the current node to the end node of the data stream, and the first match of the minimum cost path is selected as the optimal match output for the current position. If no valid match is found in any dictionary, for continuous numerical data, based on the recent historical volatility of the data stream, a prediction model based on the vehicle's historical values ​​or a prediction model based on the average change of data from nearby vehicles in time and space is selected. The residual between the predicted and actual values ​​is calculated, and the residual is output as a quantized code after non-uniform quantization. For discrete state variables and non-numerical data, they are output as text values. The calculation of the predicted value includes: Obtain recent historical data points for the current continuous numerical data and calculate the normalized variance; Set corresponding normalized variance thresholds for different types of data. If the calculated normalized variance is less than the normalized variance threshold, the data flow is determined to be stable, and a linear extrapolation prediction model based on the historical values ​​of this vehicle is selected to calculate the predicted value. If the calculated normalized variance is greater than or equal to the normalized variance threshold, then data stream fluctuation is determined. It is then determined whether the continuous numerical data belongs to a preset physical quantity with fleet spatial correlation. If so, a prediction model based on the mean change of data of spatiotemporally adjacent vehicles is selected to obtain the latest change of a predetermined number of adjacent vehicles with respect to the data item within the communication range. Based on the positive and negative correlation of the physical quantity in the fleet spatial topology and the spatial phase difference, the change of adjacent vehicles is compensated for time delay and weighted to obtain a comprehensive change. The comprehensive change is then superimposed on the latest value of the current vehicle as the predicted value. If not, a higher-order extrapolation prediction model based on the historical data of the current vehicle is selected to calculate the predicted value. The generated dictionary matches, residual quantization codes, or text values ​​are sequentially encoded into uniform format data units containing type identifiers and then combined in order into a compressed data stream.

2. The intelligent compression method for cooperative communication data of commercial vehicle fleets according to claim 1, characterized in that, The spatiotemporal proximity dictionary, updated using a first-in-first-out (FIFO) strategy, aggregates communication data segments that have been confirmed as successful interactions between the vehicle and neighboring nodes based on the vehicle's real-time location and communication topology. This includes: The vehicle periodically broadcasts a heartbeat packet containing its own location information; Other vehicles within a predetermined communication range are identified as neighboring nodes; Allocate a pre-defined buffer for the spatiotemporal proximity dictionary, managed using a first-in-first-out (FIFO) strategy; Extract and aggregate the communication data segments that have been confirmed to have been successfully interacted between this vehicle and neighboring nodes, and store the communication data segments and source identifiers as an entry in the cache area; When the buffer is full, the insertion of a new entry will cause the oldest entry to be removed.

3. The intelligent compression method for cooperative communication data of commercial vehicle fleets according to claim 1, characterized in that, After performing non-uniform quantization on the residuals, the output residual quantization code is included, including: The residual is calculated by subtracting the predicted value from the actual value; The quantization interval to which the residual belongs is determined based on the dedicated non-uniform quantization boundary preset for continuous numerical data; The residual is mapped to the preset quantization encoding value corresponding to the quantization interval and then output.

4. The intelligent compression method for cooperative communication data of commercial vehicle fleets according to claim 1, characterized in that, The generated dictionary matches, residual quantization codes, or literals are sequentially encoded into uniform format data units containing type identifiers, including: Each data unit is assigned a type identifier prefix to distinguish its data type; For dictionary matches, the data unit format is a type identifier, a hierarchy identifier, a dictionary index field, and a match length field; For residual quantization coding, the data unit format is a type identifier, a data item ID, and a residual quantization coding value; For text data, the data unit is formatted as a type identifier, a length field indicating the length of the original data, and one or more bytes of the original data.

5. The intelligent compression method for cooperative communication data of commercial vehicle fleets according to claim 1, characterized in that, The global basic dictionary is constructed by offline analysis of historical CAN bus data and SAEJ1939 standard messages of commercial vehicles. Fixed and high-frequency communication identifiers, control commands, and standard fault codes are extracted using the word frequency statistics TF-IDF algorithm and stored in a hash table structure.

6. The intelligent compression method for cooperative communication data of commercial vehicle fleets according to claim 1, characterized in that, The Aho-Corasick algorithm is used to search all matching items in a three-layer heterogeneous data dictionary in parallel.

Images