Internet of vehicles high-density data compression method based on last data

Abstract

The invention provides an Internet of Vehicles high-density data compression method based on last data, and the method comprises the following steps: S1, collecting various data of a vehicle in real time through a terminal according to a specific collection task, and storing the data in a queue; s2, the terminal periodically acquires the queue data according to the reporting period, compresses the queue data and reports the compressed queue data to the cloud; s3, the cloud receives the data reported by the terminal and decompresses the data; according to the Internet of Vehicles high-density data compression method based on the last-time data, the last-time data hash table is maintained, the last-time parameter value of the data is recorded, if the next piece of data is consistent with the data in the hash table, the next piece of data can be directly deleted, and the cloud recovers the data through the same method; the compression efficiency is improved, extra computing resources are not needed for compression and decompression processing, and zero delay is basically achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a vehicle network high-density data compression method based on last data. Background Art

[0002] With the continuous development of science and technology, the Internet of Vehicles has become a hot topic in the field of intelligent transportation at this stage. By connecting vehicles to the Internet, the Internet of Vehicles has built an intelligent interconnected platform that integrates resources such as vehicles and roads. Among them, data collection is one of the most important links in the Internet of Vehicles, which directly affects the efficiency of traffic management and service quality. The vehicle data collection terminal (T-BOX) collects various vehicle data according to different collection schemes, and then synchronizes the data to the cloud platform. When it comes to vehicle fault diagnosis, such as insufficient vehicle power, high-density data needs to be collected once a second and periodically packaged and sent to the cloud platform. At this time, the amount of data to be collected is too large, which will consume a lot of terminal traffic and platform bandwidth resources, so the data needs to be compressed; The current lossless compression method is file compression, which uses the statistical redundancy of data for compression. The compression efficiency is usually between 2:1 and 5:1, that is, the data is compressed to 1 / 2 to 1 / 5 of the original data without losing any data. Although this compression method has high compression efficiency, terminal compression requires a lot of CPU resources and there is an upload delay problem. Decompressing all terminal data in the cloud requires a separate server, which consumes a lot of resources. Summary of the invention

[0003] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a high-density data compression method for the Internet of Vehicles based on last data.

[0004] To achieve the above object, the present invention adopts the following technical solutions: A high-density data compression method for an Internet of Vehicles based on last data comprises the following steps: S1: The terminal collects various vehicle data in real time according to specific collection tasks and stores them in the queue; The terminal collects various vehicle data in real time based on vehicle-mounted sensors, cameras, GPS and other equipment according to specific collection tasks, and stores the collected vehicle data in a queue; The data include position, speed, acceleration, fuel volume, etc. Each item of data includes parameter name, parameter value, number of parameters, and queue length; The parameter item name refers to the identifier of each data, such as position, speed, acceleration, etc.; The value of the parameter item refers to the specific numerical value corresponding to the parameter item name. For example, if the parameter item name is speed, the value of the parameter item is 60, and the unit is 60 km / h; The number of parameters refers to the number of parameters contained in each data item, and the number of parameters is equal to the data dimension. For example, the position of a vehicle on a plane is two-dimensional data, including two parameters of longitude and latitude; acceleration is three-dimensional data, including acceleration in the three directions of X, Y, and Z, i.e., three parameters; The queue length refers to the amount of data stored in the data queue; Specifically, the data is stored in the queue as follows: ; Where P is the parameter name, V is the value of the parameter, n is the number of parameters, and i is the queue length; S2: The terminal periodically obtains queue data according to the reporting cycle, compresses it, and reports it to the cloud; The terminal obtains data from the queue regularly according to the actual specified reporting cycle, compresses the obtained data, and reports it to the cloud; The specific compression method includes the following sub-steps: S21: The compression module maintains a last data hash table; The compression module is a system component used for data optimization; the last data hash table is used to record the latest value of each data; The last data hash table stores key-value pairs through a hash table, wherein the key is the parameter item name and the value is the parameter item value; Define a hash function H, H(P)=V, where P is the parameter item name and V is the parameter item value; the hash function H performs a hash operation on the obtained parameter item name using the SHA-256 algorithm, outputs the hash value as the key in the last data hash table, and stores the value corresponding to the key, i.e., the parameter item value, in the last data hash table; S22: traverse each row of data in the queue; The compression module traverses all the data in the queue and reads each line of data in the queue one by one; each line of data contains multiple data; S23: traverse the parameter items in each row of data and compare; The compression module checks the value of the parameter item corresponding to each row of data in the queue one by one, and compares the value of the parameter item with the value of the corresponding parameter item in the last data hash table. If the two are the same, it means that the data has not changed, and the data is deleted from the queue; if the two are different, it means that the data has changed, and the value of the parameter item corresponding to the data in the last data hash table is replaced with the value of the current parameter item in the queue; S24: Repeat the traversal until the queue data compression is completed; Repeat steps S22 and S23 to continue traversing all the data in the queue until all the data has been processed and compressed; After compression is completed, all data is synchronously stored in the last data hash table; The terminal reports the data in the queue to the cloud in the form of data packets.

[0005] S3: The cloud receives the data reported by the terminal and decompresses it; The cloud receives the data packets reported by the terminal and decompresses the received data packets; Specifically, the decompression module maintains the last data hash table; the decompression module is a system component for data optimization; The decompression module traverses each line of data in the received data packet; wherein each line of data contains multiple items of data; The decompression module traverses each data item in turn, and compares the value of the parameter item corresponding to the data with the record in the last data hash table, checks whether the value of this parameter item exists in the last data hash table, and if so, updates the value of this parameter item as the latest value to the last data hash table; if not, obtains the value of the parameter item of this data from the last data hash table and fills it into the data packet; Until all the data is decompressed.

[0006] Compared with the prior art, the present invention has the following beneficial effects: The high-density data compression method for the Internet of Vehicles based on the last data proposed in the present invention maintains a last data hash table to record the last parameter value of the data. If the next data is consistent with the data in the hash table, it can be directly deleted, and the cloud can restore the data by the same method; the compression efficiency is improved, no additional computing resources are required for compression and decompression processing, and basically zero delay is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 The present invention is a flowchart of the steps of a method for high-density data compression of an Internet of Vehicles based on last data. DETAILED DESCRIPTION

[0008] In order to provide a further understanding of the purpose, structure, features, and functions of the present invention, the following detailed description is given in conjunction with the embodiments.

[0009] like Figure 1 As shown, a high-density data compression method for an Internet of Vehicles based on last data includes the following steps: S1: The terminal collects various vehicle data in real time according to specific collection tasks and stores them in the queue; The terminal collects various vehicle data in real time based on vehicle-mounted sensors, cameras, GPS and other equipment according to specific collection tasks, and stores the collected vehicle data in a queue; The data include position, speed, acceleration, fuel volume, etc. Each item of data includes parameter name, parameter value, number of parameters, and queue length; The parameter item name refers to the identifier of each data, such as position, speed, acceleration, etc.; The value of the parameter item refers to the specific numerical value corresponding to the parameter item name. For example, if the parameter item name is speed, the value of the parameter item is 60, and the unit is 60 km / h; The number of parameters refers to the number of parameters contained in each data item, and the number of parameters is equal to the data dimension. For example, the position of a vehicle on a plane is two-dimensional data, including two parameters of longitude and latitude; acceleration is three-dimensional data, including acceleration in the three directions of X, Y, and Z, i.e., three parameters; The queue length refers to the amount of data stored in the data queue; Specifically, the data is stored in the queue as follows: ; Where P is the parameter name, V is the value of the parameter, n is the number of parameters, and i is the queue length; By clearly controlling the number of parameters and queue length, data redundancy can be avoided and the system can only process the necessary amount of data. Appropriate queue length management can avoid the waste of memory or storage resources and improve system performance. Parameters of different dimensions (such as longitude and latitude of location, XYZ direction of acceleration, etc.) are emphasized, which enables the data to more comprehensively express the status of the vehicle and supports multi-dimensional data analysis, which is very important for high-precision behavior analysis, anomaly detection, and the development of autonomous driving algorithms. By storing data in a queue, the system can upload batches of data or transmit asynchronously on demand, optimizing communication bandwidth and transmission latency. For large-scale fleet management systems or cloud data analysis platforms, it can reduce network load while ensuring real-time performance.

[0010] S2: The terminal periodically obtains queue data according to the reporting cycle, compresses it, and reports it to the cloud; The terminal obtains data from the queue regularly according to the actual specified reporting cycle, compresses the obtained data, and reports it to the cloud; The specific compression method includes the following sub-steps: S21: The compression module maintains a last data hash table; The compression module is a system component used for data optimization; the last data hash table is used to record the latest value of each data; The last data hash table stores key-value pairs through a hash table, wherein the key is the parameter item name and the value is the parameter item value; Define a hash function H, H(P)=V, where P is the parameter item name and V is the parameter item value; the hash function H performs a hash operation on the obtained parameter item name using the SHA-256 algorithm, outputs the hash value as the key in the last data hash table, and stores the value corresponding to the key, i.e., the parameter item value, in the last data hash table; By maintaining the last data hash table to track the historical value of each data item, each calculation can be compared with the previous value to ensure data integrity or process certain data changes.

[0011] S22: traverse each row of data in the queue; The compression module traverses all the data in the queue and reads each line of data in the queue one by one; each line of data contains multiple data; S23: traverse the parameter items in each row of data and compare; The compression module checks the value of the parameter item corresponding to each row of data in the queue one by one, and compares the value of the parameter item with the value of the corresponding parameter item in the last data hash table. If the two are the same, it means that the data has not changed, and the data is deleted from the queue; if the two are different, it means that the data has changed, and the value of the parameter item corresponding to the data in the last data hash table is replaced with the value of the current parameter item in the queue; S24: Repeat the traversal until the queue data compression is completed; Repeat steps S22 and S23 to continue traversing all the data in the queue until all the data has been processed and compressed; After compression is completed, all data is synchronously stored in the last data hash table; The terminal reports the data in the queue to the cloud in the form of data packets.

[0012] By comparing the current value of each data item with the last recorded value, only the changed data is uploaded; if the value of a parameter has not changed, the data item is directly deleted to avoid repeated data transmission; this significantly reduces the amount of data reported each time, especially when most data items have not changed; for sensors or devices that generate a large amount of similar data, this compression method can significantly reduce network bandwidth consumption, especially in bandwidth-limited or cost-sensitive scenarios; By uploading only the changed data, the system can reduce the size of each data upload, thus saving the time required for uploading; S3: The cloud receives the data reported by the terminal and decompresses it; The cloud receives the data packets reported by the terminal and decompresses the received data packets; Specifically, the decompression module maintains the last data hash table; the decompression module is a system component for data optimization; The decompression module traverses each line of data in the received data packet; wherein each line of data contains multiple items of data; The decompression module traverses each data item in turn, and compares the value of the parameter item corresponding to the data with the record in the last data hash table, checks whether the value of this parameter item exists in the last data hash table, and if so, updates the value of this parameter item as the latest value to the last data hash table; if not, obtains the value of the parameter item of this data from the last data hash table and fills it into the data packet; Until all the data is decompressed.

[0013] By comparing the received data with the records in the last data hash table, it is possible to detect which data items already exist in the system. If the data items already exist, it is possible to avoid storing the same data repeatedly, which not only saves storage space but also improves the efficiency of data processing. By updating the value of the parameter item to the last data hash table, it is ensured that each data item reflects the latest status; even when processing multiple data packets, the cloud always maintains data consistency and avoids interference from old data.

[0014] The present invention has been described by the above-mentioned relevant embodiments, however, the above-mentioned embodiments are only examples for implementing the present invention. It must be pointed out that the disclosed embodiments do not limit the scope of the present invention. On the contrary, changes and modifications made without departing from the spirit and scope of the present invention are all within the scope of patent protection of the present invention.

Claims

1. A high-density data compression method for Internet of Vehicles based on last data, characterized in that: The following steps are involved: S1: The terminal collects various vehicle data in real time according to specific collection tasks and stores them in the queue; Each data item includes parameter name, parameter value, number of parameters, and queue length; The parameter item name refers to the identifier of each data item; The value of the parameter item refers to the specific numerical value corresponding to the parameter item name; The number of parameters refers to the number of parameters contained in each data item, and the number of parameters is equal to the data dimension; The queue length refers to the amount of data stored in the data queue; S2: The terminal periodically obtains queue data according to the reporting cycle, compresses it, and reports it to the cloud; The terminal periodically obtains data from the queue according to the actual specified reporting period, compresses the obtained data, and reports it to the cloud; the specific compression method includes the following sub-steps: S21: The compression module maintains a last data hash table; The compression module is a system component used for data optimization; S22: traverse each row of data in the queue; The compression module traverses all the data in the queue and reads each line of data in the queue one by one; each line of data contains multiple data; S23: traverse the parameter items in each row of data and compare; S24: Repeat the traversal until the queue data compression is completed; Repeat steps S22 and S23 to continue traversing all the data in the queue until all the data has been processed and compressed; After compression is completed, all data is synchronously stored in the last data hash table; The terminal reports the data in the queue to the cloud in the form of data packets; S3: The cloud receives the data reported by the terminal and decompresses it.

2. The vehicle network high-density data compression method based on last data as claimed in claim 1, characterized in that: In step S1, the data is stored in the queue in the following form: ； Among them, P is the parameter item name, V is the value of the parameter item, n is the number of parameters, and i is the queue length.

3. The vehicle network high-density data compression method based on last data as claimed in claim 1, characterized in that: In step S21, the last data hash table is used to record the latest value of each data; The last data hash table stores key-value pairs through a hash table, wherein the key is the parameter item name and the value is the parameter item value; Define a hash function H, H(P)=V, where P is the parameter item name and V is the parameter item value; the hash function H performs a hash operation on the obtained parameter item name through the SHA-256 algorithm, and the output hash value is used as the key in the final data hash table, and the value corresponding to the key, that is, the parameter item value, is stored in the final data hash table.

4. The vehicle network high-density data compression method based on last data according to claim 1, characterized in that: In step S23, the compression module checks the value of the parameter item corresponding to each row of data in the queue one by one, and compares the value of the parameter item with the value of the corresponding parameter item in the last data hash table. If the two are the same, it means that the data has not changed, and the data is deleted from the queue; if the two are different, it means that the data has changed, and the value of the parameter item corresponding to the data in the last data hash table is replaced by the value of the current parameter item in the queue.

5. The vehicle network high-density data compression method based on last data as claimed in claim 1, characterized in that: In step S3, the cloud receives the data packet reported by the terminal and decompresses the received data packet; Specifically, the decompression module maintains the last data hash table; the decompression module is a system component for data optimization; The decompression module traverses each line of data in the received data packet; wherein each line of data contains multiple items of data; The decompression module traverses each data item in turn, and compares the value of the parameter item corresponding to the data with the record in the last data hash table, checks whether the value of this parameter item exists in the last data hash table, and if so, updates the value of this parameter item as the latest value to the last data hash table; if not, obtains the value of the parameter item of this data from the last data hash table and fills it into the data packet; Until all the data is decompressed.

Images