Data storage and transmission method of high-precision floating-point type time series data

CN116089385BActive Publication Date: 2026-09-29CHONGQING UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211519183.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-30
Publication Date
2026-09-29
Estimated Expiration
2042-11-30

AI Technical Summary

Technical Problem

然而,当浮点数精度较高的时(如GPS经纬度数据),存储不同数据的连续区块较大,所以现有标准的异或压缩算法的压缩率不高

Benefits of technology

[0034]本发明通过高精度浮点类型时序数据的时序数据相差较小以及在同个时间序列中大部分浮点数的占用的有效二进制位是类似的,并且往往只有中间的一个连续区块存储着不同数据的特点,将时序数据与其相邻时序数据进行异或计算得到异或值,并去除异或值的前导零生成有效位并确定有效位位数,使得能够通过存储有效位及其有效位位数来实现时序数据的存储,进而能够有效实现高精度浮点类型时序数据的无损压缩,并提高时序数据的压缩率;同时,本发明直接从最低位开始存储时序数据的异或值,并且只省略存储了前导零,而舍弃了对尾随零的处理(异或后数据尾随零数量较少,但存储尾随零的个数需要6位,而异或后数据尾随零个数极大概率不到6位),使得能够减少时序数据的存储位数,在解压时也无需额外对尾随零进行操作,即无需进行额外的优化设计,能够降低时序数据的压解时间,从而能够保证浮点类型时序数据的压缩效果和压解成本,进而有效降低数据存储量和传输成本。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116089385B_ABST
    Figure CN116089385B_ABST
Patent Text Reader

Abstract

The application relates to the technical field of data lossless compression, in particular to a lossless compression and decompression method for high-precision floating-point type time series data, which comprises the following steps: sorting the time series data to be compressed; performing exclusive or calculation on each piece of time series data and its adjacent time series data in sequence to obtain corresponding exclusive or values; removing the leading zeros in the exclusive or values of each piece of time series data to generate corresponding valid bits; determining the valid bit number of each piece of time series data according to the valid bits of the time series data; and storing the valid bits and the valid bit number of the corresponding time series data when the time series data is compressed. The lossless compression method can realize lossless compression of high-precision floating-point type time series data, effectively improve the compression rate and reduce the compression and decompression time, thereby guaranteeing the compression effect and compression and decompression cost of the floating-point type time series data, and effectively reducing the data storage amount and transmission cost.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of lossless data compression technology, specifically to a method for storing and transmitting high-precision floating-point time-series data. Background Technology

[0002] With technological advancements and the widespread adoption of mobile devices, massive amounts of time-series data are constantly being generated, such as vehicle trajectories, mobile phone signaling data, and information collected by sensors. The diversity and exponential growth of this data have placed enormous pressure on data storage and transmission, severely hindering the application and development of high-performance computing in scientific fields. Data compression has long been a hot topic in addressing these problems; finding efficient data compression techniques can effectively reduce data storage volume and transmission costs.

[0003] Data compression can be categorized into lossy compression and lossless compression based on whether the decoded data can completely restore the original data. Lossy compression refers to the inability to accurately recover the original data during decompression, resulting in some information loss. This method is mainly used in fields such as images, videos, and audio where a small amount of information loss is acceptable without affecting the original data quality. Lossless compression, also known as entropy coding or lossless coding, works by reducing or removing redundancy in the data, while ensuring that the original data can be accurately recovered. Time-series data, used in applications such as heatmap visualization and personnel tracking, cannot easily suffer information loss, thus requiring lossless compression. There are various time-series data types, such as integer data, Boolean data, and floating-point data, with floating-point time-series data being one of the more difficult types to compress.

[0004] In 2015, Facebook proposed a floating-point compression algorithm based on the XOR algorithm for compressing floating-point time-series data. Researchers discovered that in the same time series, most floating-point numbers occupy similar effective binary bits, and generally only one contiguous block stores different data. Therefore, existing techniques use XOR calculations to generate a large number of 0s for the same data, thus omitting the storage of these leading and trailing zeros to achieve compression. However, when floating-point precision is high (such as GPS latitude and longitude data), the contiguous blocks storing different data are large, so the compression ratio of existing standard XOR compression algorithms is not high. Furthermore, existing decompression algorithms require concatenating leading and trailing zeros and effective bits during decompression, which consumes a significant amount of time or requires additional optimization design, resulting in high data compression costs. Therefore, designing a method that can achieve lossless compression of high-precision floating-point time-series data, effectively improve the compression ratio, and reduce decompression time is an urgent technical problem to be solved. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is: how to provide a data storage and transmission method for high-precision floating-point time-series data, which can achieve lossless compression of taxi GPS trajectory point data, effectively improve the compression rate of taxi GPS trajectory point data and reduce compression and decompression time, thereby improving the compression effect and decompression cost of taxi trajectory data, and thus effectively reducing the storage volume and transmission cost of taxi trajectory data.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] Methods for storing and transmitting high-precision floating-point time-series data include:

[0008] S1: Sort the time-series data to be compressed;

[0009] S2: Perform an XOR operation on each time series data with its adjacent time series data in sequence to obtain the corresponding XOR value;

[0010] S3: Remove leading zeros from the XOR values ​​of each time series data and generate the corresponding valid bits;

[0011] S4: Determine the number of significant bits based on the significant bits of each time series data;

[0012] S5: When compressing time-series data, store the valid bits of the corresponding time-series data and the number of valid bits.

[0013] Preferably, in step S2, starting from the second-ranked time series data, each time series data is XORed with its adjacent preceding time series data in sequence to obtain the XOR value corresponding to each time series data.

[0014] Preferably, in step S3, the XOR value of each time sequence data is first converted into binary form, and then leading zeros are removed to generate valid bits in binary form.

[0015] Preferably, leading zeros refer to all the zeros before the first significant digit in the XOR value, which in binary is represented by all the zeros before the highest bit with a value of 1.

[0016] Preferably, in step S4, the number of valid bits is first determined based on the valid bit length of the second-ranked time series data; then, starting from the third-ranked time series data, the valid bit length of each time series data is compared with the valid bit length of the adjacent preceding time series data in sequence to determine the number of valid bits corresponding to each time series data.

[0017] Preferably, the number of significant bits in time-series data is determined according to the following rules:

[0018] 1) If the effective bit length of the current time series data is greater than the effective bit length of its adjacent previous time series data, then the number of effective bits is determined according to the effective bit length of the current time series data;

[0019] 2) If the effective bit length of the current time series data is less than or equal to the effective bit length of its adjacent previous time series data, then the effective bit length of the current time series data is consistent with the effective bit length of its adjacent previous time series data, and the insufficient effective bit length is padded with 0.

[0020] Preferably, a flag bit with two control bits is generated for each time series data according to the following rules:

[0021] 1) For time-series data with an XOR value of 0, the flag bit is 0;

[0022] 2) If the XOR value of the corresponding timing data is not 0, then the first control bit of its flag is 1;

[0023] 3) If the effective bit length of the timing data is less than or equal to the effective bit length of the adjacent preceding timing data and the length difference is less than 6 bits, then the second control bit of its flag bit is 0;

[0024] 4) If the effective bit length of the timing data is greater than the effective bit length of the adjacent preceding timing data, then the second control bit of its flag bit is 1.

[0025] Preferably, in step S5, when compressing timing data, the timing data in the first position is stored first, and then the flag bits, valid bits and the number of valid bits of other timing data are stored in sequence to generate the corresponding timing data compressed package.

[0026] Preferably, when compressing time-series data, time-series data with an XOR value of 0 is stored as a 1-bit flag bit 0.

[0027] This invention also discloses a method for decompressing high-precision floating-point time-series data, which is implemented based on the lossless compression method for high-precision floating-point time-series data in this invention, specifically including:

[0028] S01: Obtain the compressed time series data package to be decompressed;

[0029] S02: Obtain the first-ranked time series data;

[0030] S03: For the second-order timing data: First, pad the leading zeros of the corresponding timing data's valid bits according to the valid bit length to obtain the binary XOR value; then, convert the corresponding timing data's binary XOR value into the format corresponding to the first-order timing data, i.e., the original format, to obtain the corresponding XOR value; finally, perform an XOR calculation based on the corresponding timing data's XOR value and the first-order timing data to obtain the second-order timing data;

[0031] S04: Starting from the third timing data: First, determine the number of significant bits of the corresponding timing data based on the flag bit of the corresponding timing data and the number of significant bits of the adjacent preceding timing data; then, pad the significant bits with leading zeros to obtain the binary XOR value; subsequently, convert the binary XOR value of the corresponding timing data back to its original format to obtain the corresponding XOR value; finally, perform an XOR calculation based on the XOR value of the corresponding timing data and the timing data of the adjacent preceding timing data to obtain the corresponding timing data.

[0032] S05: Repeat step S04 until all time series data is decompressed.

[0033] The lossless compression method for high-precision floating-point time-series data in this invention has the following beneficial effects:

[0034] This invention leverages the characteristics of high-precision floating-point time-series data, such as the small differences between time-series data, the similarity in the effective binary bits occupied by most floating-point numbers within the same time series, and the fact that often only a single contiguous block stores different data. By performing an XOR operation on the time-series data with its adjacent time-series data, removing leading zeros from the XOR value to generate significant bits, and determining the number of significant bits, this invention enables the storage of time-series data by storing the significant bits and their number. This allows for effective lossless compression of high-precision floating-point time-series data and improves the compression ratio. In this invention, the XOR value of the timing data is stored directly from the least significant bit, and only the leading zeros are omitted, while the processing of trailing zeros is discarded (the number of trailing zeros in the XOR data is small, but storing the number of trailing zeros requires 6 bits, while the number of trailing zeros in the XOR data is very likely less than 6 bits). This reduces the number of bits required to store the timing data, and no additional operation on trailing zeros is needed during decompression, i.e. no additional optimization design is required. This reduces the compression and decompression time of the timing data, thereby ensuring the compression effect and decompression cost of floating-point timing data, and thus effectively reducing the amount of data stored and the transmission cost. Attached Figure Description

[0035] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0036] Figure 1 This is a logic block diagram of a lossless compression method.

[0037] Figure 2 An example diagram of a lossless compression method;

[0038] Figure 3 This is a diagram illustrating the comparison of compression ratios;

[0039] Figure 4 This is a diagram illustrating the compression time comparison.

[0040] Figure 5 This is a diagram showing the comparison of decompression times. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0042] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the figures, or the orientation or positional relationship commonly used when the product is in use. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance. In addition, the terms "horizontal," "vertical," etc., do not indicate that the component is required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0043] The following detailed explanation illustrates the specific implementation methods:

[0044] Example 1:

[0045] This embodiment discloses a lossless compression method for high-precision floating-point time-series data.

[0046] like Figure 1 and Figure 2 As shown, a lossless compression method for high-precision floating-point time-series data includes:

[0047] S1: Sort the time-series data to be compressed;

[0048] S2: Perform an XOR operation on each time series data with its adjacent time series data in sequence to obtain the corresponding XOR value;

[0049] In this embodiment, starting from the second-ranked time series data, each time series data is XORed with its adjacent preceding time series data in sequence to obtain the XOR value corresponding to each time series data.

[0050] S3: Remove leading zeros from the XOR values ​​of each time series data and generate the corresponding valid bits;

[0051] In this embodiment, the XOR value of each time sequence data is first converted into binary form, and then leading zeros are removed to generate valid bits in binary form.

[0052] Leading zeros are all zeros in an XOR value before the first significant digit, which in binary is represented by all zeros before the most significant bit that has a value of 1; trailing zeros are all zeros in an XOR value after the last significant digit, which in binary is represented by all zeros after the least significant bit that has a value of 1.

[0053] S4: Determine the number of significant bits based on the significant bits of each time series data;

[0054] S5: When compressing time-series data, store the valid bits of the corresponding time-series data and the number of valid bits.

[0055] In this embodiment, when compressing timing data, the timing data in the first position is stored first, and then the flag bits, valid bits, and the number of valid bits of other timing data are stored sequentially to generate the corresponding timing data compressed package. For timing data with an XOR value of 0, it is stored as a 1-bit flag bit 0.

[0056] It should be noted that the lossless compression method of this invention is not only applicable to floating-point time-series data, but also suitable for float-type time-series data, although the compression ratio improvement is only slight compared to floating-point time-series data. Furthermore, the lossless compression method of this invention can also be used for lower-precision floating-point time-series data without requiring additional read / write optimizations.

[0057] This invention leverages the characteristics of high-precision floating-point time-series data, such as the small differences between time-series data, the similarity in the effective binary bits occupied by most floating-point numbers within the same time series, and the fact that often only a single contiguous block stores different data. By performing an XOR operation on the time-series data with its adjacent time-series data, removing leading zeros from the XOR value to generate significant bits, and determining the number of significant bits, this invention enables the storage of time-series data by storing the significant bits and their number. This allows for effective lossless compression of high-precision floating-point time-series data and improves the compression ratio. In this invention, the XOR value of the timing data is stored directly from the least significant bit, and only the leading zeros are omitted, while the processing of trailing zeros is discarded (the number of trailing zeros in the XOR data is small, but storing the number of trailing zeros requires 6 bits, while the number of trailing zeros in the XOR data is very likely less than 6 bits). This reduces the number of bits required to store the timing data, and no additional operation on trailing zeros is needed during decompression, i.e. no additional optimization design is required. This reduces the compression and decompression time of the timing data, thereby ensuring the compression effect and decompression cost of floating-point timing data, and thus effectively reducing the amount of data stored and the transmission cost.

[0058] In the specific implementation process, the number of valid bits is first determined based on the valid bit length of the second-ranked time series data; then, starting from the third-ranked time series data, the valid bit length of each time series data is compared with the valid bit length of the adjacent preceding time series data in sequence to determine the number of valid bits corresponding to each time series data.

[0059] The number of significant bits in time-series data is determined using the following rules:

[0060] 1) If the effective bit length of the current time series data is greater than the effective bit length of its adjacent previous time series data, then the number of effective bits is determined according to the effective bit length of the current time series data;

[0061] 2) If the effective bit length of the current time series data is less than or equal to the effective bit length of its adjacent previous time series data, then the effective bit length of the current time series data is consistent with the effective bit length of its adjacent previous time series data, and the insufficient effective bit length is padded with 0.

[0062] This invention determines the number of valid bits in a time-series data by comparing the valid bit length of the data with the valid bit length of the preceding adjacent time-series data. This allows the number of valid bits in a time-series data to be associated with its adjacent data, enabling the storage of time-series data by storing the valid bits and their number of valid bits. This, in turn, effectively achieves lossless compression of high-precision floating-point time-series data and further improves the compression rate of time-series data.

[0063] In the specific implementation process, a flag bit with two control bits is generated for each time series data according to the following rules:

[0064] 1) For time-series data with an XOR value of 0, the flag bit is 0;

[0065] 2) If the XOR value of the corresponding timing data is not 0, then the first control bit of its flag is 1;

[0066] 3) such as Figure 2 As shown in (a), if the effective bit length of the timing data is less than or equal to the effective bit length of its adjacent preceding timing data and the length difference is less than 6 bits, then the second control bit of its flag bit is 0.

[0067] 4) such as Figure 2 As shown in (b), if the effective bit length of the timing data is greater than the effective bit length of the adjacent preceding timing data, then the second control bit of its flag bit is 1.

[0068] This invention generates a flag bit with two control bits for the timing data, enabling the timing data type and the storage structure of the current XOR value to be known during decompression (e.g., if the flag bit is 10, the length of the previous valid bit is used to store the current valid bit). This reduces the length of the stored valid bits (reducing the number of storage bits by 6 bits - |the length of the valid bits of the previous XOR value - the length of the current XOR value|). It also helps to complete the storage of timing data by storing the valid bits and their number of valid bits, thereby effectively achieving lossless compression of high-precision floating-point timing data and further improving the compression ratio of timing data.

[0069] To verify the effectiveness of the technical solution of this invention, two real trajectory datasets were selected for compression experiments in this embodiment. The trajectory data consists of a large number of GPS points, where the latitude and longitude are high-precision floating-point (double) time-series data.

[0070] 1. Dataset

[0071] In this experiment, we used two real datasets:

[0072] 1) Chongqing Taxi Dataset. We used trajectory data of Chongqing taxis on a specific day in 2017, which contains 69,430,938 GPS trajectory points of 12,041 taxis, with a sampling rate of 15 seconds.

[0073] 2) Beijing taxi dataset T-drive (from), which contains 8,619,111 GPS trajectory points of 7,992 taxis, with a sampling frequency of 177s.

[0074] 2. Experimental Setup

[0075] We primarily verified the compression ratio and compression / decompression time of the above algorithm. Experimental analysis was conducted using different sampling rates and datasets (implemented in Java). All experiments were performed on a 64-bit host running Windows 11, with an 11th-generation Intel(R) Core(TM) i5-11400 CPU and 16GB of RAM.

[0076] 3. Experimental Methods

[0077] 1) An improved XOR without Trail algorithm (XORWT) for trajectory data.

[0078] 2) Comparison method: XOR algorithm.

[0079] 4. Comparison Standards

[0080] We mainly evaluate the effectiveness of the algorithm from three aspects:

[0081] 1) Compression Ratio = Original Data Size / Compressed Data Size. In other words, the higher the compression ratio, the better the compression effect.

[0082] 2) Compression Time, unit: milliseconds (ms);

[0083] 3) Decompression Time, unit: milliseconds (ms). The shorter the compression and decompression time, the shorter the algorithm's usage time, indicating a better algorithm.

[0084] 1) Compression ratio

[0085] like Figure 3 As shown, the improved XOR algorithm achieves a significant improvement in compression ratio in both datasets. Due to the high precision of latitude and longitude information, a large number of significant bits remain after the XOR operation. The XORWT algorithm achieves an improvement of about 10% over the XOR algorithm, verifying the effectiveness of our method for removing trailing zeros and making it more suitable for compressing high-precision floating-point data.

[0086] 2) Compression time

[0087] We also conducted time compression experiments on XOR and XORWT, and the results are as follows: Figure 4 As shown. Because it eliminates the need to determine the position of the valid bits, XORWT is faster than XOR in terms of compression speed.

[0088] 3) Decompression time

[0089] Since we haven't optimized the XOR stitching process here, the result is as follows: Figure 5As shown, it is clear that the concatenation operation significantly slows down the decompression efficiency of XOR, while XORWT discards the storage of trailing zeros and can directly decode into an XOR value, thus achieving higher decompression efficiency.

[0090] Example 2:

[0091] This embodiment also discloses a method for decompressing high-precision floating-point time-series data based on the lossless compression method for high-precision floating-point time-series data in Embodiment 1.

[0092] A method for decompressing high-precision floating-point time-series data, specifically including:

[0093] S01: Obtain the compressed time series data package to be decompressed;

[0094] S02: Obtain the first-ranked time series data;

[0095] S03: For the second-order timing data: First, pad the leading and trailing zeros of the corresponding timing data's significant bits to obtain the binary XOR value; then, convert the corresponding timing data's binary XOR value into the original format corresponding to the first-order timing data to obtain the corresponding XOR value; finally, restore the XOR calculation based on the corresponding timing data's XOR value and the first-order timing data to obtain the second-order timing data.

[0096] S04: Starting from the third timing data: First, determine the number of significant bits of the corresponding timing data based on the flag bit of the corresponding timing data and the number of significant bits of the adjacent preceding timing data; then, pad the significant bits with leading and trailing zeros to obtain the binary XOR value; subsequently, convert the binary XOR value of the corresponding timing data back to its original format to obtain the corresponding XOR value; finally, restore the XOR calculation by combining the XOR value of the corresponding timing data with the timing data of the adjacent preceding timing data to obtain the corresponding timing data.

[0097] S05: Repeat step S04 until all time series data is decompressed.

[0098] This invention directly stores the XOR value of timing data starting from the least significant bit, omitting only the storage of leading zeros and discarding the processing of trailing zeros (the number of trailing zeros in the XOR data is small, but storing the number of trailing zeros requires 6 bits, while the number of trailing zeros in the XOR data is very likely less than 6 bits). This reduces the number of bits required to store timing data, and no additional leading zeros, trailing zeros, or significant bits need to be concatenated during decompression. In other words, no additional optimization design is required, which can reduce the compression and decompression time of timing data, thereby ensuring the compression effect and decompression cost of floating-point timing data, and thus effectively reducing the amount of data stored and the transmission cost.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A method for storing and transmitting high-precision floating-point time-series data, characterized in that, include: S1: Sort the time-series data to be compressed; The time-series data consists of GPS trajectory point data of taxis; S2: Perform an XOR operation on each time series data with its adjacent time series data in sequence to obtain the corresponding XOR value; S3: Remove leading zeros from the XOR values ​​of each time series data and generate the corresponding valid bits; S4: Determine the number of significant bits based on the significant bits of each time series data; In step S4, the number of effective bits is first determined based on the effective bit length of the second-order timing data. Then, starting from the third time series data, the effective bit length of each time series data is compared with the effective bit length of the adjacent preceding time series data in order to determine the number of effective bits corresponding to each time series data. The number of significant bits in time-series data is determined using the following rules: 1) If the effective bit length of the current time series data is greater than the effective bit length of its adjacent previous time series data, then the number of effective bits is determined according to the effective bit length of the current time series data; 2) If the effective bit length of the current time series data is less than or equal to the effective bit length of its adjacent previous time series data, then the number of effective bits of the current time series data is consistent with the number of effective bits of its adjacent previous time series data, and the insufficient effective bit length is padded with 0. Generate a flag bit with two control bits for each time series data entry using the following rules: 1) For time-series data with an XOR value of 0, the flag bit is 0; 2) If the XOR value of the corresponding timing data is not 0, then the first control bit of its flag is 1; 3) If the effective bit length of the timing data is less than or equal to the effective bit length of the adjacent preceding timing data and the length difference is less than 6 bits, then the second control bit of its flag bit is 0; 4) If the effective bit length of a timing data is greater than the effective bit length of its adjacent preceding timing data, then the second control bit of its flag is 1; S5: When compressing time-series data, store the valid bits of the corresponding time-series data and the number of valid bits; When compressing timing data, the timing data in the first position is stored first, and then the flag bits, valid bits and the number of valid bits of other timing data are stored in sequence to generate the corresponding timing data compressed package. Store and transmit time-series data compressed packages.

2. The method for storing and transmitting high-precision floating-point time-series data as described in claim 1, characterized in that: In step S2, starting from the second-ranked time series data, each time series data is XORed with its adjacent preceding time series data in sequence to obtain the XOR value corresponding to each time series data.

3. The method for storing and transmitting high-precision floating-point time-series data as described in claim 2, characterized in that: In step S3, the XOR value of each time sequence data is first converted into binary form, and then leading zeros are removed to generate the valid bits in binary form.

4. The method for storing and transmitting high-precision floating-point time-series data as described in claim 3, characterized in that: Leading zeros refer to all the zeros before the first significant digit in the XOR value, which in binary is represented by all the zeros before the highest bit with a value of 1.

5. The method for storing and transmitting high-precision floating-point time-series data as described in claim 1, characterized in that: In step S5, when compressing timing data, the timing data in the first position is stored first, and then the flag bits, valid bits and the number of valid bits of other timing data are stored in sequence to generate the corresponding timing data compressed package.

6. The method for storing and transmitting high-precision floating-point time-series data as described in claim 5, characterized in that: When compressing time-series data, time-series data with an XOR value of 0 are stored as a 1-bit flag bit 0.

7. A method for decompressing high-precision floating-point time-series data, characterized in that: The implementation of the data storage and transmission method for high-precision floating-point time-series data based on claim 6 specifically includes: S01: Obtain the compressed time-series data package to be decompressed for storage and transmission; S02: Obtain the first-ranked time series data; S03: For the second-order timing data: First, pad the leading zeros of the corresponding timing data's valid bits according to the valid bit length to obtain the binary XOR value; then, convert the corresponding timing data's binary XOR value into the format corresponding to the first-order timing data, i.e., the original format, to obtain the corresponding XOR value; finally, perform an XOR calculation based on the corresponding timing data's XOR value and the first-order timing data to obtain the second-order timing data; S04: Starting from the third timing data: First, determine the number of significant bits of the corresponding timing data based on the flag bit of the corresponding timing data and the number of significant bits of the adjacent preceding timing data; then, pad the significant bits with leading zeros to obtain the binary XOR value; subsequently, convert the binary XOR value of the corresponding timing data back to its original format to obtain the corresponding XOR value; finally, perform an XOR calculation based on the XOR value of the corresponding timing data and the timing data of the adjacent preceding timing data to obtain the corresponding timing data. S05: Repeat step S04 until all timing data is decompressed.

Citation Information

Patent Citations

  • Data compression method for streaming time series data

    CN109871362A

  • Data compression and decompression method, device and equipment

    CN110266316A