Data compression method and electronic device

Abstract

The present invention discloses a data compression method and electronic device. In this embodiment, raw data is converted into integer data based on business precision, and then the integer data is converted into a binary byte array to generate a storage file for storage. Compared to current common compression methods, this embodiment improves compression speed and compression ratio.

Description

Technical Field

[0001] The present application relates to data compression technology, and in particular to a data compression method and electronic device. Background Art

[0002] The high-frequency data generated by various industrial equipment requiring real-time monitoring during operation is crucial for condition monitoring, online diagnosis, and offline analysis. However, the millisecond-level high-frequency data of numerous monitoring variables poses a significant challenge to the historical storage of this massive amount of data. Due to the massive volume of high-frequency data, conventional data compression algorithms suffer from slow speeds and low compression ratios, making them difficult to meet the practical requirements of production in large data-intensive scenarios. Summary of the Invention

[0003] To address the above issues, the present application provides a data compression method and an electronic device to improve the compression ratio and compression speed.

[0004] The technical solutions provided in the embodiments of this application include:

[0005] A data compression method, comprising:

[0006] Acquire N raw data at the same time interval, and generate a first array based on the N raw data;

[0007] Converting each original data in the first array into integer type data according to the business precision and the specified value in the first array to obtain a second array, wherein the specified value includes a maximum value or a minimum value;

[0008] determining a binary storage bit width for binary storage of the data in the second array according to a maximum value in the second array;

[0009] According to the binary storage bit width, convert each data in the second array into a bit stream of equal width, and then convert it into a binary byte array;

[0010] A storage file is generated and stored according to the binary byte array.

[0011] Optionally, converting each original data in the first array into integer type data according to the business precision and the specified value in the first array includes:

[0012] For any original data in the first array, convert the original data into integer type data according to the difference between the original data and the minimum value in the first array and the business precision; or

[0013] For any original data in the first array, the original data is converted into integer type data according to the difference between the original data and the maximum value in the first array and the business precision.

[0014] Optionally, converting the original data into integer type data based on the difference between the original data and the minimum value in the first array and the business precision includes:

[0015] For the original data P[i], the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0016] V[i]=(P[i]-Pmin) / 2δ

[0017] Among them, Pmin is the minimum value in the first array, and δ is the service accuracy.

[0018] Optionally, converting the original data into integer type data based on the difference between the original data and the maximum value in the first array and the business precision includes:

[0019] For the original data P[i], the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0020] V[i]=(Pmax-P[i]) / 2δ

[0021] Among them, Pmax is the maximum value in the first array, and δ is the service accuracy.

[0022] Optionally, determining a binary storage bit width for binary storage of data in the second array according to the maximum value in the second array includes:

[0023] The bit width of the binary value corresponding to the maximum value in the second array is determined as the binary storage bit width for binary storage of the data in the second array.

[0024] Optionally, the bit width of the binary value corresponding to the maximum value in the second array is determined by the following formula:

[0025]

[0026] Wherein, Vmax is the maximum value in the second array, For a floor operation, W is the bit width of the binary data value corresponding to the maximum value in the second array.

[0027] Optionally, generating a storage file according to the binary byte array includes:

[0028] Creating a file header according to the total number of original data, the maximum value of the first array, the minimum value of the first array, the maximum value in the second array, and the binary storage bit width;

[0029] A temporary file is created according to the file header and the binary byte array, and the temporary file is compressed into a storage file.

[0030] Optionally, after compressing the temporary file into a storage file, the method further includes:

[0031] Decompressing the storage file to obtain the binary byte array;

[0032] According to the binary storage bit width, each data in the binary byte array is converted into a bit stream of equal width, and then converted into the second array;

[0033] The data in the second array is converted into original data according to the maximum value of the first array, the minimum value of the first array, and the maximum value of the second array.

[0034] Optionally, converting the data of the second array into original data according to the maximum value of the first array, the minimum value of the first array, and the maximum value of the second array includes:

[0035] For the data V[i] of the second array, the data of the second array is converted using the following formula to obtain the original data corresponding to the data of the second array:

[0036] P[i]=Pmin+(Pmax-Pmin) / Vmax*V[i]

[0037] Among them, Pmax is the maximum value of the first array, Pmin is the minimum value of the first array, and Vmax is the maximum value of the second array.

[0038] An embodiment of the present application further provides an electronic device, comprising:

[0039] one or more processors;

[0040] A machine-readable storage medium is used to store one or more computer-readable instructions, and when the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement any of the methods disclosed above.

[0041] It can be seen from the above technical solution that in this application, the original data is converted into integer type data according to business precision to remove invalid data that exceeds the business precision, thereby improving the compression ratio, and the converted integer type data is also stored in binary format, so that the data occupies less space and has faster reading and writing speeds, thereby achieving improved compression ratio and compression speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0043] Figure 1 A flow chart of the method provided in the embodiment of the present application;

[0044] Figure 2 A flowchart of a method provided for another embodiment of the present application;

[0045] Figure 3 A diagram of the device structure provided in an embodiment of the present application;

[0046] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0048] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0049] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various devices, this information should not be limited to these terms. These terms are only used to distinguish devices of the same type from each other. For example, a first device may also be referred to as a second device, and similarly, a second device may be referred to as a first device without departing from the scope of this application. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining".

[0050] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0051] The present application provides a data compression method for optimizing existing data compression methods to improve compression ratio and compression speed.

[0052] See also Figure 1 , Figure 1 A flow chart of the method provided in an embodiment of the present application.

[0053] like Figure 1 As shown, the process may include the following steps:

[0054] Step S101 : acquiring N original data at the same time interval, and generating a first array based on the N original data.

[0055] In this embodiment, the same time interval can be ensured at the data source end to obtain the original data. For example, the data source end sends the original data to the data receiving end at the same time interval, so that the same time interval can be achieved for obtaining the original data. In order to prevent the loss of the original data at a certain time point sent by the data source end, the data receiving end can also perform corresponding processing on the received original data. For example, when the original data at a certain time point received by the data receiving end is lost, the lost original data can be supplemented at the data receiving end, for example, the previous lost original data can be directly supplemented as the lost original data, or the average of the previous original data and the next original data of the lost original data can be taken as the lost original data. This application does not impose any restrictions on this.

[0056] Step S102 : converting each original data in the first array into integer type data according to the business precision and the specified value in the first array to obtain a second array, wherein the specified value includes a maximum value or a minimum value.

[0057] In this embodiment, the device receiving the raw data can directly obtain the service accuracy of the raw data. For example, the service accuracy of this compression is defined on the device as an absolute accuracy of 0.1mm or a relative accuracy of 0.1% of the full scale. In some more complex service scenarios, the device can also perform calculations based on the received raw data to obtain the service accuracy. For example, the service scenario contains multiple variables, such as wind speed, temperature, voltage, etc., each of which corresponds to different raw data. When the number of variables is large, it is cumbersome to define the absolute accuracy of each variable on the device. In this case, the service accuracy of each variable can be uniformly defined as a relative accuracy, such as a service accuracy of 0.1% of the full scale for each variable. However, the full scale of each variable is also different, and setting the full scale of each variable on the device is also cumbersome, especially when each variable is increased. It requires manual maintenance. Those skilled in the art know that the more data a measuring device measures, the closer the peak-to-valley difference of all measured data is to the full scale. Therefore, in actual scenarios, in order to reduce manual maintenance costs, the peak-to-valley difference of the raw data corresponding to each variable obtained 24 hours a day can be set as the corresponding full scale to obtain the service accuracy of each variable.

[0058] In this embodiment, the maximum value and the minimum value in the first array can be obtained by comparing the original data in the first array.

[0059] Optionally, the converting of the original data in the first array into integer type data based on the business precision and the maximum value or the minimum value in the first array includes:

[0060] For any original data in the first array, convert the original data into integer data according to the difference between the original data and the minimum value in the first array and the business precision; or

[0061] For any original data in the first array, the original data is converted into integer type data according to the difference between the original data and the maximum value in the first array and the business precision.

[0062] Optionally, converting the original data into integer type data based on the difference between the original data and the minimum value in the first array and the business precision includes:

[0063] For the original data P[i], the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0064] V[i]=(P[i]-Pmin) / 2δ

[0065] Among them, Pmin is the minimum value in the first array, and δ is the above-mentioned business accuracy.

[0066] In this embodiment, in addition to converting the original data into integer type data, data details that exceed the business precision are removed through the difference between the original data and the minimum value in the first array and the business precision to reduce data, thereby achieving the effect of improving the compression ratio.

[0067] Optionally, the converting of the original data into integer data based on the difference between the original data and the maximum value in the first array and the business precision includes:

[0068] For the original data P[i], the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0069] V[i]=(Pmax-P[i]) / 2δ

[0070] Among them, Pmax is the maximum value in the first array, and δ is the above-mentioned service accuracy.

[0071] In this embodiment, through the difference between the original data and the maximum value in the first array and the business precision, the original data is converted into integer type data as in the above embodiment and the data details exceeding the business precision are removed to reduce the data, thereby achieving the effect of improving the compression ratio.

[0072] Step S103 : determining a binary storage bit width for binary storage of the data in the second array according to the maximum value in the second array.

[0073] In this embodiment, the maximum value in the second array can be obtained by comparing the integer data in the second array. If the data in the second array is obtained by converting the original data using the formula V[i]=(P[i]-Pmin) / 2δ, it is obvious that when P[i] is Pmax, the value of V[i] is the largest. Therefore, the maximum value in the second array can be calculated by Vmax=(Pmax-Pmin) / 2δ.

[0074] Optionally, the determining, based on the maximum value in the second array, a binary storage bit width for binary storage of the data in the second array includes:

[0075] The bit width of the binary value corresponding to the maximum value in the second array is determined as the binary storage bit width for binary storage of the data in the second array.

[0076] Because data stored in binary format occupies less space, in this embodiment, to improve compression effectiveness and compression ratio, the integer data in the second array is stored in binary format. Those skilled in the art will appreciate that when integer data is stored in binary format, the value of the data is proportional to the binary storage bit width. For example, the integer 5, when converted to binary format, has a value of 101, and a binary storage bit width of 3. The value of 8, when converted to binary format, has a value of 1000, and a binary storage bit width of 8 is 4. Therefore, in this embodiment, the binary storage bit width only needs to meet the maximum value of the integer data to meet the requirements of any other data in the second array.

[0077] Optionally, the bit width of the binary value corresponding to the maximum value in the second array is determined by the following formula:

[0078]

[0079] Among them, Vmax is the maximum value in the second array above, For a floor operation, W is the bit width of the binary data value corresponding to the maximum value in the second array.

[0080] In this embodiment, the calculation formula The bit width W of the binary data value of the maximum value Vmax in the second array is determined. Obviously, W also satisfies the minimum value W of any data in the second array. This allows W to satisfy any data in the second array without affecting the compression effect or reducing the compression ratio due to W being too large.

[0081] Step S104: convert the data in the second array into a bit stream of equal width according to the binary storage bit width, and then convert it into a binary byte array. Generate a storage file according to the binary byte array and store it.

[0082] In this embodiment, the length of the binary byte array can be calculated using the bit width W of the binary data value and the total number of original data. For example, if W is 10 and the total number of original data is 1 million, the length of the binary byte array can be calculated as 1.25 million using the formula N*W / 8. The data receiving device then creates a binary byte array of 1.25 million in length, converts the data in the second array into a bit stream of equal width, and stores it in the binary byte array.

[0083] It should be noted that when the calculated result of the length of the binary byte array contains a decimal, the calculated result is rounded up to ensure that the created binary byte array can completely store all the data in the second array.

[0084] Optionally, the above-mentioned storage file generated based on the binary byte array includes:

[0085] Creating a file header according to the total number of the original data, the maximum value of the first array, the minimum value of the first array, the maximum value of the second array, and the binary storage bit width;

[0086] A temporary file is created according to the file header and the binary byte array, and the temporary file is compressed into a storage file.

[0087] In this embodiment, the data receiving device can create a corresponding file header after the data in the second array is stored in the aforementioned binary byte array. Furthermore, after creating a temporary file based on the file header and the aforementioned binary byte array, the user can locally configure a corresponding compression tool on the data receiving device to compress the temporary file based on speed and compression ratio requirements. For example, if a compression tool with a fast compression speed is desired, the local compression tool can be set to gzip. If a compression tool with a high compression ratio is desired, the local compression tool can be set to bzip2.

[0088] So far, completed Figure 1 The process shown.

[0089] In this embodiment, the data receiving device converts the original data into integer data based on the service precision to remove invalid data that exceeds the service precision, thereby improving the compression ratio. Furthermore, the integer data is stored in binary format, which reduces the data space and increases the read and write speed, thereby improving the compression ratio and compression speed.

[0090] Based on the data compression method described above in this application, this application also provides a data decompression method corresponding to the above data compression method.

[0091] like Figure 2 As shown, the process may include the following steps:

[0092] Step S201: decompress the stored file to obtain the binary byte array.

[0093] In this embodiment, the stored file may be decompressed by a compression tool locally provided on the device at the data receiving end to obtain a temporary file storing the binary byte data, and the temporary file may be read to obtain the above binary byte data.

[0094] Step S202 : converting each data in the binary byte array into a bit stream of equal width according to the binary storage bit width, and then converting it into the second array.

[0095] In this embodiment, the binary storage bit width of the binary byte array is saved when creating the file header in the data compression method. Therefore, the binary storage bit width can be obtained by reading the file header in the temporary file. The data in the binary byte array is then converted into integer type data based on the binary storage bit width to obtain the second array.

[0096] Step S203 , converting the data in the second array into original data according to the maximum value of the first array, the minimum value of the first array, and the maximum value of the second array.

[0097] Optionally, converting the data of the second array into original data according to the maximum value of the first array, the minimum value of the first array, and the maximum value of the second array includes:

[0098] For the data V[i] of the second array, the data of the second array is converted using the following formula to obtain the original data corresponding to the data of the second array:

[0099] P[i]=Pmin+(Pmax-Pmin) / Vmax*V[i]

[0100] Among them, Pmax is the maximum value of the first array, Pmin is the minimum value of the first array, and Vmax is the maximum value of the second array.

[0101] In this embodiment, since the maximum value Pmax of the first array, the minimum value Pmin of the first array, and the maximum value Vmax of the second array are saved when creating the file header in the above data compression method, the binary storage bit width can be obtained by reading the file header in the above temporary file. And the conversion is performed by using the data conversion formula V[i]=(P[i]-Pmin) / 2δ in the above data compression method. The conversion method is as follows: Vmax=(Pmax-Pmin) / 2δ, and δ=(Pmax-Pmin) / 2Vmax is obtained. After replacing δ in the formula, the formula P[i]=Pmin+(Pmax-Pmin) / Vmax*V[i] can be obtained. By using this formula, the integer type data V[i] is converted into the original data P[i], and the converted original data P[i] does not contain invalid data that exceeds the business precision.

[0102] So far, completed Figure 2 The process shown.

[0103] In this embodiment, a data decompression method is also provided, which can decompress the compressed file according to the above compression method to obtain the original data, and the decompressed original data does not contain invalid data that exceeds the business accuracy.

[0104] The following is a specific embodiment of Figure 1 、 2 In this embodiment, the compression and decompression of 20ms high-frequency data files of a wind turbine generator set for 24 hours is used as an example.

[0105] First, through calculation, we know that the amount of raw data in one second is 1000ms / 20ms=50, and the total amount of raw data in 24 hours is N=50 / second*86400 seconds / day=4.32 million. The 4.32 million raw data constitute the first array P[].

[0106] In this embodiment, the data compression step includes:

[0107] 1. Define the business accuracy requirement δ of this variable as 0.0125% of the full scale.

[0108] 2. Comparing all the original data, the maximum value Pmax=0.745439 and the minimum value Pmin=-0.649694 among the 4.32 million original data are obtained.

[0109] 3. Since the peak-to-valley value is less than the full scale, the actual accuracy calculated by the peak-to-valley value is higher than the business requirement. Here, the peak-to-valley difference (Pmax-Pmin) of the day is used as the full scale, and δ = (Pmax-Pmin) * 0.0125% is obtained.

[0110] 4. By using the formula V[i]=(P[i]-Pmin) / 2δ, the original data in the first array are converted into integer type data and form the second array V[].

[0111] 5. Calculate the maximum value of the second array V[] using the formula Vmax=(Pmax-Pmin) / 2δ, and obtain Vmax=(Pmax-Pmin) / (2*(Pmax-Pmin)*0.0125%)=4000.

[0112] 5. Through the formula Calculate the binary storage bit width of the second array V[] and you can get

[0113] 6. Calculate the length of the binary byte array: L = N * W / 8 = 4.32 million * 12 / 8 = 6.48 million, create a binary byte array Y with a length of 6.48 million, and convert each data in the second array into an equal-width bit stream with a width of 12 and store it in the binary byte array Y.

[0114] 7. Create a file header: H = {4320000 (N), 0.745439 (Pmax), -0.649694 (Pmin), 4000 (Vmax), 12 (W)}, and create a temporary file F = {H (file header), Y (binary byte array)}.

[0115] 8. Set the local compression tool of the wind turbine to bzip2 and compress the temporary file F into a storage file. In this embodiment, the compression ratio is about 50 and the compression time is about 0.6 seconds.

[0116] The data decompression steps include:

[0117] 1. Decompress the stored file with bzip2 to obtain a temporary file F = {H (file header), Y (binary byte array)}.

[0118] 2. Read the file header in the temporary file and obtain H = {4320000 (N), 0.745439 (Pmax), -0.649694 (Pmin), 4000 (Vmax), 12 (W)}.

[0119] 3. Using the binary storage bit width W in the file header, convert each data in the binary byte array Y in the temporary file into a bit stream of equal width, and then convert it into integer type data to obtain the above-mentioned second array.

[0120] 4. Convert each data in the second array into original data using the formula P[i]=Pmin+(Pmax-Pmin) / Vmax*V[i], P[i]=Pmin+(Pmax-Pmin) / Vmax*V[i]=-0.649694+1.395133 / 4000*V[i].

[0121] At this point, the process shown in the above embodiment is completed.

[0122] In this embodiment, by converting the 24-hour raw data of the wind turbine into integer data to remove invalid data that exceeds the business precision, and then storing the converted integer data in binary format, the raw data compression ratio reaches 50, and the compression time is 0.6 seconds. The raw data is compressed using currently used compression methods, and the effect is as follows:

[0123] Compression method Compression ratio Compression speed (ms) Decompression speed (ms) CSV (lossless) gzip compression 3.17 2222 331 CSV (lossless) bzip2 compression 3.3 3078 1453 BIN (lossless) gzip compression 1.16 756 164 BIN (lossless) bzip2 compression 1.17 1902 1167

[0124] By comparison, it can be seen that the compression method provided in this embodiment achieves the effect of improving the compression ratio and compression speed.

[0125] Furthermore, the present application compares the present application with currently commonly used compression methods under different precisions and different compression tools.

[0126] The contrast effect under the smooth curve is as follows:

[0127] Compression method Compression ratio Compression speed (ms) Decompression speed (ms) CSV (lossless) gzip compression 3.17 2222 331 CSV (lossless) bzip2 compression 3.3 3078 1453 BIN (lossless) gzip compression 1.16 756 164 BIN (lossless) bzip2 compression 1.17 1902 1167 This method (accuracy 0.012%) gzip compression 17.37 259 73 This method (accuracy 0.012%) bzip2 compression 54.35 603 186 This method (accuracy 0.00076%) gzip compression 2.68 399 130 This method (accuracy 0.00076%) bzip2 compression 5.81 703 410

[0128] The comparison effect under the jump curve is as follows:

[0129] Compression method Compression ratio Compression speed (ms) Decompression speed (ms) CSV (lossless) gzip compression 2.96 3505 364 CSV (lossless) bzip2 compression 3.63 4731 2239 BIN (lossless) gzip compression 1.08 683 135 BIN (lossless) bzip2 compression 1.07 2162 1270 This method (accuracy 0.20%) gzip compression 9.19 453 100 This method (accuracy 0.20%) bzip2 compression 10.92 483 339 This method (accuracy 0.012%) gzip compression 5.96 909 106 This method (accuracy 0.012%) bzip2 compression 8.04 637 359

[0130] It can be seen that this application achieves the effect of improving the compression ratio and compression speed under different precisions and different compression tools.

[0131] The method provided by this application is described above. The device provided by this application is described below:

[0132] See also Figure 3 , Figure 3 This is a diagram of the device structure provided in the embodiment of this application. Figure 3 As shown, the device may include:

[0133] An acquiring unit 301 is configured to acquire N raw data at the same time interval and generate a first array based on the N raw data;

[0134] A first data conversion unit 302 is configured to convert each original data in the first array into integer type data according to the business precision and the specified value in the first array to obtain a second array, wherein the specified value includes a maximum value or a minimum value;

[0135] A storage bit width determining unit 303 is configured to determine a binary storage bit width for binary storage of the data in the second array according to a maximum value in the second array;

[0136] The second data conversion unit 304 is configured to convert each data in the second array into a bit stream of equal width according to the binary storage bit width, and then convert the bit stream into a binary byte array;

[0137] Storage unit 305: used to generate a storage file based on the binary byte array and store it.

[0138] Optionally, the first data conversion unit converts each original data in the first array into integer type data according to the business precision and the specified value in the first array, including:

[0139] For any original data in the first array, convert the original data into integer type data according to the difference between the original data and the minimum value in the first array and the business precision; or

[0140] For any original data in the first array, the original data is converted into integer type data according to the difference between the original data and the maximum value in the first array and the business precision.

[0141] Optionally, converting the original data into integer type data based on the difference between the original data and the minimum value in the first array and the business precision includes:

[0142] For the original data P[i], the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0143] V[i]=(P[i]-Pmin) / 2δ

[0144] Among them, Pmin is the minimum value in the first array, and δ is the service accuracy.

[0145] Optionally, converting the original data into integer type data based on the difference between the original data and the maximum value in the first array and the business precision includes:

[0146] For the original data P[i], the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0147] V[i]=(Pmax-P[i]) / 2δ

[0148] Among them, Pmax is the maximum value in the first array, and δ is the service accuracy.

[0149] Optionally, the determining unit for determining the storage bit width for binary storage of the data in the second array according to the maximum value in the second array includes:

[0150] The bit width of the binary value corresponding to the maximum value in the second array is determined as the binary storage bit width for binary storage of the data in the second array.

[0151] Optionally, the bit width of the binary value corresponding to the maximum value in the second array is determined by the following formula:

[0152]

[0153] Wherein, Vmax is the maximum value in the second array, For a floor operation, W is the bit width of the binary data value corresponding to the maximum value in the second array.

[0154] Optionally, the storage unit generates a storage file according to the binary byte array, including:

[0155] Creating a file header according to the total number of original data, the maximum value of the first array, the minimum value of the first array, the maximum value in the second array, and the binary storage bit width;

[0156] A temporary file is created according to the file header and the binary byte array, and the temporary file is compressed into a storage file.

[0157] Optionally, after the storage unit compresses the temporary file into a storage file, the storage unit further includes:

[0158] Decompression unit 306: configured to decompress the stored file to obtain the binary byte array;

[0159] According to the binary storage bit width, each data in the binary byte array is converted into a bit stream of equal width, and then converted into the second array;

[0160] The data in the second array is converted into original data according to the maximum value of the first array, the minimum value of the first array, and the maximum value of the second array.

[0161] Optionally, the decompression unit converting the data of the second array into original data according to the maximum value of the first array, the minimum value of the first array, and the maximum value of the second array includes:

[0162] For the data V[i] of the second array, the data of the second array is converted using the following formula to obtain the original data corresponding to the data of the second array:

[0163] P[i]=Pmin+(Pmax-Pmin) / Vmax*V[i]

[0164] Among them, Pmax is the maximum value of the first array, Pmin is the minimum value of the first array, and Vmax is the maximum value of the second array.

[0165] The present application also provides Figure 3 The hardware structure of the device shown. Figure 4 , Figure 4 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 4As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0166] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0167] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0168] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0169] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0170] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] Furthermore, these computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0174] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A data compression method, characterized in that: The method comprises: Acquire N raw data at the same time interval, and generate a first array based on the N raw data; Converting each original data in the first array into integer type data according to the business precision and the specified value in the first array to obtain a second array, wherein the specified value includes a maximum value or a minimum value; Determining a binary storage bit width for binary storage of the data in the second array according to the maximum value in the second array, where the binary storage bit width is the minimum number of binary bits required to satisfy storage requirements for any data in the second array; According to the binary storage bit width, convert each data in the second array into a bit stream of equal width, and then convert it into a binary byte array; A storage file is generated and stored according to the binary byte array.

2. The method according to claim 1, characterized in that The converting of each original data in the first array into integer type data according to the business precision and the specified value in the first array includes: For any original data in the first array, convert the original data into integer type data according to the difference between the original data and the minimum value in the first array and the business precision; or For any original data in the first array, the original data is converted into integer type data according to the difference between the original data and the maximum value in the first array and the business precision.

3. The method according to claim 2, characterized in that The converting the original data into integer type data according to the difference between the original data and the minimum value in the first array and the business precision includes: For the original data P[i], the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data: V[i]=(P[i]-Pmin) / 2δ Among them, Pmin is the minimum value in the first array, and δ is the service accuracy.

4. The method according to claim 2, characterized in that The converting of the original data into integer type data according to the difference between the original data and the maximum value in the first array and the business precision includes: For the original data P[i], the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data: V[i]=(Pmax-P[i]) / 2δ Among them, Pmax is the maximum value in the first array, and δ is the service accuracy.

5. The method according to claim 1, wherein Determining the binary storage bit width for binary storage of the data in the second array according to the maximum value in the second array includes: The bit width of the binary value corresponding to the maximum value in the second array is determined as the binary storage bit width for binary storage of the data in the second array.

6. The method according to claim 4, characterized in that The bit width of the binary value corresponding to the maximum value in the second array is determined by the following formula: Wherein, Vmax is the maximum value in the second array, For a floor operation, W is the bit width of the binary data value corresponding to the maximum value in the second array.

7. The method according to claim 1, characterized in that Generating a storage file according to the binary byte array includes: Creating a file header according to the total number of original data, the maximum value of the first array, the minimum value of the first array, the maximum value in the second array, and the binary storage bit width; A temporary file is created according to the file header and the binary byte array, and the temporary file is compressed into a storage file.

8. The method according to claim 7, characterized in that After compressing the temporary file into a storage file, the method further includes: Decompressing the storage file to obtain the binary byte array; According to the binary storage bit width, each data in the binary byte array is converted into a bit stream of equal width, and then converted into the second array; The data in the second array is converted into original data according to the maximum value of the first array, the minimum value of the first array, and the maximum value of the second array.

9. The method according to claim 8, characterized in that The converting the data of the second array into original data according to the maximum value of the first array, the minimum value of the first array, and the maximum value of the second array comprises: For the data V[i] of the second array, the data of the second array is converted using the following formula to obtain the original data corresponding to the data of the second array: P[i]=Pmin+(Pmax-Pmin) / Vmax*V[i] Among them, Pmax is the maximum value of the first array, Pmin is the minimum value of the first array, and Vmax is the maximum value of the second array.

10. An electronic device, characterized in that: The electronic device includes: one or more processors; A machine-readable storage medium for storing one or more computer-readable instructions, wherein when the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

Images