Data compression method and electronic device

Abstract

The present invention discloses a data compression method and electronic device. In this embodiment, after dividing raw data into multiple data segments, each segment is converted into integer data based on the conversion order and the maximum and minimum values of each segment. The integer data is then differentially processed, and the differentially processed data is then converted into a binary byte array to generate a storage file for storage. Compared to currently available compression methods, the present invention improves compression speed and compression ratio.

Description

Technical Field

[0001] The present application relates to data compression technology, and in particular to a data compression method and electronic device. Background Art

[0002] The high-frequency data generated by various industrial equipment requiring real-time monitoring during operation is crucial for condition monitoring, online diagnosis, and offline analysis. However, the millisecond-level high-frequency data of numerous monitoring variables poses a significant challenge to the historical storage of this massive amount of data. Due to the massive volume of high-frequency data, conventional data compression algorithms suffer from slow speeds and low compression ratios, making them difficult to meet the practical requirements of production in large data-intensive scenarios. Summary of the Invention

[0003] To address the above issues, the present application provides a data compression method and an electronic device to improve the compression ratio and compression speed.

[0004] The technical solutions provided in the embodiments of this application include:

[0005] A data compression method, comprising:

[0006] Acquire raw data at the same time intervals, and divide the acquired raw data into a plurality of first arrays;

[0007] For any first array, determining a conversion step size of the first array according to the conversion order and the maximum and minimum values in the first array;

[0008] Converting the first array into a second array according to the conversion step and a specified value in the original data, wherein the data type in the second array is an integer type, and the specified value includes a maximum value or a minimum value;

[0009] Convert the second array into a third array according to the set difference type;

[0010] determining a binary storage bit width for binary storage of the data in the third array according to a maximum value in the third array;

[0011] According to the binary storage bit width, each data in the third array is converted into a bit stream of equal width, and then converted into a binary byte array;

[0012] According to the binary byte array corresponding to the acquired original data, a storage file is generated and stored. Optionally, the conversion step is determined by the following formula:

[0013] S=(Pmax-Pmin) / (C-1)

[0014] Wherein, Pmax is the maximum value in the first array, Pmin is the minimum value in the first array, and C is the conversion order.

[0015] Optionally, the above method further includes:

[0016] The conversion step size is updated according to the maximum allowed precision and / or the minimum allowed precision.

[0017] Optionally, updating the conversion step size according to the maximum allowable precision and / or the minimum allowable precision includes:

[0018] When the conversion step length is less than the maximum allowable precision, halving the conversion order and recalculating the conversion step length, and comparing the recalculated conversion step length with the maximum allowable precision again, until the conversion step length is greater than or equal to the maximum allowable precision, then determining the conversion step length greater than or equal to the maximum allowable precision as the updated conversion step length; or

[0019] When the conversion step length to be determined is greater than the minimum allowable precision, the conversion order is multiplied and the conversion step length is recalculated, and the recalculated conversion step length is compared with the minimum allowable precision again until the conversion step length is less than or equal to the minimum allowable precision. The conversion step length that is less than or equal to the minimum allowable precision is then determined as the updated conversion step length.

[0020] Optionally, converting the first array into a second array according to the conversion step and a specified value in the original data includes:

[0021] For any original data P[i] in the first array, the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0022] V[i]=(P[i]-Pmin) / S

[0023] Wherein, Pmin is the minimum value in the original data, and S is the conversion step size.

[0024] Optionally, converting the first array into a second array according to the conversion step and a specified value in the original data includes:

[0025] For any original data P[i] in the first array, the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0026] V[i]=(Pmax-P[i]) / S

[0027] Wherein, Pmax is the maximum value in the original data, and S is the conversion step size.

[0028] Optionally, converting the second array into a third array according to the set difference type includes:

[0029] performing first-order difference processing on the second array to obtain a first-order difference array, and determining the first-order difference array as a target difference array;

[0030] The target difference array is converted into a third array according to a specified value in the target difference array, wherein the specified value includes a maximum value or a minimum value.

[0031] Optionally, converting the second array into a third array according to the set difference type includes:

[0032] performing first-order difference processing on the second array to obtain a first-order difference array;

[0033] performing second-order difference processing on the second array to obtain a second-order difference array;

[0034] Comparing a binary storage bit width determined based on the maximum and minimum values in the first-order difference array with a binary storage bit width determined based on the maximum and minimum values in the second-order difference array to determine a target difference array;

[0035] The target difference array is converted into a third array according to a specified value in the target difference array, wherein the specified value includes a maximum value or a minimum value.

[0036] Optionally, determining a binary storage bit width for binary storage of data in the third array according to the maximum value in the third array includes:

[0037] The bit width of the binary value corresponding to the maximum value in the third array is determined as the binary storage bit width for binary storage of the data in the third array.

[0038] Optionally, the bit width of the binary value corresponding to the maximum value in the third array is determined by the following formula:

[0039]

[0040] Wherein, Dmax is the maximum value in the third array, For a floor operation, W is the bit width of the binary data value corresponding to the maximum value in the third array.

[0041] Optionally, generating a storage file based on the binary byte array corresponding to the acquired original data includes:

[0042] Creating a compressed block according to the binary byte array converted from the third array, the compressed block including: a specified value in the original data, the first data in the second array, the minimum value in the target difference array, the difference type, the binary storage bit width, the conversion step size, and the binary byte array, wherein the specified value includes a maximum value or a minimum value;

[0043] Creating a file header, wherein the file header includes the total number of the original data and the number of the original data in the compressed block;

[0044] A temporary file is created according to the file header and the compressed block, and the temporary file is compressed into a storage file.

[0045] Optionally, after compressing the temporary file into a storage file, the method further includes:

[0046] Decompressing the storage file to obtain the temporary file;

[0047] Reading the temporary file to obtain the file header and the compressed block;

[0048] Obtaining a binary storage bit width and a binary byte array in the compressed block, converting each data in the binary byte array into a bit stream of equal width according to the binary storage bit width, and converting the bit stream into the third array;

[0049] The data in the third array is converted into original data based on the first data of the second array recorded in the compressed block, the differential type, the conversion step, the minimum value in the target differential array, and the specified value in the original data, where the specified value includes the maximum value or the minimum value.

[0050] An embodiment of the present application further provides an electronic device, comprising:

[0051] one or more processors;

[0052] A machine-readable storage medium is used to store one or more computer-readable instructions, and when the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement any of the methods disclosed above.

[0053] It can be seen from the above technical solutions that in this application, by dividing the original data into multiple data segments, the local peak-to-valley difference in the subsequent encoding process is reduced to improve the compression speed and compression ratio. Each data segment determines a different conversion step size through the maximum and minimum values of each data, ensuring the high-frequency details of the curve of each data segment and reducing unnecessary excessively high resolution. Each data segment is converted into integer type data based on the conversion order and the maximum and minimum values of each data segment to remove invalid data that exceeds the business precision, thereby further improving the compression speed and compression ratio. The integer type data is also differentially processed to reduce the data value that needs to be stored, thereby further improving the compression speed and compression ratio. Finally, the differentially processed data is stored in binary format, so that the data occupies less space and has faster reading and writing speeds, thereby improving the compression ratio and compression speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0055] Figure 1 A flow chart of the method provided in the embodiment of the present application;

[0056] Figure 2 A flowchart of a method provided for another embodiment of the present application;

[0057] Figure 3 A diagram of the device structure provided in an embodiment of the present application;

[0058] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0060] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0061] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various devices, this information should not be limited to these terms. These terms are only used to distinguish devices of the same type from each other. For example, a first device may also be referred to as a second device, and similarly, a second device may be referred to as a first device without departing from the scope of this application. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining".

[0062] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0063] The present application provides a data compression method for optimizing existing data compression methods to improve compression ratio and compression speed.

[0064] See also Figure 1 , Figure 1 A flow chart of the method provided in an embodiment of the present application.

[0065] like Figure 1 As shown, the process may include the following steps:

[0066] Step S101 : acquiring original data at the same time interval, and dividing the acquired original data into a plurality of first arrays.

[0067] In this embodiment, the same time interval can be ensured at the data source end to obtain the original data. For example, the data source end sends the original data to the data receiving end at the same time interval, so that the same time interval can be achieved for obtaining the original data. In order to prevent the loss of the original data at a certain time point sent by the data source end, the data receiving end can also perform corresponding processing on the received original data. For example, when the original data at a certain time point received by the data receiving end is lost, the lost original data can be supplemented at the data receiving end, for example, the previous lost original data can be directly supplemented as the lost original data, or the average of the previous original data and the next original data of the lost original data can be taken as the lost original data. This application does not impose any restrictions on this.

[0068] In this embodiment, a partitioning time can be set at the data receiving end, and the acquired raw data can be partitioned according to the partitioning time. For example, in this embodiment, the data receiving end acquires a piece of raw data every 20 milliseconds. When compressing raw data received for one hour, assuming the partitioning time is one minute, the raw data received for one hour is partitioned into 60 first arrays. The data receiving end acquires 1 / (20 / 1000) = 50 pieces of raw data per second. Therefore, the number of raw data contained in each first array is 60 seconds * 50 seconds / piece = 3000 pieces.

[0069] Step S102 : For any first array, determine the conversion step length of the first array according to the conversion order, and the maximum value and the minimum value in the first array.

[0070] In this embodiment, a conversion order C can be set in the device at the data receiving end, for example, 2^8=256. For any first array, the conversion step of the first array is determined by the difference between the maximum value and the minimum value in the first array and the conversion order C.

[0071] Optionally, the above conversion step is determined by the following formula:

[0072] S=(Pmax-Pmin) / (C-1)

[0073] Among them, Pmax is the maximum value in the above-mentioned first array, Pmin is the minimum value in the above-mentioned first array, and C is the conversion order.

[0074] Optionally, the above method further includes:

[0075] The conversion step size is updated according to the maximum allowed precision and / or the minimum allowed precision.

[0076] In this embodiment, the data receiving device may also obtain the maximum allowable precision or minimum allowable precision of all original data and compare the conversion step size with the maximum allowable precision or minimum allowable precision to determine whether the original conversion step size needs to be updated. Alternatively, the maximum allowable precision and the minimum allowable precision may be obtained simultaneously and compared with the original conversion step size to determine whether the original conversion step size needs to be updated.

[0077] Optionally, updating the conversion step size according to the maximum allowable precision and / or the minimum allowable precision includes:

[0078] When the above conversion step size is less than the above maximum allowable accuracy, halve the above conversion order and recalculate the conversion step size, and compare the recalculated conversion step size with the above maximum allowable accuracy again until the conversion step size is greater than or equal to the maximum allowable accuracy, then determine the conversion step size greater than or equal to the maximum allowable accuracy as the updated conversion step size; or

[0079] When the above conversion step size to be determined is greater than the above minimum allowable accuracy, double the above conversion order and recalculate the conversion step size, and compare the recalculated conversion step size with the above minimum allowable accuracy again until the conversion step size is less than or equal to the minimum allowable accuracy, then determine the conversion step size less than or equal to the minimum allowable accuracy as the updated conversion step size.

[0080] In this embodiment, after the device at the data receiving end determines the conversion step size S through the preset conversion order and the maximum and minimum values in the first array, it can also make a comparison based on the obtained maximum allowable accuracy Rmax and / or minimum allowable accuracy Rmin. The specific comparison situations are as follows:

[0081] 1. When the device at the data receiving end only obtains the maximum allowable accuracy and S >= Rmax, then S does not need to be updated.

[0082] 2. When the device at the data receiving end only obtains the maximum allowable accuracy Rmax and S < Rmax, then halve the conversion order, record it as C1, and obtain the conversion step size S1 through the formula S = (Pmax - Pmin) / (C1 - 1).

[0083] Compare S1 with Rmax. If S1 < Rmax, then halve C1, record it as C2, and obtain the conversion step size S2 through the formula S = (Pmax - Pmin) / (C2 - 1).

[0084] Compare S2 with Rmax. If S2 < Rmax, then perform the above operation again.

[0085] Repeat the above steps until Sn >= Rmax, then determine Sn as the updated conversion step size.

[0086] 3. When the device at the data receiving end only obtains the minimum allowable accuracy Rmin and S <= Rmin, then S does not need to be updated.

[0087] 4. When the device at the data receiving end only obtains the minimum allowable accuracy Rmin and S > Rmin, then double the conversion order, record it as C1, and obtain the conversion step size S1 through the formula S = (Pmax - Pmin) / (C1 - 1).

[0088] Compare S1 with Rmin. If S1 > Rmin, double C1 and record it as C2. Obtain the conversion step S2 through the formula S = (Pmax - Pmin) / (C2 - 1).

[0089] Compare S2 with Rmin. If S2 > Rmin, perform the above operations again.

[0090] Repeat the above steps until Sn <= Rmin, then determine Sn as the updated conversion step.

[0091] 5. When the device at the data receiving end simultaneously obtains Rmin and Rmax, and Rmax <= S <= Rmin, then S does not need to be updated.

[0092] 6. When the device at the data receiving end simultaneously obtains Rmin and Rmax, when S < Rmax, refer to Step 2; when S > Rmin, refer to Step 4.

[0093] In this embodiment, each data segment determines different conversion steps through the maximum and minimum values of each data, and updates the conversion steps according to the maximum allowable accuracy and minimum allowable accuracy of the original data, so that the conversion steps of each data block are within the range of the maximum allowable accuracy and minimum allowable accuracy, ensuring the high-frequency details of the curves of each data segment and reducing unnecessary high resolutions.

[0094] Step S103, convert the first array into a second array according to the above conversion step and the specified value in the above original data.

[0095] In this embodiment, the specified values in the original data include the maximum value and the minimum value. The data type in the second array is an integer type.

[0096] Optionally, the conversion of the first array into the second array according to the above conversion step and the specified value in the first array includes:

[0097] For any original data P[i] in the first array, convert the original data through the following formula and perform a rounding operation on the converted data to obtain the integer type data corresponding to the original data:

[0098] V[i] = (P[i] - Pmin) / S

[0099] where Pmin is the minimum value in the first array and S is the above conversion step.

[0100] In this embodiment, through the difference between the original data in the first array and the minimum value in the first array and the conversion step, in addition to converting the original data into integer type data, data details that exceed the business precision are also removed to reduce data, thereby achieving the effect of improving the compression ratio.

[0101] Optionally, converting the first array into the second array according to the conversion step and the specified value in the first array includes:

[0102] For any original data P[i] in the first array, the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0103] V[i]=(Pmax-P[i]) / S

[0104] Wherein, Pmax is the maximum value in the first array, and S is the conversion step.

[0105] In this embodiment, through the difference between the original data in the first array and the maximum value in the first array and the business precision, the original data is converted into integer type data as in the above embodiment and the data details exceeding the business precision are removed to reduce the data, thereby achieving the effect of improving the compression ratio.

[0106] In step S104 , the second array is converted into a third array according to the set difference type.

[0107] In this embodiment, a differential type can be set at the data receiving end, and differential processing can be performed on the second array based on the differential type, and the differentially processed data can be determined as the third array. In this embodiment, differential processing of the second array can effectively reduce the data size and improve the data compression ratio.

[0108] Optionally, the converting of the second array into a third array according to the set difference type includes:

[0109] Performing first-order difference processing on the second array to obtain a first-order difference array;

[0110] The first-order difference array is converted into a third array according to a specified value in the first-order difference array, wherein the specified value includes a maximum value or a minimum value.

[0111] In this embodiment, the first-order difference processing can be performed on the second array using the formula D1[i]=V[i+1]-V[i] (i=0...N-2), where N is the number of data in the second array.

[0112] At the same time, each value in the first-order difference array is compared to obtain the maximum value D1max and the minimum value D1min.

[0113] When the first-order difference array is converted into a third array according to D1max, the data size of the first-order difference array is reduced through the formula H1[i]=D1max-D1[i], further improving the compression ratio.

[0114] When the first-order difference array is converted into a third array according to D1min, the data size of the first-order difference array is reduced through the formula H1[i]=D1[i]-D1min, thereby further improving the compression ratio.

[0115] Optionally, the converting of the second array into a third array according to the set difference type includes:

[0116] Performing first-order difference processing on the second array to obtain a first-order difference array;

[0117] Performing second-order difference processing on the second array to obtain a second-order difference array;

[0118] Comparing the binary storage bit width determined based on the maximum and minimum values in the first-order difference array with the binary storage bit width determined based on the maximum and minimum values in the second-order difference array to determine a target difference array;

[0119] The target difference array is converted into a third array according to a specified value in the target difference array, wherein the specified value includes a maximum value or a minimum value.

[0120] In this embodiment, the first-order difference processing can be performed on the second array using the formula D1[i]=V[i+1]-V[i] (i=0...N-2) to obtain a first-order difference array, where N is the number of data in the second array. The first-order difference array is then subjected to the first-order difference processing using the formula D2[0]=D1[0], D2[i]=D1[i]-D1[i-1] (i=1...N-2) to obtain a second-order difference array of the second array.

[0121] At this time, the first-order difference array can be processed using the formula H1[i]=D1max-D1[i] or H1[i]=D1[i]-D1min to further reduce the data size of the first-order difference array. At the same time, the second-order difference array can be processed using the formula H2[i]=D2max-D2[i] or H2[i]=D2[i]-D2min to further reduce the data size of the second-order difference array.

[0122] Because the smaller the binary storage bit width, the better the compression effect, the binary storage bit width W1 of the processed first-order difference array can be calculated using the maximum value H1max in the processed first-order difference array, and the binary storage bit width W2 of the processed second-order difference array can be calculated using the maximum value H2max in the processed second-order difference array. Comparing W1 and W2, when W1 <= W2, the first-order difference is determined to be the difference type in this embodiment, and the first-order difference array is determined to be the target difference type. When W1 > W2, the second-order difference is determined to be the difference type in this embodiment, and the second-order difference array is determined to be the target difference type.

[0123] It should be noted that in the above scheme, the first-order difference array and the second-order difference array are processed respectively, and then the corresponding binary storage bit width is obtained through H1max and H2max. The maximum value in the processed array is actually the difference between the maximum and minimum values in the first-order difference array and the second-order difference array. Therefore, in order to reduce the amount of calculation, the first-order difference array and the second-order difference array can be processed without processing. W1 and W2 are calculated directly based on the difference between the maximum and minimum values in the first-order difference array and the second-order difference array. By comparing W1 and W2, the difference type is determined, and the second array is processed based on the difference type. The differential array processed by the row is determined as the target differential array. For example, if W1<=W2, the target differential data is the first-order differential array D1[i]. At this time, the array D1[i] is processed by the formula H1[i]=D1max-D1[i] or H1[i]=D1[i]-D1min to obtain the third array H1[i]. When W1>W2, the target differential data is the second-order differential array D2[i]. At this time, the second-order differential array is processed by the formula H2[i]=D2max-D2[i] or H2[i]=D2[i]-D2min to obtain the third array H2[i].

[0124] Step S105 : determining a binary storage bit width for binary storage of the data in the third array according to the maximum value in the third array.

[0125] In this embodiment, the maximum value in the third array can be obtained by comparing each integer type data in the third array.

[0126] Optionally, determining the binary storage bit width for binary storage of the data in the third array based on the maximum value in the third array includes:

[0127] The bit width of the binary value corresponding to the maximum value in the third array is determined as the binary storage bit width for binary storage of the data in the third array.

[0128] Because data stored in binary format occupies less space, in this embodiment, to improve compression effectiveness and compression ratio, the integer data in the second array is stored in binary format. Those skilled in the art will appreciate that when integer data is stored in binary format, the value of the data is proportional to the binary storage bit width. For example, the integer 5, when converted to binary format, has a value of 101, and a binary storage bit width of 3. The value of 8, when converted to binary format, has a value of 1000, and a binary storage bit width of 8 is 4. Therefore, in this embodiment, the binary storage bit width only needs to meet the maximum value of the integer data to meet the requirements of any other data in the second array.

[0129] Optionally, the bit width of the binary value corresponding to the maximum value in the third array is determined by the following formula:

[0130]

[0131] Wherein, Dmax is the maximum value in the third array, For a floor operation, W is the bit width of the binary data value corresponding to the maximum value in the third array.

[0132] In this embodiment, the calculation formula The bit width W of the binary data value of the maximum value Vmax in the second array is determined. Obviously, W also satisfies the minimum value W of any data in the second array. This allows W to satisfy any data in the second array without affecting the compression effect or reducing the compression ratio due to W being too large.

[0133] Step S106: Convert each data in the third array into a bit stream of equal width according to the binary storage bit width, and then convert it into a binary byte array. Generate a storage file and store it according to the binary byte array corresponding to the original data obtained.

[0134] In this embodiment, the length of the binary byte array can be calculated using the bit width W of the binary data value and the total number of original data in the first array. Since performing differential processing on the second array to generate the third array will cause the data to be subtracted by 1, the length of the binary byte array is obtained using the formula (N-1)*W / 8. The data receiving device creates a binary byte array based on this length, and converts each data in the third array into a bit stream of equal width, which is stored in the binary byte array.

[0135] It should be noted that when the calculated length of a binary byte array contains a decimal, the result is rounded up to ensure that the created binary byte array can fully store all the data in the second array. For example, if W is 10 and the total number of original data in the first array is 1 million, the length of the binary byte array is calculated using the formula (N-1)*W / 8. After rounding up the calculated result, the length of the binary byte array is 1.25 million.

[0136] Optionally, the above-mentioned generation of a storage file based on the binary byte array corresponding to the acquired original data includes:

[0137] Creating a compressed block based on the binary byte array converted from the third array, the compressed block including: the specified value in the first array, the first data in the second array, the minimum value in the target difference array, the difference type, the binary storage bit width, the conversion step size, and the binary byte array, wherein the specified value includes the maximum value or the minimum value;

[0138] Creating a file header, wherein the file header includes the total number of the original data and the number of the original data in the compressed block;

[0139] A temporary file is created according to the file header and the compressed block, and the temporary file is compressed into a storage file.

[0140] In this embodiment, taking a first array as an example, the device at the data receiving end creates a compressed block corresponding to the first array. The compressed block records a specified value in the first array, which is the value when the first array is converted to the second array, the conversion step of the first array, the first data of the second array converted from the first array, the differential type of the second array converted to the third array, the minimum value in the above-mentioned target differential array, the binary storage bit width of the third array, and the binary byte group converted from the third array. The recorded data is used to restore the compressed block to the first array.

[0141] The other first arrays are converted into corresponding compressed blocks according to the above-mentioned first array conversion method, and the order of the compressed blocks is recorded according to the order of the first arrays.

[0142] After converting each first array into a compressed block, the data receiving device creates a file header, which records the total number of original data and the number of data in the first array, and is used to restore the compressed data to the original data.

[0143] A temporary file is created based on the sequentially merged compressed blocks and the creation header. This temporary file records the location of each compressed block, each compressed block, and the aforementioned file header. The user can also configure the corresponding compression tool on the data receiving device to compress the temporary file based on speed and compression ratio requirements. For example, if a compression tool with a fast compression speed is required, the local compression tool can be set to gzip. If a compression tool with a high compression ratio is required, the local compression tool can be set to bzip2.

[0144] It should be noted that, in this embodiment, according to the actual business scenario, each compression block may be compressed using the above-mentioned compression tool first, and then the compression blocks may be sequentially merged to create a temporary file.

[0145] For example, if actual business scenarios require frequent batch decompression of multiple compressed blocks within a certain time period, you can choose to merge the compressed blocks first and then compress them using a compression tool. If actual usage scenarios frequently require decompression of only a single compressed block, you can choose to compress it using a compression tool first and then merge it. This way, you can decompress the corresponding compressed blocks individually. This application does not specify whether to merge first or compress using a compression tool first.

[0146] So far, completed Figure 1 The process shown.

[0147] In this embodiment, the original data is divided into multiple data segments to reduce the local peak-to-valley difference in the subsequent encoding process to improve the compression speed and compression ratio. Each data segment is converted into integer type data based on the conversion order and the maximum and minimum values of each data segment to remove invalid data that exceeds the business precision, thereby further improving the compression speed and compression ratio. The integer type data is also differentially processed to reduce the data value that needs to be stored, thereby further improving the compression speed and compression ratio. Finally, the differentially processed data is stored in binary format, so that the data occupies less space and has faster reading and writing speeds, thereby improving the compression ratio and compression speed.

[0148] Based on the data compression method described above in this application, this application also provides a data decompression method corresponding to the above data compression method.

[0149] like Figure 2 As shown, the process may include the following steps:

[0150] Step S201: decompress the stored file to obtain the temporary file, and read the temporary file to obtain the file header and the compressed block.

[0151] In this embodiment, the stored file can be decompressed by a compression tool locally set on the device at the data receiving end to obtain a temporary file storing the file header and each compressed block, and the temporary file can be read to obtain the above file header and each compressed block.

[0152] Step S202: Obtain the binary storage bit width and binary byte array in the compressed block, convert each data in the binary byte array into a bit stream of equal width according to the binary storage bit width, and convert the bit stream into the third array.

[0153] In this embodiment, the binary byte array and the binary storage bit width corresponding to the binary byte array are saved when creating the compressed block in the data compression method. Therefore, the binary byte array and the binary storage bit width corresponding to the binary byte array can be obtained by reading the compressed block in the temporary file. The data in the binary byte array is then converted into integer data based on the binary storage bit width to obtain the third array.

[0154] S203, converting the data in the third array into original data based on the first data of the second array recorded in the compression block, the differential type, the conversion step, the minimum value in the target differential array, and the specified value in the first array, wherein the specified value includes the maximum value or the minimum value.

[0155] In this embodiment, the third array can be converted into the second array according to the difference type in each compressed block. For example, a difference array is converted into the third array according to the minimum value in the difference array.

[0156] When the difference type is first-order difference, the third array is converted to a first-order difference array using the formula D1[i]=H1[i]+D1min(i=0…N-2). Since the first data V[0] of the second array is recorded in the compressed block, the first-order difference array can be converted to the second array using the formula V[i+1]=V[i]+D1[i](i=0…N-2).

[0157] When the differential type is a second-order differential, the third array can be converted into a second-order differential array by the formula D2[i]=D[i]+Dmin(i=0…N-2), and the second-order differential array can be converted into a first-order differential array by the formula D1[0]=D2[0]+Dmin,D1[i]=D1[i-1]+D2[i], and then the first-order differential array is converted into the second array based on the first data V[0] of the second array recorded in the compressed block and the formula V[i+1]=V[i]+D1[i](i=0…N-2).

[0158] Taking the minimum value in the first array during data compression as an example, the minimum value of the first array in the compressed block is read, the step size is converted, and the second array is converted to the first array according to the formula P[i]=Pmin+S*V[i](i=0…N-1),V[i+1]=V[i]+D1[i](i=0…N-2).

[0159] The above method is executed on each compressed block to obtain the first array corresponding to each compressed block, and according to the order of each compressed block, the order of each first array can be known, so that the order of each original data will not be changed.

[0160] So far, completed Figure 2 The process shown.

[0161] In this embodiment, a data decompression method is also provided, which can decompress the compressed file according to the above compression method to obtain the original data, and the decompressed original data does not contain invalid data that exceeds the business accuracy.

[0162] The following is a specific embodiment of Figure 1 、 2 In this embodiment, the compression and decompression of 20ms high-frequency data files of a wind turbine generator set for 24 hours is used as an example.

[0163] First, through calculation, we know that the amount of raw data in one second is 1000ms / 20ms=50, and the total amount of raw data in 24 hours is N=50 / second*86400 seconds / day=4.32 million.

[0164] In this embodiment, the data compression step includes:

[0165] 1. Define the maximum allowable accuracy of the raw data Rmax = 1 / 65535 full scale, the global minimum allowable accuracy Rmin = 1 / 255 full scale, and the preset conversion order Cpre = 256.

[0166] 2. Comparing all the raw data, we obtain the maximum value Tmax = 0.745439 and the minimum value Tmin = -0.649694 among the 4.32 million raw data points. The peak-to-valley difference Tmax - Tmin = 1.395133. Because the peak-to-valley difference is less than or equal to the full scale, the actual accuracy calculated using the peak-to-valley value exceeds the business requirement. In this solution, the peak-to-valley difference is used as the full scale, resulting in Rmax = 2.128E-5 and Rmin = 5.471E-3.

[0167] 3. Cut the 4.32 million raw data of 24 hours into 8640 segments of 10 seconds in length. Each segment contains N = fT = (1000ms / 20ms) * 10 = 500 raw data, which is recorded as the first array P[].

[0168] 4. Compare to obtain the maximum value Pmax and the minimum value Pmin of each first array, and calculate the conversion step size at the preset conversion order C. Taking the first 10-second data segment as an example: S = (Pmax - Pmin) / (C - 1) = (-0.047262 + 0.050072) / (256 - 1) = 1.11×10⁻⁵ < Rmax. Halve C to get C1 as 128, recalculate to get S1 = 2.212×10⁻⁵ > Rmax, then determine S1 as the conversion step size.

[0169] 5. Convert the first array P[] to the second array V[] by rounding according to the formula V[i] = (P[i] - Pmin) / S = (P[i] + 0.050072) / 2.212×10⁻⁵ (i = 0…N - 1).

[0170] 6. Calculate the first-order difference D1[i] = V[i + 1] - V[i] (i = 0…N - 2) of the second array V[], obtain the first-order difference array D1 and the maximum value D1max and the minimum value D1min of D1: D1max = 2, D1min = -3.

[0171] 7. Calculate the peak-to-valley difference D1pp and the storage bit number W1 of the first-order difference array D1: D1pp = D1max - D1min = 2 + 3 = 5, W1 = log(D1pp) / log(2) + 1 = log(5) / log(2) + 1 = 3 (round down).

[0172] 8. Calculate the second-order difference D2[0] = D1[0], D2[i] = D1[i] - D1[i - 1] (i = 1…N - 2) of the second array V[], obtain the second-order difference array D2 and the maximum value D2max and the minimum value D2min of D2: D2max = 3, D2min = -2.

[0173] 9. Calculate the peak-to-valley difference D2pp and the storage bit number W2 of the second-order difference sequence D2: D2pp = D2max - D2min = 3 + 2 = 5, W2 = log(D2pp) / log(2) + 1 = log(5) / log(2) + 1 = 3 (round down)

[0174] 10. Compare the sizes of W1 and W2, and take the difference array D1 corresponding to W1 as the target difference array.

[0175] 11. Convert the target difference array to the third array H[] according to the formula H[i] = D[i] - Dmin. And sequentially and seamlessly push H[] into the binary byte array Y with a length of W*(N - 1) / 8 = 3*499 / 8 = 188 (round up).

[0176] 12. Create a compressed block R for each segment of data (duration T = 10 seconds). The first segment of data example is: R = {Pmin (minimum value of the first array = -0.050072), S (conversion step = 2.212E-5), V[0] (first data of the second array = 127), DT (difference type = first-order difference), Dmin (minimum value of the target difference array = -3), W (binary storage bit width = 3), Y (binary array length = 188 bytes)}.

[0177] 13. Create file header: H = {X (total number of original data = 4.32 million), N (data block capacity = 500)}

[0178] 14. Sequentially merge all R blocks to generate a temporary file: F = {H (file header), O[i] (i = 0...N-1) (each R block position), R[i] (i = 0...N-1) (each R block content)}

[0179] 15. Set the local compression tool of the wind turbine to bzip2, and compress the temporary file F into a storage file. In this embodiment, the compression ratio is approximately 40, and the compression time is approximately 0.15 seconds.

[0180] The data decompression steps include:

[0181] 1. Use bzip2 to decompress the stored file and obtain a temporary file F = {H (file header), O[i] (i = 0 ... N-1) (each R block position), R[i] (i = 0 ... N-1) (each R block content)}.

[0182] 2. Read F to obtain the file header H = {4.32 million (total number of original data X), 3000 (data block capacity N)}.

[0183] 3. Calculate the total number of data blocks: M = X / N = 4320000 / 500 = 8640 (round up).

[0184] 4. Read the data block index O and obtain the address offset O[i] (i=0...M-1) and length L[i]=O[i+1]-O[i] (i=0...M-1) of each data block relative to the file header, where L[M] is the file length.

[0185] 5. Read the data block R. The first data example R = {Pmin (minimum value of the first array = -0.050072), S (conversion step = 2.212E-5), V[0] (first data of the second array = 127), DT (differential type = first-order difference), Dmin (minimum value of the target differential array = -3), W (binary storage bit width = 3), Y (binary array length = 188 bytes)}.

[0186] 6. Using the binary storage bit width W in the file header, convert each data in the binary byte array Y in the data block into a bit stream of equal width, and then convert it into integer type data to obtain the third array H[].

[0187] 7) The differential type in the read data block is first-order differential, and the target differential array minimum value Dmin=-3. According to the formula D1[i]=H[i]+Hmin=H[i]-1(i=0…N-2), the third array H[] is converted into the target differential array, which is the first-order differential array D1[] in this embodiment.

[0188] 8. Read the minimum value Pmin of the original data in the data block, the conversion step S and the first data V[0] of the second array, according to the formula

[0189] V[0]=127, P[0]=Pmin+S*V[0]=-0.050072+2.212E-5*127,

[0190] V[1]=V[0]+D1[0]=127+0=127,

[0191] P[1]=Pmin+S*V[1]=-0.050072+2.212E-5*127…Convert the data in the first-order difference array into the original data.

[0192] At this point, the process shown in the above embodiment is completed.

[0193] In this embodiment, the original data of the wind turbine for 24 hours is converted into Figure 1 、 2 The compression and decompression methods in the above table achieve a compression ratio of 36, a compression time of 0.15 seconds, and a decompression time of 0.1 seconds for the original data. The results of the compression of the original data using the currently used compression methods are shown in Table 1:

[0194] Compression method Compression ratio Compression speed (ms) Decompression speed (ms) CSV (lossless) gzip compression 3.17 2222 331 CSV (lossless) bzip2 compression 3.30 3078 1453 BIN (lossless) gzip compression 1.16 756 164 BIN (lossless) bzip2 compression 1.17 1902 1167

[0195] Table 1

[0196] By comparison, it can be seen that the compression method provided in this embodiment achieves the effect of improving the compression ratio, compression speed and decompression speed.

[0197] Furthermore, the present application compares the present application with currently commonly used compression methods under different precisions and different compression tools.

[0198] The comparison effect under the smooth curve is shown in Table 2:

[0199] Compression method Compression ratio Compression speed (ms) Decompression speed (ms) CSV (lossless) gzip compression 3.17 2222 331 CSV (lossless) bzip2 compression 3.30 3078 1453 BIN (lossless) gzip compression 1.16 756 164 BIN (lossless) bzip2 compression 1.17 1902 1167 This method (T = 10 seconds, C = 64 steps) does not compress 14.49 57 45 This method (T = 10 seconds, C = 64 steps) gzip compression 32.99 184 54 This method (T = 10 seconds, C = 64 steps) bzip2 compression 37.96 168 115 This method (T = 10 seconds, C = 256 steps) does not compress 13.61 52 48 This method (T = 10 seconds, C = 256 steps) gzip compression 27.88 151 59 This method (T = 10 seconds, C = 256 steps) bzip2 compression 35.82 159 117

[0200] The comparison results under the jump curve of Table 2 are shown in Table 3:

[0201] Compression method Compression ratio Compression speed (ms) Decompression speed (ms) CSV (lossless) gzip compression 2.96 3505 364 CSV (lossless) bzip2 compression 3.63 4731 2239 BIN (lossless) gzip compression 1.08 683 135 BIN (lossless) bzip2 compression 1.07 2162 1270 This method (T = 10 seconds, C = 64 steps) does not compress 5.25 61 53 This method (T = 10 seconds, C = 64 steps) gzip compression 5.47 181 80 This method (T = 10 seconds, C = 64 steps) bzip2 compression 5.55 436 297 This method (T = 10 seconds, C = 256 steps) does not compress 3.96 69 50 This method (T = 10 seconds, C = 256 steps) gzip compression 4.18 225 87 This method (T = 10 seconds, C = 256 steps) bzip2 compression 4.22 549 376

[0202] Table 3

[0203] It can be seen that this application achieves the effect of improving the compression ratio, compression speed and decompression speed under different precisions and different compression tools.

[0204] The method provided by this application is described above. The device provided by this application is described below:

[0205] See also Figure 3 , Figure 3 This is a diagram of the device structure provided in the embodiment of the present application. Figure 1 and 2 As shown in the process. Figure 3 As shown:

[0206] The present application provides a data compression device, characterized in that the device includes:

[0207] The data division unit 301 is configured to obtain original data at the same time interval and divide the obtained original data into a plurality of first arrays;

[0208] A conversion step determining unit 302 is configured to determine, for any first array, a conversion step of the first array based on the conversion order and the maximum and minimum values in the first array;

[0209] a first data conversion unit 303, configured to convert the first array into a second array according to the conversion step and a specified value in the first array, wherein the data type of the second array is an integer type, and the specified value includes a maximum value or a minimum value;

[0210] A second data conversion unit 304 is configured to convert the second array into a third array according to a set difference type;

[0211] a storage bit width determining unit 305, configured to determine a binary storage bit width for binary storage of the data in the third array according to a maximum value in the third array;

[0212] A third data conversion unit 306 is configured to convert each data in the third array into a bit stream of equal width according to the binary storage bit width, and then convert the bit stream into a binary byte array;

[0213] The storage unit 307 is used to generate and store a storage file based on the binary byte array corresponding to the acquired original data.

[0214] Optionally, the conversion step size in the conversion step size determining unit 302 is determined by the following formula:

[0215] S=(Pmax-Pmin) / (C-1)

[0216] Wherein, Pmax is the maximum value in the first array, Pmin is the minimum value in the first array, and C is the conversion order.

[0217] Optionally, the conversion step determining unit 302 further includes:

[0218] The conversion step size is updated according to the maximum allowed precision and / or the minimum allowed precision.

[0219] Optionally, the conversion step size determining unit 302 updates the conversion step size according to the maximum allowable precision and / or the minimum allowable precision, including:

[0220] When the conversion step length is less than the maximum allowable precision, halving the conversion order and recalculating the conversion step length, and comparing the recalculated conversion step length with the maximum allowable precision again, until the conversion step length is greater than or equal to the maximum allowable precision, then determining the conversion step length greater than or equal to the maximum allowable precision as the updated conversion step length; or

[0221] When the conversion step length to be determined is greater than the minimum allowable precision, the conversion order is multiplied and the conversion step length is recalculated, and the recalculated conversion step length is compared with the minimum allowable precision again until the conversion step length is less than or equal to the minimum allowable precision. The conversion step length that is less than or equal to the minimum allowable precision is then determined as the updated conversion step length.

[0222] Optionally, the first data conversion unit 303 converts the first array into a second array according to the conversion step and a specified value in the first array, including:

[0223] For any original data P[i] in the first array, the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0224] V[i]=(P[i]-Pmin) / S

[0225] Wherein, Pmin is the minimum value in the first array, and S is the conversion step size.

[0226] Optionally, the first data conversion unit 303 converts the first array into a second array according to the conversion step and a specified value in the first array, including:

[0227] For any original data P[i] in the first array, the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data:

[0228] V[i]=(Pmax-P[i]) / S

[0229] Wherein, Pmax is the maximum value in the first array, and S is the conversion step size.

[0230] Optionally, the second data conversion unit 304 converts the second array into a third array according to a set difference type, including:

[0231] performing first-order difference processing on the second array to obtain a first-order difference array, and determining the first-order difference array as a target difference array;

[0232] The target difference array is converted into a third array according to a specified value in the target difference array, wherein the specified value includes a maximum value or a minimum value.

[0233] Optionally, the second data conversion unit 304 converts the second array into a third array according to a set difference type, including:

[0234] performing first-order difference processing on the second array to obtain a first-order difference array;

[0235] performing second-order difference processing on the second array to obtain a second-order difference array;

[0236] Comparing a binary storage bit width determined based on the maximum and minimum values in the first-order difference array with a binary storage bit width determined based on the maximum and minimum values in the second-order difference array to determine a target difference array;

[0237] The target difference array is converted into a third array according to a specified value in the target difference array, wherein the specified value includes a maximum value or a minimum value.

[0238] Optionally, the determining unit 305 of the storage bit width determines the binary storage bit width for binary storage of the data in the third array according to the maximum value in the third array, including:

[0239] The bit width of the binary value corresponding to the maximum value in the third array is determined as the binary storage bit width for binary storage of the data in the third array.

[0240] Optionally, the bit width of the binary value corresponding to the maximum value in the third array is determined by the following formula:

[0241]

[0242] Wherein, Dmax is the maximum value in the third array, For a floor operation, W is the bit width of the binary data value corresponding to the maximum value in the third array.

[0243] Optionally, the storage unit 307 generates a storage file according to the binary byte array corresponding to the acquired original data, including:

[0244] Creating a compressed block according to the binary byte array converted from the third array, the compressed block including: a specified value in the first array, the first data in the second array, the minimum value in the target difference array, the difference type, the binary storage bit width, the conversion step size, and the binary byte array, wherein the specified value includes a maximum value or a minimum value;

[0245] Creating a file header, wherein the file header includes the total number of the original data and the number of the original data in the compressed block;

[0246] A temporary file is created according to the file header and the compressed block, and the temporary file is compressed into a storage file.

[0247] Optionally, after the storage unit 307 compresses the temporary file into a storage file, the steps further include:

[0248] A decompression unit 308, configured to decompress the stored file to obtain the temporary file;

[0249] Reading the temporary file to obtain the file header and the compressed block;

[0250] Obtaining a binary storage bit width and a binary byte array in the compressed block, converting each data in the binary byte array into a bit stream of equal width according to the binary storage bit width, and converting the bit stream into the third array;

[0251] According to the first data of the second array recorded in the compressed block, the differential type, the conversion step, the minimum value in the target differential array, and the specified value in the first array, the data in the third array is converted into original data, and the specified value includes the maximum value or the minimum value.

[0252] The present application also provides Figure 3 The hardware structure of the device shown. Figure 4 , Figure 4 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 4As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0253] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0254] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0255] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0256] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0257] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0258] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0259] Furthermore, these computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0260] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0261] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A data compression method, characterized in that: The method comprises: Acquire raw data at the same time intervals, and divide the acquired raw data into a plurality of first arrays; For any first array, determining a conversion step size of the first array according to the conversion order and the maximum and minimum values in the first array; wherein the conversion step size is updated according to the maximum allowed precision and / or the minimum allowed precision; Converting the first array into a second array according to the conversion step and a specified value in the first array, wherein the data type in the second array is an integer type, and the specified value includes a maximum value or a minimum value; Convert the second array into a third array according to the set difference type; determining a binary storage bit width for binary storage of the data in the third array according to a maximum value in the third array; According to the binary storage bit width, each data in the third array is converted into a bit stream of equal width, and then converted into a binary byte array; Generate a storage file and store it based on the binary byte array corresponding to the acquired original data.

2. The method according to claim 1, characterized in that The conversion step size is determined by the following formula: S=(Pmax-Pmin) / (C-1) Wherein, Pmax is the maximum value in the first array, Pmin is the minimum value in the first array, and C is the conversion order.

3. The method according to claim 1, characterized in that The updating of the conversion step size according to the maximum allowed precision and / or the minimum allowed precision includes: When the conversion step length is less than the maximum allowable precision, halving the conversion order and recalculating the conversion step length, and comparing the recalculated conversion step length with the maximum allowable precision again, until the conversion step length is greater than or equal to the maximum allowable precision, then determining the conversion step length greater than or equal to the maximum allowable precision as the updated conversion step length; or When the conversion step length is greater than the minimum allowable precision, the conversion order is multiplied and the conversion step length is recalculated, and the recalculated conversion step length is compared with the minimum allowable precision again until the conversion step length is less than or equal to the minimum allowable precision. The conversion step length that is less than or equal to the minimum allowable precision is then determined as the updated conversion step length.

4. The method according to claim 1, wherein The converting the first array into the second array according to the conversion step and the specified value in the first array includes: For any original data P[i] in the first array, the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data: V[i]=(P[i]-Pmin) / S Wherein, Pmin is the minimum value in the first array, and S is the conversion step size.

5. The method according to claim 1, wherein The converting the first array into the second array according to the conversion step and the specified value in the first array includes: For any original data P[i] in the first array, the original data is converted using the following formula, and the converted data is rounded to obtain the integer type data corresponding to the original data: V[i]=(Pmax-P[i]) / S Wherein, Pmax is the maximum value in the first array, and S is the conversion step size.

6. The method according to claim 1, characterized in that The converting the second array into a third array according to the set difference type includes: performing first-order difference processing on the second array to obtain a first-order difference array, and determining the first-order difference array as a target difference array; The target difference array is converted into a third array according to a specified value in the target difference array, wherein the specified value includes a maximum value or a minimum value.

7. The method according to claim 1, characterized in that The converting the second array into a third array according to the set difference type includes: performing first-order difference processing on the second array to obtain a first-order difference array; performing second-order difference processing on the second array to obtain a second-order difference array; Comparing a binary storage bit width determined based on the maximum and minimum values in the first-order difference array with a binary storage bit width determined based on the maximum and minimum values in the second-order difference array to determine a target difference array; The target difference array is converted into a third array according to a specified value in the target difference array, wherein the specified value includes a maximum value or a minimum value.

8. The method according to claim 6 or 7, characterized in that Determining, based on the maximum value in the third array, a binary storage bit width for binary storage of the data in the third array includes: The bit width of the binary value corresponding to the maximum value in the third array is determined as the binary storage bit width for binary storage of the data in the third array.

9. The method according to claim 8, characterized in that The bit width of the binary value corresponding to the maximum value in the third array is determined by the following formula: Wherein, Dmax is the maximum value in the third array, For a floor operation, W is the bit width of the binary data value corresponding to the maximum value in the third array.

10. The method according to claim 9, characterized in that Generating a storage file based on the binary byte array corresponding to the acquired original data includes: Creating a compressed block according to the binary byte array converted from the third array, the compressed block including: a specified value in the first array, the first data in the second array, the minimum value in the target difference array, the difference type, the binary storage bit width, the conversion step size, and the binary byte array, wherein the specified value includes a maximum value or a minimum value; Creating a file header, wherein the file header includes the total number of the original data and the number of the original data in the compressed block; A temporary file is created according to the file header and the compressed block, and the temporary file is compressed into a storage file.

11. The method according to claim 10, characterized in that After compressing the temporary file into a storage file, the method further includes: Decompressing the storage file to obtain the temporary file; Reading the temporary file to obtain the file header and the compressed block; Obtaining a binary storage bit width and a binary byte array in the compressed block, converting each data in the binary byte array into a bit stream of equal width according to the binary storage bit width, and converting the bit stream into the third array; According to the first data of the second array recorded in the compressed block, the differential type, the conversion step, the minimum value in the target differential array, and the specified value in the first array, the data in the third array is converted into original data, and the specified value includes the maximum value or the minimum value.

12. An electronic device, characterized in that: The electronic device includes: one or more processors; A machine-readable storage medium for storing one or more computer-readable instructions, wherein when the one or more computer-readable instructions are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 11.

Images