Power load data compression method and terminal
By analyzing the characteristics of adjacent data differences of power load data, using a data structure with smaller memory when the difference is smaller than the preset value, the problem of low compression rate in the prior art is solved, and efficient compression and fast reading of power load data is achieved.
Patent Information
- Application Number
- CN202110800449.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-15
- Publication Date
- 2025-08-12
- Estimated Expiration
- 2041-07-15
AI Technical Summary
The existing power load data compression method has a low compression rate, which leads to huge storage overhead and long query time, and may even lead to database downtime.
By obtaining the adjacent data difference value in the data set to be compressed, if the difference value is less than the preset value, the difference value is stored using a data structure with a small memory occupancy of the memory, including the first preset data structure and the second preset data structure, respectively, for data whose difference value is less than and greater than the preset value, reducing the storage amount and increasing the compression rate.
Lossless compression is achieved, the amount of data storage is reduced and the compression rate is increased, ensuring the rapid reading and display of data, and avoiding the problem of untimely database response.
Smart Images

Figure CN113742303B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer technology, and in particular relates to a method and a terminal for compressing power load data. Background Art
[0002] Data compression refers to reducing the amount of data or reorganizing data through data compression algorithms without losing useful information, reducing data redundancy to reduce storage space, and improving transmission, storage and processing efficiency.
[0003] In the power industry, important loads need to continuously record power characteristic data such as load curves, voltage, and current. Due to the short user data sampling cycle, large continuous time span, and large user base, the industry usually uses database direct recording, which will result in a huge volume of current power data and huge storage overhead. The query time span should not be too long, otherwise the query time will be too long, and even cause problems such as database downtime.
[0004] To solve the above problem, the power load data is usually compressed before being stored. However, the existing compression method for power load data has a low compression rate. Summary of the Invention
[0005] In view of this, the present invention provides a method and terminal for compressing power load data, which can improve the compression rate of power load data.
[0006] A first aspect of an embodiment of the present invention provides a method for compressing power load data, comprising:
[0007] Obtaining a data set to be compressed, taking first data as reference data for second data, and calculating a difference between the second data and the first data, wherein the first data is a non-empty data in the data set that precedes the second data and is closest to the second data;
[0008] If the difference between the second data and the first data is less than or equal to a first preset value, the difference between the second data and the first data is stored in the position corresponding to the second data in the compressed sequence according to the first preset data structure, wherein the absolute value of the difference is a×10 n The first preset data structure includes a first value, a second value and a third value, wherein the first value is used to indicate whether the difference is positive or negative, and the second value is 10 n The third value is the value of the exponent n in , wherein n is a negative integer.
[0009] In one possible implementation, the first preset data structure also includes type information of the data structure. The first preset data structure consists of two bytes, the first bit is used to represent the type information of the first preset data structure, the second bit is used to represent the first value, the third to fifth bits are used to represent the absolute value of the second value, and the sixth to sixteenth bits are used to represent the third value.
[0010] In a possible implementation, the method further includes:
[0011] If there is third data in the data set and the third data is missing data, a second preset data structure is inserted into the position corresponding to the third data in the compressed sequence, and the second preset structure includes a second preset value, and the second preset value is used to indicate that the third data is missing data.
[0012] In a possible implementation, the method further includes:
[0013] If the first data is the first data in the data set, or the difference between the second data and the first data is greater than the first preset value, the first data or the second data is stored in the compression sequence through a third preset data structure.
[0014] In a possible implementation, the second preset data structure and the third data structure further include data structure type information, wherein the second preset data structure and the third preset data structure are of the same type, and the second preset data structure and the first data structure are of different types.
[0015] In a possible implementation, the second preset data structure consists of one byte, the first bit is used to indicate type information of the second preset data structure, and the second to eighth bits are used to indicate the second preset value;
[0016] The first bit of the third preset data structure is used to represent type information of the third preset data structure, and the second to eighth bits are used to represent a fourth value, and the fourth value is used to represent type information of data stored in the third data structure;
[0017] If the data type information represented by the fourth value is integer data, the third preset data structure is composed of three bytes, and the ninth to twenty-fourth bits are used to store the value of the data;
[0018] If the data type represented by the fourth value is single-precision floating-point data, the third preset data structure is composed of five bytes, and the ninth to fortieth bits are used to store the value of the data;
[0019] If the data type represented by the fourth value is double-precision floating-point data, the third preset data structure is composed of nine bytes, and the ninth to seventy-second bits are used to store the value of the data.
[0020] In a possible implementation, the method further includes performing a data decompression process:
[0021] Get the data sequence to be decompressed;
[0022] According to the first bit of data, obtain the type information of the data structure;
[0023] The data is decompressed according to the type information of the data structure.
[0024] In a possible implementation, decompressing the data according to the type information of the data structure includes:
[0025] If the type information of the data structure is used to represent the second preset data structure or the third preset data structure, the second to eighth bits are read;
[0026] If the values of the second to eighth bits are the second preset values, it is determined that the data corresponding to the data structure is missing data;
[0027] If the values of the second to eighth bits are the fourth value, then obtaining the data stored in the data structure according to the data type information corresponding to the fourth value, and inserting the data into the decompression sequence;
[0028] If the type information of the data structure is used to represent a first preset data structure, the first value, the second value and the third value are obtained, the difference between the second data and the first data is obtained according to the first value, the second value and the third value, the difference is added to the first data to obtain the second data, and the second data is inserted into the decompression sequence.
[0029] In a possible implementation, the method further includes: after obtaining the compressed sequence corresponding to the data set, creating a compressed file containing the compressed sequence;
[0030] Naming the compressed file according to a preset strategy to obtain a file name;
[0031] The identification information of the power load device and the data set, and the directory path information of the compressed file are stored in a database.
[0032] In a second aspect, an embodiment of the present invention provides a terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect or any possible implementation of the first aspect are implemented.
[0033] An embodiment of the present invention provides a method and device for compressing power load data. By analyzing the characteristics of power load curve data, it is found that the difference between two adjacent data is usually small. By compressing the data by storing the difference in a compression sequence, not only the data storage volume is reduced, but also the complexity of the compression algorithm is reduced and the compression rate is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0035] Figure 1 This is a flow chart of an implementation method of a power load data compression method provided by an embodiment of the present invention;
[0036] Figure 2 is a schematic diagram of a data structure provided by an embodiment of the present invention;
[0037] Figure 3 is a schematic diagram of another data structure provided by an embodiment of the present invention;
[0038] Figure 4 is a schematic diagram of another data structure provided by an embodiment of the present invention;
[0039] Figure 5 This is a flowchart of another method for compressing power load data provided by an embodiment of the present invention;
[0040] Figure 6 1 is a schematic structural diagram of a power load data compression device provided by an embodiment of the present invention;
[0041] Figure 7 is a schematic diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0042] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
[0043] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below with reference to the accompanying drawings.
[0044] See also Figure 1 , which shows a flow chart of an implementation method of a power load data compression method provided by an embodiment of the present invention, and is described in detail as follows:
[0045] S101, obtaining a data set to be compressed, taking first data as reference data for second data, and calculating the difference between the second data and the first data, wherein the first data is the non-empty data in the data set that precedes the second data and is closest to the second data.
[0046] The data set to be compressed is data of a preset type within a preset time period corresponding to the same power load device, and the data in the data set to be compressed are arranged in chronological order.
[0047] Optionally, the preset time period may be one month or one year, which is not limited in the embodiment of the present invention.
[0048] The data set includes data of a preset type of an electric load device within a preset time period, that is, single-item collection information of the electric load device within the preset time period, such as total power data, voltage data, or current data.
[0049] The data in the data set are arranged in the order of collection, that is, from early to late.
[0050] There are three forms of binary storage for data: integer data occupies two bytes, or 16 bits; single-precision floating-point data occupies four bytes, or 32 bits; and double-precision floating-point data occupies eight bytes, or 64 bits.
[0051] In actual data storage, data type information must also be stored. Therefore, integer data, single-precision floating-point data, and double-precision floating-point data occupy more than 2 bytes, 4 bytes, and 8 bytes of memory during storage.
[0052] Due to the diversity of power load data, the data set that needs to be compressed usually contains the above three types of data.
[0053] If the power load data is not compressed but directly stored, it can be seen from the above analysis that it will occupy a very large amount of memory.
[0054] Considering that during the data collection process of the power load equipment, the change between two adjacent data is small, that is, the absolute value of the difference between the latter data and the previous data is small.
[0055] Therefore, the first data is used as the reference data of the second data, and the difference between the second data and the first data is calculated.
[0056] It should be noted that the first data is the non-empty data in the data set that is located before the second data and is closest to the second data.
[0057] When there is missing data, the position corresponding to the missing data may be a null data.
[0058] For example, there are 5 data arranged in order in the data set, namely data 1, data 2, missing data, data 4 and data 5. Then the benchmark data of data 2 is data 1, the benchmark data of data 4 is data 2, and the benchmark data of data 5 is data 4.
[0059] S102: If the difference between the second data and the first data is less than or equal to a first preset value, the difference between the second data and the first data is stored in the position corresponding to the second data in the compressed sequence according to the first preset data structure, wherein the absolute value of the difference is a×10 n The first preset data structure includes a first value, a second value and a third value, wherein the first value is used to indicate whether the difference is positive or negative, and the second value is 10 n The third value is the value of the exponent n in , wherein n is a negative integer.
[0060] The embodiment of the present invention provides Figure 2 A data structure shown in Figure 2 The data structure shown is the first preset data structure in this step.
[0061] Optional, such as Figure 2 As shown, the first preset data structure also includes type information of the data structure. The first preset data structure consists of two bytes, the first bit is used to represent the type information of the first preset data structure, the second bit is used to represent the first value, the third to fifth bits are used to represent the absolute value of the second value, and the sixth to sixteenth bits are used to represent the third value.
[0062] like Figure 2The first preset data structure shown only occupies 16 bits, ie, two bytes. Compared with the storage method in step S101 , ie, integer data, single-precision floating-point data, and double-precision floating-point data, its memory usage is greatly reduced.
[0063] If the absolute value of the difference between the second data and the first data is a×10 n The first preset data structure includes a first value, a second value and a third value, the first value is used to indicate whether the difference is positive or negative, and the second value is 10 n The third value is the value of the exponent n in , and the third value is the value of a, where n is a negative integer.
[0064] Combine Figure 2 Optionally, the first bit is used to represent the type information of the first preset data structure. The type information of the first preset data structure can be represented by 0 or 1, and the embodiment of the present invention is not limited to this.
[0065] The second bit is used to represent the first value, and the first value is used to indicate whether the difference between the second data and the first data is a positive number or a negative number. Optionally, when the second bit is 0, the difference is a positive number, and when the second bit is 1, the difference is a negative number; alternatively, when the second bit is 0, the difference is a negative number, and when the second bit is 1, the difference is a positive number. This embodiment of the present invention is not limited to this.
[0066] The third to fifth digits represent the exponent n, which is 0 or a negative integer. Therefore, the value of n can range from -7, -6, -5, -4, -3, -2, -1, to 0.
[0067] The sixth to sixteenth bits, a total of 11 bits, are used to represent the third value, that is, the value of a. It can be seen that the value range of a is any value from 0 to 2047.
[0068] Therefore, the first preset data structure can be used to represent the absolute value in 1×10 -7 Any data between 0 and 2047.
[0069] Based on this, the first preset value can be set to 2047.
[0070] Optionally, if there is third data in the data set and the third data is missing data, a second preset data structure is inserted into the position corresponding to the third data in the compressed sequence, and the second preset structure includes a second preset value, and the second preset value is used to indicate that the third data is missing data.
[0071] If the first data is the first data in the data set, or the difference between the second data and the first data is greater than the first preset value, the first data or the second data is stored in the compression sequence through a third preset data structure.
[0072] The second preset data structure and the third data structure further include data structure type information, wherein the second preset data structure and the third preset data structure are of the same type, and the second preset data structure and the first data structure are of different types.
[0073] Optionally, the second preset data structure consists of one byte, the first bit is used to indicate type information of the second preset data structure, and the second to eighth bits are used to indicate the second preset value;
[0074] The first bit of the third preset data structure is used to represent type information of the third preset data structure, and the second to eighth bits are used to represent a fourth value, and the fourth value is used to represent type information of data stored in the third data structure;
[0075] If the data type information represented by the fourth value is integer data, the third preset data structure is composed of three bytes, and the ninth to twenty-fourth bits are used to store the value of the data;
[0076] If the data type represented by the fourth value is single-precision floating-point data, the third preset data structure is composed of five bytes, and the ninth to fortieth bits are used to store the value of the data;
[0077] If the data type represented by the fourth value is double-precision floating-point data, the third preset data structure is composed of nine bytes, and the ninth to seventy-second bits are used to store the value of the data.
[0078] The following combination Figure 3 and Figure 4 The second preset data structure and the third preset data structure are described respectively.
[0079] Figure 3 A schematic diagram of a data structure provided by an embodiment of the present invention, optionally a schematic diagram of a second preset data structure; Figure 4 A schematic diagram of another data structure provided for an embodiment of the present invention, optionally a schematic diagram of a third preset data structure.
[0080] like Figure 3 and Figure 4 As shown, the first bit of the second preset data structure and the first bit of the third preset data structure are both used to indicate the type information of the data structure, wherein the second preset data structure and the third preset data structure are of the same type, and the second preset data structure and the first data structure are of different types.
[0081] For example, if the type information of the first preset data structure is represented by 1, the type information of the second preset data structure and the third preset data structure is represented by 0; or, if the type information of the first preset data structure is represented by 0, the type information of the second preset data structure and the third preset data structure is represented by 1. This embodiment of the present invention is not limited to this.
[0082] If a piece of data to be compressed is missing data, a second preset data structure is inserted into the position corresponding to the data in the compression sequence, such as Figure 3 As shown, the second to eighth bits of the data structure store a second preset value, and the second preset value is used to indicate that the data is missing data.
[0083] From the above analysis, we can see that the first preset data structure can only store values with absolute values less than or equal to 2047. If the difference between the second data and the first data is greater than 2047, the difference cannot be stored through the first preset data structure. Based on this, for the second data with a difference greater than 2047 from the first data, or the first data in the data set, combined with Figure 4 The embodiment of the present invention also provides a Figure 4 The third preset data structure shown.
[0084] The first bit in the third preset data structure is used to indicate the type information of the data structure, and its type is the same as that of the second preset data structure. The second to eighth bits are used to indicate a fourth value, and the fourth value is used to indicate the type information of the data stored in the third data structure, for example:
[0085] If the data type information represented by the fourth value is integer data, the third preset data structure is composed of three bytes, and the ninth to twenty-fourth bits are used to store the value of the data. In this case, optionally, the fourth value is 24, which can be used to indicate that the data structure occupies 24 bits, a total of 3 bytes;
[0086] If the data type represented by the fourth value is single-precision floating-point data, the third preset data structure is composed of five bytes, and the ninth to fortieth bits are used to store the value of the data. In this case, optionally, the fourth value is 40, which can be used to indicate that the data structure occupies 40 bits, a total of 5 bytes;
[0087] If the data type represented by the fourth value is double-precision floating-point data, the third preset data structure is composed of nine bytes, and the ninth to seventy-second bits are used to store the value of the data. At this time, optionally, the fourth value is 72, which can be used to indicate that the data structure occupies 72 bits, a total of 9 bytes.
[0088] It should be noted that the second preset value is different from the fourth value. For example, the second preset value can be set to 8 to indicate that the data is missing data.
[0089] Furthermore, the second preset value may also be other numerical values used to represent other special types of data that may be encountered in actual applications. The storage of special data based on the concept of the present invention is within the protection scope of the embodiments of the present invention.
[0090] The following is a specific example to illustrate the compression process of the above data.
[0091] For example, there are 5 data arranged in order in the data set, namely data 1, data 2, missing data, data 4 and data 5. Then the benchmark data of data 2 is data 1, the benchmark data of data 4 is data 2, and the benchmark data of data 5 is data 4.
[0092] The absolute value of the difference between data 2 and data 1 is less than 2047, the absolute value of the difference between value 4 and value 2 is greater than 2047, and the absolute value of the difference between value 5 and value 4 is less than 2047.
[0093] Start the compression process and compress the data in the order of the data in the data set. First, compress data 1. Data 1 is the first data in the data set. Figure 4 The third preset data structure shown compresses data 1 and inserts it into a preset compression sequence;
[0094] The absolute value of the difference between data 2 and data 1 is less than 2047, so the difference X between data 2 and data 1 is obtained, and X1 is converted to an absolute value of a×10 n In the form of, respectively, obtain the values of a and n, store X1 in the first preset data structure and insert them into the compressed sequence in sequence;
[0095] If the third data is missing data, the second preset data structure is inserted into the compressed sequence in order;
[0096] If the absolute value of the difference between data 4 and data 2 is greater than 2047, data 2 is stored in the third preset data structure and inserted into the compression sequence in sequence;
[0097] The absolute value of the difference between data 5 and data 4 is less than 2047, and the difference between data 5 and data 4 is X2. The absolute value of X2 is converted to a×10 n , respectively obtain the values of a and n, store X2 using the first preset data structure and insert them into the compressed sequence in sequence.
[0098] Data 5 is the last data in the set, so the compression is completed and the compressed sequence is output.
[0099] Traditional compression methods either directly compress files, which is not conducive to structured access, or are relatively complex and slow to process, resulting in slow system response when reading large amounts of user data. This application reduces data storage overhead, increases compression speed, and enables fast data reading and display.
[0100] From the above, it can be seen that the present invention compresses the same type of power data of the same power load equipment, calculates the difference between the latter data and the previous data based on the characteristic that the difference between two adjacent data is small, and if the difference is less than a first preset value, the difference is stored in a data structure that occupies less memory, thereby achieving lossless compression based on reducing storage capacity and improving the speed of power load data compression.
[0101] Figure 5 A flowchart illustrating another method for compressing power load data provided by an embodiment of the present invention is shown, and is described in detail as follows:
[0102] S501: Obtain a data sequence to be decompressed.
[0103] pass Figure 1 The method described above obtains a compressed sequence. When the compressed sequence needs to be decompressed, the compressed sequence is the data sequence to be decompressed in this step.
[0104] S502: Obtain type information of the data structure according to the first bit of data.
[0105] Depend on Figure 1 The method shown and Figures 2 to 4 It can be seen from the three preset data structures shown that the first bit of each data structure is used to represent the type information of the data structure, wherein the second preset data structure and the third preset data structure are of the same type, and the second preset data structure and the first data structure are of different types.
[0106] S503: Decompress the data according to the type information of the data structure.
[0107] Optionally, if the type information of the data structure is used to represent the second preset data structure or the third preset data structure, the second to eighth bits are read;
[0108] If the values of the second to eighth bits are the second preset values, it is determined that the data corresponding to the data structure is missing data;
[0109] If the values of the second to eighth bits are the fourth value, then obtaining the data stored in the data structure according to the data type information corresponding to the fourth value, and inserting the data into the decompression sequence;
[0110] If the type information of the data structure is used to represent a first preset data structure, the first value, the second value and the third value are obtained, the difference between the second data and the first data is obtained according to the first value, the second value and the third value, the difference is added to the first data to obtain the second data, and the second data is inserted into the decompression sequence.
[0111] The following is an example to illustrate the decompression process.
[0112] In this step, we continue to use the example that there are 5 data arranged in order in the original data set, namely data 1, data 2, missing data, data 4 and data 5. The benchmark data of data 2 is data 1, the benchmark data of data 4 is data 2, and the benchmark data of data 5 is data 4.
[0113] Through steps S101 to S102, a compressed sequence is obtained, which stores in sequence: data 1 stored by the third preset data structure, the difference X1 between data 2 and data 1 stored by the first preset data structure, the missing data stored by the second preset data structure, data 4 stored by the third preset data structure, and the difference X2 between data 5 and data 4 stored by the first preset data structure.
[0114] Assume that the type information of the first preset data structure is represented by 1, and the type information of the second preset data structure and the third preset data structure are represented by 0.
[0115] The decompression process is as follows: read the first bit of the sequence, obtain 0, determine that it is the second preset data structure or the third preset data structure, then read bits 2 to 8 to obtain the type information of data 1. Assuming that data 1 is integer data, the data structure includes 24 bits, read bits 9 to 24 to obtain the value of data 1, and store it in the decompression sequence;
[0116] Continue to read the data, read the first bit, get 1, judge it as the first preset data structure, read the second bit, judge whether the difference is positive or negative, read the third to fifth bits, get the value of n, read the 6th to 16th bits, get the value of a, calculate a×10 n Combined with the value of the second bit, the difference value X1 is obtained, X1 is added to data 1 to obtain data 2, and stored in the decompressed sequence;
[0117] Continue to read data. If the first bit is 0, it is determined to be the second preset data structure or the third preset data structure. Read bits 2-8 to obtain the second preset value. It is determined that the data corresponding to this position is missing data.
[0118] Continue reading data. If the first bit is 0, it is determined to be the second preset data structure or the third preset data structure. Then, bits 2 to 8 are read to obtain type information of data 4. Assuming that data 4 is single-precision floating-point data, the data structure includes 40 bits. Then, bits 9 to 40 are read to obtain the value of data 4 and store it in the decompression sequence.
[0119] Continue to read the data, read the first bit, get 1, judge it as the first preset data structure, read the second bit, judge whether the difference is positive or negative, read the third to fifth bits, get the value of n, read the 6th to 16th bits, get the value of a, calculate a×10 n Combined with the value of the second bit, we get the difference X2, add X2 to data 4 to get data 5, and store it in the decompressed sequence;
[0120] Data reading and decompression are completed.
[0121] Furthermore, after compression is completed, the compressed sequence is stored in a preset database to facilitate subsequent indexing and searching, including: after obtaining the compressed sequence corresponding to the data set, creating a compressed file containing the compressed sequence; naming the compressed file according to a preset strategy to obtain a file name; and storing the identification information of the power load device and the data set, and the directory path information of the compressed file in the database.
[0122] Optionally, the load curve data of a single power load device can be compressed with a natural year as the preset time period to obtain a compressed file. The file name is stored with the user number + the year of the load copy as the file name, and the account name, account number, recording start time point, total number of data points, daily load points, time interval between load records, directory path of the compressed file, etc. of the load are recorded in the database.
[0123] The load curves of all electricity customers are compressed, and the compressed files are stored in the database using the same method as above.
[0124] The embodiment of the present invention also provides an example of a database storage format, as shown below:
[0125] Field type illustrate ID number Data entry unique ID Cons_name Char(64) Account Name Cons_no Char(10) Account Number Start_time datetime Recording start time Sum_num number Total data points Day_piont number Daily load points Sepa_time number Time interval between load records (min) File_path Varchar2 The directory path of the compressed file
[0126] From the above, it can be seen that the present invention compresses the same type of power data of the same power load equipment, calculates the difference between the latter data and the previous data based on the characteristic that the difference between two adjacent data is small, and if the difference is less than the first preset value, the difference is stored in a data structure that occupies less memory, thereby achieving lossless compression based on reducing storage capacity and improving the speed of power load data compression.
[0127] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0128] The following are device embodiments of the present invention. For details not fully described therein, reference may be made to the corresponding method embodiments described above.
[0129] Figure 6 A schematic diagram of the structure of a power load data compression device provided by an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, which are described in detail as follows:
[0130] like Figure 6 As shown, the power load data compression device 6 includes: a data acquisition module 61, a difference calculation module 62 and a compression module 63;
[0131] The data acquisition module 61 is used to: acquire a set of data to be compressed, wherein the set of data to be compressed is data of a preset type within a preset time period corresponding to the same power load device, and the data in the set of data to be compressed are arranged in chronological order;
[0132] The difference calculation module 62 is configured to: use the first data as the reference data of the second data, and calculate the difference between the second data and the first data, wherein the first data is the non-empty data in the data set that is located before the second data and is closest to the second data;
[0133] The compression module 63 is configured to: if the difference between the second data and the first data is less than or equal to a first preset value, store the difference between the second data and the first data in the position corresponding to the second data in the compression sequence according to the first preset data structure, wherein the absolute value of the difference is a×10 n The first preset data structure includes a first value, a second value and a third value, wherein the first value is used to indicate whether the difference is positive or negative, and the second value is 10 n The third value is the value of the index n in , wherein n is 0 or a negative integer.
[0134] Optionally, the first preset data structure also includes type information of the data structure. The first preset data structure consists of two bytes, the first bit is used to represent the type information of the first preset data structure, the second bit is used to represent the first value, the third to fifth bits are used to represent the absolute value of the second value, and the sixth to sixteenth bits are used to represent the third value.
[0135] Optionally, the compression module 63 shown is also used to: if there is third data in the data set and the third data is missing data, then insert a second preset data structure at the position corresponding to the third data in the compression sequence, and the second preset structure includes a second preset value, and the second preset value is used to indicate that the third data is missing data.
[0136] Optionally, the compression module 63 is also used to: if the first data is the first data in the data set, or the difference between the second data and the first data is greater than the first preset value, then store the first data or the second data in the compression sequence through a third preset data structure.
[0137] Optionally, the second preset data structure and the third data structure further include data structure type information, wherein the second preset data structure and the third preset data structure are of the same type, and the second preset data structure and the first data structure are of different types.
[0138] Optionally, the second preset data structure consists of one byte, the first bit is used to indicate type information of the second preset data structure, and the second to eighth bits are used to indicate the second preset value;
[0139] The first bit of the third preset data structure is used to represent type information of the third preset data structure, and the second to eighth bits are used to represent a fourth value, and the fourth value is used to represent type information of data stored in the third data structure;
[0140] If the data type information represented by the fourth value is integer data, the third preset data structure is composed of three bytes, and the ninth to twenty-fourth bits are used to store the value of the data;
[0141] If the data type represented by the fourth value is single-precision floating-point data, the third preset data structure is composed of five bytes, and the ninth to fortieth bits are used to store the value of the data;
[0142] If the data type represented by the fourth value is double-precision floating-point data, the third preset data structure is composed of nine bytes, and the ninth to seventy-second bits are used to store the value of the data.
[0143] Optionally, the device further includes a decompression module 64, configured to obtain a data sequence to be decompressed; obtain type information of a data structure according to the first bit of data; and decompress the data according to the type information of the data structure.
[0144] The decompression module 64 is further configured to: if the type information of the data structure is used to represent the second preset data structure or the third preset data structure, read the second to eighth bits;
[0145] If the values of the second to eighth bits are the second preset values, it is determined that the data corresponding to the data structure is missing data;
[0146] If the values of the second to eighth bits are the fourth value, then obtaining the data stored in the data structure according to the data type information corresponding to the fourth value, and inserting the data into the decompression sequence;
[0147] If the type information of the data structure is used to represent a first preset data structure, the first value, the second value and the third value are obtained, the difference between the second data and the first data is obtained according to the first value, the second value and the third value, the difference is added to the first data to obtain the second data, and the second data is inserted into the decompression sequence.
[0148] Optionally, the device also includes a storage module 65, which is used to create a compressed file containing the compressed sequence after obtaining the compression sequence corresponding to the data set; name the compressed file according to a preset strategy to obtain a file name; and store the identification information of the power load equipment and the data set, and the directory path information of the compressed file in a database.
[0149] From the above, it can be seen that the device of the present invention compresses the same type of power data of the same power load equipment, and calculates the difference between the latter data and the previous data based on the characteristic that the difference between two adjacent data is small. If the difference is less than the first preset value, the difference is stored in a data structure that occupies less memory, thereby achieving lossless compression based on reducing storage capacity and improving the speed of power load data compression.
[0150] Figure 7 Schematic diagram of a terminal provided by an embodiment of the present invention. Figure 7 As shown, the terminal 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, the steps in the above-mentioned various power load data compression method embodiments are implemented, such as Figure 1 Alternatively, when the processor 70 executes the computer program 72, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 6 Functions of the modules / units 61 to 65 are shown.
[0151] Exemplarily, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 72 in the terminal 7. For example, the computer program 72 may be divided into Figure 6 Modules / units 61 to 65 are shown.
[0152] The terminal 7 can be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal 7 can include, but is not limited to, a processor 70 and a memory 71. It can be understood by those skilled in the art that Figure 7 It is only an example of terminal 7 and does not constitute a limitation on terminal 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.
[0153] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.
[0154] The memory 71 may be an internal storage unit of the terminal 7, such as a hard disk or memory of the terminal 7. The memory 71 may also be an external storage device of the terminal 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the terminal 7. Furthermore, the memory 71 may include both an internal storage unit of the terminal 7 and an external storage device. The memory 71 is used to store the computer program and other programs and data required by the terminal. The memory 71 may also be used to temporarily store data that has been output or is about to be output.
[0155] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0156] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.
[0157] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.
[0158] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or units, and can be electrical, mechanical, or other forms.
[0159] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0160] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0161] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the above-mentioned embodiment method by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of the above-mentioned power load data compression method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.
[0162] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.
Claims
1. A method for compressing power load data, characterized in that: include: Obtaining a data set to be compressed, using first data as reference data for second data, and calculating a difference between the second data and the first data, wherein the first data is non-empty data in the data set that precedes the second data and is closest to the second data; the data set to be compressed is data of a preset type within a preset time period corresponding to the same power load device, the data in the data set to be compressed are arranged in chronological order, and the preset type of data is any one of total power data, voltage data, and current data; If the difference between the second data and the first data is less than or equal to the first preset value, the difference between the second data and the first data is stored in the position corresponding to the second data in the compressed sequence according to the first preset data structure, wherein the absolute value of the difference is a×10 n The first preset data structure includes a first value, a second value and a third value, wherein the first value is used to indicate whether the difference is positive or negative, and the second value is 10 n The value of the index n in , the third value is the value of a, wherein n is 0 or a negative integer; The method further comprises: If the first data is the first data in the data set, or the difference between the second data and the first data is greater than the first preset value, storing the first data or the second data in the compressed sequence using a third preset data structure; The first bit of the third preset data structure is used to represent type information of the third preset data structure, and the second to eighth bits are used to represent a fourth value, and the fourth value is used to represent type information of data stored in the third preset data structure; If the data type information represented by the fourth value is integer data, the third preset data structure is composed of three bytes, and the ninth to twenty-fourth bits are used to store the value of the data; If the data type represented by the fourth value is single-precision floating-point data, the third preset data structure is composed of five bytes, and the ninth to fortieth bits are used to store the value of the data; If the data type represented by the fourth value is double-precision floating-point data, the third preset data structure is composed of nine bytes, and the ninth to seventy-second bits are used to store the value of the data.
2. The method according to claim 1, characterized in that The first preset data structure also includes type information of the data structure. The first preset data structure consists of two bytes, the first bit is used to represent the type information of the first preset data structure, the second bit is used to represent the first value, the third to fifth bits are used to represent the absolute value of the second value, and the sixth to sixteenth bits are used to represent the third value.
3. The method according to claim 2, characterized in that The method further includes: If there is third data in the data set and the third data is missing data, a second preset data structure is inserted into the position corresponding to the third data in the compressed sequence, and the second preset structure includes a second preset value, and the second preset value is used to indicate that the third data is missing data.
4. The method according to claim 3, characterized in that The second preset data structure and the third preset data structure further include data structure type information, wherein the second preset data structure and the third preset data structure are of the same type, and the second preset data structure and the first preset data structure are of different types.
5. The method according to claim 4, characterized in that The second preset data structure is composed of one byte, the first bit is used to indicate the type information of the second preset data structure, and the second to eighth bits are used to indicate the second preset value.
6. The method according to claim 5, characterized in that The method further includes performing a data decompression process: Get the data sequence to be decompressed; According to the first bit of data, obtain the type information of the data structure; The data is decompressed according to the type information of the data structure.
7. The method according to claim 6, characterized in that Decompressing the data according to the type information of the data structure includes: If the type information of the data structure is used to represent the second preset data structure or the third preset data structure, the second to eighth bits are read; If the values of the second to eighth bits are the second preset values, it is determined that the data corresponding to the data structure is missing data; If the values of the second to eighth bits are the fourth value, then obtaining the data stored in the data structure according to the data type information corresponding to the fourth value, and inserting the data into the decompression sequence; If the type information of the data structure is used to represent a first preset data structure, the first value, the second value and the third value are obtained, the difference between the second data and the first data is obtained according to the first value, the second value and the third value, the difference is added to the first data to obtain the second data, and the second data is inserted into the decompression sequence.
8. The method according to any one of claims 1 to 7, characterized in that The method further includes: After obtaining the compressed sequence corresponding to the data set, creating a compressed file containing the compressed sequence; Naming the compressed file according to a preset strategy to obtain a file name; The identification information of the power load device and the data set, and the directory path information of the compressed file are stored in a database.
9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.
Citation Information
Patent Citations
Data compression method and device as well as data decompression method and device
CN102103630A
Data cleaning method and device based on Internet of Things platform
CN111538464A