Compression method for reducing storage space of power intelligent terminal and optimizing transmission

Abstract

The invention relates to the technical field of electric power engineering, and discloses a compression method for reducing the storage space of an electric power intelligent terminal and optimizing transmission, which comprises the following steps of: acquiring a measurement request of a data receiving end, generating a first data packet, and generating a second data packet based on the data content of a wave recording file in the first data packet; compressing the first data packet by adopting different compression modes to obtain a second data packet; obtaining the size of the maximum file received by the data receiving end, calculating and judging the size of a second data packet, and splitting the second data packet into a plurality of first sub-data packets when the size of the second data packet is greater than the size of the maximum file received by the data receiving end; sending the compressed second data packets to a data receiving end one by one through a wireless communication network; after the data receiving end receives the second data packet, the second data packet is decompressed based on the label information and stored in the storage unit of the data receiving end, the storage space occupied by the wave recording file is reduced, and the flow consumption in the transmission process is also reduced.

Description

Technical Field

[0001] The present application relates to the field of electric power engineering technology, and in particular to a compression method for reducing storage space and optimizing transmission of electric power intelligent terminals. Background Art

[0002] In the power system, FTU (Feeder Terminal Unit) and other power intelligent terminals will collect a large amount of data, such as current, voltage, power, etc., and generate waveform files. These record files are numerous and take up a large storage space. When calling for test files, due to the large file size, the transmission time is long and the traffic consumption is large. If they are not effectively compressed, it will cause great pressure on storage and transmission.

[0003] Similar prior art includes a Chinese patent application with publication number CN114077589A, which relates to a device for optimizing the scheduling of files sent to an application server at regular time intervals, wherein a first file set (S1) is retrieved from a database within the time interval, the first file set is stored in a priority queue and carries information sent from a sensor linked to an industrial machine, the retrieved first file set is applied to a scheduling algorithm (S2) using a compression ratio and combined with reinforcement learning, so as to select a second file set, the second file set is compressed into a compressed file based on the compression ratio (S3), wherein the size of the compressed file is lower than the size limit, and the compressed file is sent to an application server (S4), wherein the compression ratio is updated based on the size of the second file set and the size of the compressed file, wherein reinforcement learning uses the size of the previous compressed file from the previous time interval. However, the application only considers the compression ratio of the file, and does not consider how to compress in combination with the content of the file, so the compression efficiency of the file is not high.

[0004] Similar prior art also includes a Chinese patent application with publication number CN107332860A, which relates to a data transmission method and storage medium, including: classifying transmission data packets containing the same target identification bit into a classified data packet; reading the data recorded in each data transmission packet in the classified data packet and the data application identification matching the data; forming a transmission packet according to the target identification bit, the data application identification, and the data. Although this application improves the efficiency of output transmission, the file compression efficiency is not high, and the traffic consumption is still too high during the transmission process. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a compression method for reducing the storage space and transmission optimization of power intelligent terminals, aiming to reduce the storage space occupied by recording files and reduce the traffic consumption during the transmission process.

[0006] In the first aspect, the present application provides a compression method for reducing storage space and transmission optimization of an electric power intelligent terminal, wherein an electric power intelligent terminal device is installed at a feeder terminal of a distribution network, and the electric power intelligent terminal device is responsible for collecting feeder current, voltage and power data and generating a recording file stored in a storage unit of the electric power intelligent terminal, and is implemented by executing the following method: Step S1: obtaining a test call request from a data receiving end, generating a first data packet based on the test call request, compressing the first data packet using different compression methods based on the data content of the recording file in the first data packet, and obtaining a compressed second data packet, wherein the first data packet includes one or more recording files corresponding to the test call request, and the test call request includes a maximum file size received by the data receiving end and a maximum data time corresponding to an existing recording file at the data receiving end; Step S2: based on the test request, obtaining a maximum file size received by the data receiving end, calculating and determining a size of the second data packet, and when the size of the second data packet is larger than the maximum file size received by the data receiving end, splitting the second data packet into a plurality of first sub-data packets; Step S3: sending the compressed second data packets one by one to the data receiving end through the wireless communication network, each data packet including label information, the label information including the total number of first sub-data packets in the second data packet and the number of each first sub-data packet; Step S4: After receiving the second data packet, the data receiving end first records the label information corresponding to the second data packet. After all the second data packets are received, the second data packet is decompressed based on the label information and stored in the storage unit of the data receiving end.

[0007] In combination with the first aspect, in a first implementation of the first aspect of the present application, after step S4, the following step is further included: The data receiving end determines whether the second data packet needs to be re-acquired. If it needs to be re-acquired, the call unit sends a call request including label information of the data packet that needs to be re-transmitted to the power intelligent terminal device; After receiving the retest request, the electric power intelligent terminal device sends the compressed data packet corresponding to the tag information to the data receiving end.

[0008] In combination with the first aspect, in a second implementation of the first aspect of the present application, step S2 further includes: When there are multiple recording files in the second data packet, each recording file is taken as a first sub-data packet and re-compressed to obtain multiple compressed first sub-data packets. When the size of the compressed first sub-data packet is still larger than the maximum file size received by the data receiving end, the first sub-data packet is further split and compressed again until the size of the compressed first sub-data packet is less than or equal to the maximum file size received by the data receiving end. When re-compressing the recording file, the same compression method as that used to compress the first data packet is adopted.

[0009] In combination with the first aspect, in a third implementation manner of the first aspect of the present application, continuing to split the first sub-data packet means: Each of the first sub-data packets is first split into two second sub-data packets. If the size of the second sub-data packet is still larger than the maximum file size received by the data receiving end, the second sub-data packet is further split using the same splitting method as that of splitting the second sub-data packet.

[0010] In combination with the first aspect, in a fourth implementation of the first aspect of the present application, in step S1, generating a first data packet includes: All the recording files from the maximum data time to the current time corresponding to the existing recording files at the data receiving end contained in the test call request are obtained from the storage unit of the power intelligent terminal, and all the recording files are packaged to generate the first data packet.

[0011] In combination with the first aspect, in a fifth implementation of the first aspect of the present application, in step S1, compressing the first data packet using different compression methods includes: Based on the file type of the recording file in the first data packet or the content structure of the recording file, determine whether the data in the recording file is statistical data; if so, compress the first data packet using a first compression method; otherwise, obtain part of the content in the recording file and determine whether the part of the content contains a timestamp; if so, compress the first data packet using a second compression method; otherwise, compress the first data packet using a third compression method and obtain a compressed second data packet.

[0012] In combination with the first aspect, in a sixth implementation manner of the first aspect of the present application, compressing the first data packet using a first compression method further includes: Converting the original data sequence in each wave recording file in the first data packet into a binary representation; retaining the i-th original data value in the original data sequence, and calculating an XOR value between the i-th original data value and an adjacent i+1-th original data value as an i+1-th XOR value, where i represents a positive integer greater than or equal to 1; The i-th original data value and all XOR values ​​calculated in sequence are stored as a compressed wave recording file, and a plurality of compressed wave recording files are packaged as a compressed second data packet.

[0013] In combination with the first aspect, in a seventh implementation manner of the first aspect of the present application, compressing the first data packet using a second compression method further includes: Using the first original data value in the original data sequence in each of the wave recording files in the first data packet as a reference data value; Calculate the difference between the i+1th original data value and the ith original data value in the original data sequence as a first difference, and calculate the difference between consecutive first difference values ​​and the next first difference value as a second difference; The reference data value, the first difference value and the second difference value are stored as compressed wave recording files, and a plurality of compressed wave recording files are packaged as a compressed second data packet.

[0014] In combination with the first aspect, in an eighth implementation manner of the first aspect of the present application, compressing the first data packet using a third compression method further includes: Based on a preset cache size, each of the recording files in the first data packet is read in sequence, and based on a preset compression level, a recording dictionary is constructed in the process of reading the recording files, the recording dictionary records specific patterns or phrases appearing in the recording files, and the higher the compression level, the more detailed the recording dictionary is constructed; Preset data length, and read the ith original data value in the original data sequence in the wave recording file based on the preset data length, search the wave recording dictionary for a character string matching the ith original data value, and when matching the ith original data value, record the position and length of the ith original data value in the wave recording dictionary, and use the position and length of the ith original data value in the wave recording dictionary as the content of the compressed wave recording file; Repeat the above steps until the compression of the partial contents of all the recording files is completed, and package the multiple compressed recording files as a compressed second data packet.

[0015] In combination with the first aspect, in a ninth implementation of the first aspect of the present application, the compressed second data packet further includes: The compressed recording files are encapsulated into a second data packet starting with a 32-byte file header and ending with a tail file header. The 32-byte file headers are, in order: 6 bytes are used to identify the file type of the recording file or a specific sequence starting from the file header to ensure that the parser can correctly identify the beginning of the file header, 2 bytes are used to represent the version number of the recording file, the first 4 bytes are used to represent the CRC check value of the file header of the recording file, the first 8 bytes are used to represent the byte offset of the tail file header in the recording file, the second 8 bytes are used to represent the size of the tail file header, and the second 4 bytes are used to represent the CRC check value of the tail file header.

[0016] Compared with the prior art, the beneficial effects of the present invention are at least as follows: In the technical solution provided by the present application, a first data packet is generated based on the call for measurement request by obtaining a call for measurement request from a data receiving end, and the first data packet is compressed using different compression methods based on the data content of the recording file in the first data packet, and a compressed second data packet is obtained; by compressing the recording file, the size of the recording file is reduced, thereby reducing the storage space occupied. Based on the call for measurement request, the maximum file size received by the data receiving end is obtained, the size of the second data packet is calculated and determined, and when the size of the second data packet is greater than the maximum file size received by the data receiving end, the second data packet is split into multiple first sub-data packets; the size of the data packet is made smaller by compression and splitting, thereby reducing the traffic consumption during the transmission process.

[0017] The compressed second data packets are sent to the data receiving end one by one through the wireless communication network. Each data packet includes label information. The label information includes the total number of first sub-data packets in the second data packet and the number of each first sub-data packet. The reliability of data transmission is improved by adding label information, so that the data receiving end can correctly identify and reorganize the data packets. After the data receiving end receives the second data packet, it first records the label information corresponding to the second data packet. After all the second data packets are received, the second data packet is decompressed based on the label information and stored in the storage unit of the data receiving end. The integrity and accuracy of the received data packet are ensured. At the same time, the compression technology is used to reduce the traffic consumption during the transmission process and speed up the data transmission. In the embodiment of the present application, the coordination between the above steps can effectively reduce the storage space occupied by the recording file and reduce the traffic consumption during the transmission process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0019] Figure 1 A schematic diagram of an embodiment of a compression method for reducing storage space and transmission optimization of a power intelligent terminal in an embodiment of the present application; Figure 2 This is a schematic diagram of the content of the tail file header in an embodiment of the present application. DETAILED DESCRIPTION

[0020] The embodiment of the present application provides a compression for reducing the storage space of the power intelligent terminal and optimizing the transmission. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 An embodiment of the compression method for reducing storage space and optimizing transmission of an electric power intelligent terminal in the embodiment of the present application includes installing an electric power intelligent terminal device at a feeder terminal of a distribution network, the electric power intelligent terminal device is responsible for collecting feeder current, voltage and power data and generating a recording file and storing it in a storage unit of the electric power intelligent terminal, and is implemented by executing the following steps: Step S1: obtaining a test call request from a data receiving end, generating a first data packet based on the test call request, compressing the first data packet using different compression methods based on the data content of the recording file in the first data packet, and obtaining a compressed second data packet, wherein the first data packet includes one or more recording files corresponding to the test call request, and the test call request includes a maximum file size received by the data receiving end and a maximum data time corresponding to an existing recording file at the data receiving end; Specifically, in the power system, power intelligent terminals such as FTU (feeder terminal unit) will collect a large amount of data, such as current, voltage, power, etc., and generate record files. In this application, the generated record files are recorded as recording files. There are a large number of recording files, which occupy a large storage space. When transmitting data to the data receiving end, due to the large file size, the transmission time is long and the traffic consumption is large. If it is not effectively compressed, it will cause great pressure on storage and transmission. Therefore, in the above scheme, when the power intelligent terminal device receives the call request from the data receiving end, it first generates a first data packet; then, according to the maximum file size of the data receiving end provided in the call request and the maximum data time of the existing recording file, a plurality of compression methods are used to compress the first data packet to generate a second data packet. The specific compression method will be described in detail below. By compressing the recording file, the size of the recording file is significantly reduced, thereby reducing the storage space occupied.

[0022] Step S2: based on the test request, obtaining the maximum file size received by the data receiving end, calculating and determining the size of the second data packet, and when the size of the second data packet is larger than the maximum file size received by the data receiving end, splitting the second data packet into a plurality of first sub-data packets; Specifically, after the second data packet is generated, the size of the second data packet is calculated according to the maximum file size of the data receiving end in the call test request. If the size of the second data packet exceeds the maximum file size limit of the data receiving end, the second data packet is split into multiple first sub-data packets to ensure that each first sub-data packet can be successfully received by the data receiving end. By compressing and splitting, the size of the data packet is made smaller, the traffic consumption during the transmission process is reduced, and the transmission speed of the data to the receiving end is accelerated.

[0023] Step S3: sending the compressed second data packets one by one to the data receiving end through the wireless communication network, each data packet including label information, the label information including the total number of first sub-data packets in the second data packet and the number of each first sub-data packet; Specifically, the power intelligent terminal device sends the compressed second data packet (or the split first sub-data packet) to the data receiving end through the wireless communication network. Each data packet is accompanied by label information, which includes the number of first sub-data packets into which the second data packet is split and the number of each first sub-data packet. The reliability of data transmission is improved by adding label information, so that the data receiving end can correctly identify and reassemble the data packet.

[0024] Step S4: After receiving the second data packet, the data receiving end first records the label information corresponding to the second data packet. After all the second data packets are received, the second data packet is decompressed based on the label information and stored in the storage unit of the data receiving end.

[0025] Specifically, after receiving the second data packet, the data receiving end first records its label information. After confirming that all first sub-data packets have been received, the data receiving end decompresses the data packet according to the label information and stores it in the local storage unit. This solution ensures the integrity and accuracy of the received data packet, and at the same time reduces the traffic consumption during the transmission process through compression technology, thereby speeding up the data transmission speed.

[0026] In the embodiment of the present application, through the coordination between the above steps, the storage space occupied by the recording file can be effectively reduced and the traffic consumption during the transmission process can be reduced.

[0027] In a specific embodiment, after step S4, the method further includes: The data receiving end determines whether the second data packet needs to be re-acquired. If it needs to be re-acquired, the call unit sends a call request including the label information of the data packet that needs to be re-transmitted to the power intelligent terminal device; After receiving the retest request, the electric power intelligent terminal device sends the compressed data packet corresponding to the tag information to the data receiving end.

[0028] Specifically, after the data receiving end successfully receives and decompresses the second data packet, the receiving terminal will verify the integrity and accuracy of the received second data packet. If it is found that the data packet is erroneous or missing, the data receiving end needs to reacquire the specific data packet. Through the verification mechanism, the integrity and accuracy of the received data packet are ensured, providing additional protection for the reliability of the data. If the data receiving end determines that certain data packets need to be reacquired, the call unit will send a call request containing the label information of the data packet that needs to be retransmitted to the power intelligent terminal device. Through the above steps, the method of the present application can ensure that even if errors or omissions occur during the data transmission process, the integrity and accuracy of the data can be guaranteed through the re-acquisition mechanism, and when acquiring missing data packets, only the missing data packets are acquired, greatly reducing the traffic consumption during the transmission process.

[0029] In a specific embodiment, step S2 further includes: When there are multiple recording files in the second data packet, each recording file is taken as a first sub-data packet and re-compressed to obtain multiple compressed first sub-data packets. When the size of the compressed first sub-data packet is still larger than the maximum file size received by the data receiving end, the first sub-data packet is further split and compressed again until the size of the compressed first sub-data packet is less than or equal to the maximum file size received by the data receiving end. When re-compressing the recording file, the same compression method as that for compressing the first data packet is adopted.

[0030] Specifically, in the case where the second data packet contains multiple recording files, the method of the present application treats each recording file as a first sub-data packet and compresses it, ensuring that each recording file is optimized and compressed to adapt to the maximum file size limit of the data receiving end. If the size of the compressed first sub-data packet is still larger than the maximum file size received by the data receiving end, it is necessary to further split these first sub-data packets and compress the split data again. This process will be repeated until the size of all compressed first sub-data packets is less than or equal to the maximum file size of the data receiving end. Multiple compressions and splittings ensure that the size of the data packet fully meets the transmission requirements and avoid transmission problems caused by excessively large data packets. When re-compressing the recording file, the same compression method as when compressing the first data packet can be used, which can maintain the consistency of the compression algorithm and improve the compression efficiency and compression quality. The above method not only reduces the waste of storage space, but also optimizes the transmission efficiency.

[0031] In a specific embodiment, continuing to split the first sub-data packet means: Each first sub-data packet is first split into two second sub-data packets. If the size of the second sub-data packet is still larger than the maximum file size received by the data receiving end, the second sub-data packet is further split in the same splitting method as the first sub-data packet.

[0032] Specifically, to ensure that the size of the data packet can adapt to the limitations of network transmission, when the size of the compressed first sub-data packet exceeds the maximum file size limit of the data receiving end, each first sub-data packet is evenly split into two smaller second sub-data packets, laying the foundation for further compression and transmission. If the split second sub-data packet is still larger than the maximum file size of the data receiving end, then these second sub-data packets will be split again, and this process can be recursively performed until the size of all data packets meets the transmission requirements. After each split, the split data packet is compressed again to further reduce the size of the first sub-data packet. By ensuring that all data packets are smaller than the maximum file size of the data receiving end, the success rate of data transmission can be significantly improved, and transmission failures caused by excessively large data packets can be reduced. Through multiple splitting and compression, the risk of a single data packet being lost or damaged during transmission can be reduced, thereby improving the efficiency of overall data transmission.

[0033] In a specific embodiment, in step S1, generating a first data packet includes: All the recording files from the maximum data time to the current time corresponding to the existing recording files at the data receiving end contained in the test request are obtained from the storage unit of the power intelligent terminal, and all the recording files are packaged to generate a first data packet.

[0034] Specifically, in step S1, the power intelligent terminal device first retrieves all the recording files within the data time range specified in the call request (i.e., from the maximum data time corresponding to the existing recording files at the data receiving end to the current time) from its storage unit. The integrity and timeliness of the data are ensured, so that the data receiving end can obtain the latest and most complete data. Then all the retrieved recording files are packaged to generate a first data packet, which contains all the data required by the call request and is the original input of the compression process. By packaging, the data can be easily processed uniformly, providing a basis for subsequent compression and transmission.

[0035] In a specific embodiment, in step S1, the first data packet is compressed using different compression methods, including: Based on the file type or content structure of the recording file in the first data packet, determine whether the data in the recording file is statistical data; if so, compress the first data packet using the first compression method; otherwise, obtain part of the content in the recording file and determine whether the part of the content contains a timestamp; if so, compress the first data packet using the second compression method; otherwise, compress the first data packet using the third compression method and obtain the compressed second data packet.

[0036] Specifically, in step S1, the recording file in the first data packet is first analyzed to determine whether the file content is statistical data or other types of data. By identifying the data type, the most appropriate compression strategy can be selected for different types of data, thereby improving the compression efficiency. If the data in the recording file is determined to be statistical data, the first compression method is used for compression, which can more effectively compress the data and reduce redundant information. If the data in the recording file is not statistical data, it is further checked whether the file content contains a timestamp. If it is contained, the second compression method is used for compression, such as time series compression or differential encoding based on timestamps. The second compression method uses timestamp information to effectively compress time series data, especially when the data does not change much. If the data in the recording file is neither statistical data nor a timestamp, the third compression method can be used for compression. The third compression method is a universal compression algorithm that is applicable to various types of data to ensure that all data can be effectively compressed. By using different compression methods for different types of data, the method in the present application can more effectively compress data and reduce the occupancy of storage space; at the same time, the compressed data packet consumes less traffic during transmission, which helps to reduce network congestion and transmission delay and improve the efficiency of data transmission.

[0037] In a specific embodiment, compressing the first data packet using the first compression method also includes: Converting the original data sequence in each wave recording file in the first data packet into a binary representation; The i-th original data value in the original data sequence is retained, and the XOR value between the i-th original data value and the adjacent i+1-th original data value is calculated as the i+1-th XOR value, where i represents a positive integer greater than or equal to 1; The i-th original data value and all XOR values ​​calculated in sequence are stored as a compressed wave recording file, and a plurality of compressed wave recording files are packaged as a compressed second data packet.

[0038] Specifically, in order to facilitate the subsequent XOR operation, the original data sequence of each recording file in the first data packet is converted into a binary representation. The XOR operation is more intuitive and efficient at the binary level. After conversion to binary representation, the i-th data value in the original data sequence is retained, and then the XOR value between the i-th data value and the adjacent i+1-th data value is calculated to obtain the i+1-th XOR value. This process starts from i=1 until the last data value of the sequence. By retaining an original data value and calculating the XOR values ​​of the remaining data values, the entire data sequence can be effectively represented. The XOR operation can capture the changes between the data values, thereby reducing the storage space requirements while maintaining data integrity. At the same time, the compressed data packet has a smaller volume, which reduces the traffic consumption during the transmission process and helps to improve the efficiency and speed of data transmission. The retained i-th original data value and all the calculated XOR values ​​are stored as compressed recording files. Then these compressed recording files are packaged to form a compressed second data packet. This compression method not only reduces the volume of data, but also maintains the recoverability of data. The compressed second data packet can be easily stored and transmitted.

[0039] Assume that the recording file is a set of statistical data, indicating that the maximum power values ​​recorded each day within 4 days are as follows (unit: kW): Day 1: 320 kW, converted to binary: 101000000; Day 2: 325 kW, converted to binary: 101000101; Day 3: 325 kW, converted to binary: 101000101; Day 4: 330 kW, converted to binary: 101001010.

[0040] Perform XOR operation on adjacent data: Day 2 XOR Day 1: 101000101 XOR 101000000 = 101; Day 3 XOR Day 2: 101000101 XOR 101000101 = 0; Day 4 XOR Day 3: 101001010 XOR 101000101 = 1111.

[0041] In this way, instead of storing the four data 101000000, 101000101, 101000101, and 101001010, we only need to store the data of the first day (101000000 and the result of XOR operation (101, 0, and 1111). That is, through XOR operation, the valid information in the data (non-0 The first compression method has a certain degree of reduction, so that the storage space can be used more efficiently in subsequent storage. According to the compression storage rules in the recording file, for the XOR operation result, if the previous data is exactly the same as the current data, the XOR operation result is 0, then only one 0 is written, occupying one bit, indicating that the data has not changed since the last record; if the previous data is different from the current data, the XOR operation result is not 0, then only the part that has changed compared with the previous data is written. For example, if there is a series of statistical data, the XOR result of most adjacent data is 0, or the change is only very small (for example, the binary of 325 is only 101 different from 320), according to this storage rule, the overall amount of stored data will be significantly reduced. The advantage of the first compression method is that the details of the change are recorded only when the data changes, and only the changed part is recorded, which can greatly reduce the use of storage space. This storage method makes it possible to reduce the storage space occupied in many cases compared with directly storing the original data.

[0042] When decompressing, the original data can be reversely restored by using the data of day 1 and the XOR operation result. The first compression method uses the difference between adjacent statistical data to perform compression, which can effectively reduce the storage space required for statistical data sequences with certain correlation.

[0043] In a specific embodiment, compressing the first data packet using the second compression method further includes: Taking the first original data value in the original data sequence in each wave recording file in the first data packet as the reference data value; Calculate the difference between the i+1th original data value and the ith original data value in the original data sequence as the first difference, and calculate the difference between the consecutive first difference and the next first difference as the second difference; The reference data value, the first difference value and the second difference value are stored as compressed wave recording files, and a plurality of compressed wave recording files are packaged as a compressed second data packet.

[0044] Specifically, the present application adopts a variety of compression methods according to the characteristics and requirements of the data in the recording file to achieve the best compression effect. The first compression method may work best when the data changes little or the data points are highly correlated, such as statistical data. However, for data with large changes or high randomness, this compression method may not be effective enough. The second compression method is suitable for situations where the data changes regularly, such as time series data, and can capture the change pattern of the data more finely. For this type of data, the second compression method can provide a better compression ratio. The third compression method is suitable for situations where there are a large number of repeated patterns in the data. By building a dictionary and citing these patterns, the demand for storage space can be significantly reduced. The compression of the third compression method will be described in detail below.

[0045] In the original data sequence of each recording file, the first original data value is selected as the reference data value as the starting point for subsequent difference calculation. By setting a reference point, the absolute value record of the data can be reduced, thereby saving storage space. The difference between the i+1th original data value and the ith original data value in the original data sequence is calculated. This difference is called the first difference. The first difference reflects the change amount of adjacent data points in the data sequence. This change amount is usually smaller than the original data value, so the data can be represented more efficiently. The difference between the continuous first differences is further calculated, that is, the second difference. The introduction of the second difference can capture the change pattern of the data more finely, especially when the data changes regularly. The reference data value, all the calculated first differences and second differences are stored as compressed recording files. These compressed recording files are then packaged to form a compressed second data packet. The second compression method effectively reduces data redundancy and improves compression efficiency through difference calculation. The storage space occupied by the compressed recording file is significantly reduced, which helps the power intelligent terminal equipment to manage storage resources more effectively and reduce traffic consumption during transmission, thereby improving the efficiency and speed of data transmission.

[0046] Assume that the recording file is a set of time data, representing the current values ​​(in amperes) at several consecutive time points: time point 1: 100A, time point 2: 105A, time point 3: 110A, time point 4: 115A, time point 5: 120A.

[0047] First, calculate the first difference of adjacent times: Time point 2 - Time point 1: 105A - 100A = 5A Time point 3 - Time point 2: 110A - 105A = 5A Time point 4 - Time point 3: 115A - 110A = 5A Time point 5 - Time point 4: 120A - 115A = 5A Then, the difference between these first differences is calculated: Because the first difference is always 5A, the difference between the second difference and the first difference is 0 (5 - 5), the difference between the third difference and the second difference is also 0, and so on.

[0048] In this way, the five data points of 100A, 105A, 110A, 115A, and 120A that were originally needed to be stored now only need to store the initial value (for example, 100), the first difference (5), and the result of the second difference (0). Through the second compression method, it can be seen that the amount of data is significantly reduced, especially when the data changes more regularly. The second compression method is particularly suitable for continuous monitoring data in power systems, such as time series data such as current and voltage, which can effectively reduce the demand for storage space and optimize data transmission efficiency.

[0049] When decompressing, starting from the reference data value and adding the first difference, the original data sequence can be restored, such as in the above implementation, time point 1: 100A (reference value), time point 2: 100A + 5A = 105A, time point 3: 105A + 5A = 110A, time point 4: 110A + 5A = 115A, time point 5: 115A + 5A = 120A.

[0050] In a specific embodiment, compressing the first data packet using the third compression method further includes: Based on the preset cache size, each recording file in the first data packet is read in sequence, and based on the preset compression level, a recording dictionary is constructed in the process of reading the recording file, the recording dictionary records specific patterns or phrases appearing in the recording file, and the higher the compression level, the more detailed the recording dictionary is constructed; Preset the data length, and read the i-th original data value in the original data sequence in the recording file based on the preset data length, search for a character string matching the i-th original data value in the recording dictionary, and when the i-th original data value is matched, record the position and length of the i-th original data value in the recording dictionary, and use the position and length of the i-th original data value in the recording dictionary as the content of the compressed recording file; Repeat the above steps until the compression of the partial contents of all the recording files is completed, and package the multiple compressed recording files as a compressed second data packet.

[0051] Specifically, the third compression method achieves more efficient compression by building a dictionary and referencing the position and length in the dictionary instead of storing the complete data value, identifying and utilizing repeated patterns in the data. The compressed data is more compact and easy to store and transmit.

[0052] In a specific embodiment, the compressed second data packet further includes: The compressed recording files are encapsulated into the second data packet starting with a 32-byte file header and ending with a tail file header. The 32-byte file headers are in order: Use 6 bytes to identify the file type of the recording file or a specific sequence at the beginning of the file header to ensure that the parser can correctly identify the beginning of the file header; The version number of the recording file is represented by 2 bytes to ensure compatibility and normal parsing; The first 4 bytes represent the CRC check value of the recording file header, which is used to verify the integrity of the file header and ensure that the data is not damaged during transmission or storage. The first 8 bytes represent the byte offset of the tail file header in the recording file, allowing quick positioning of the tail file header, which is essential for parsing the file structure and extracting information from the recording file. The second 8-byte word indicates the size of the tail file header, which helps determine the end position of the tail file header and the amount of data contained in the tail file header. The second 4 bytes represent the CRC check value of the tail file header, which together with the offset and size of the tail file header ensures the integrity and correctness of the tail file header.

[0053] For the contents of the tail file header, see the attached Figure 2 The tail file header data records some additional information of the recording file (such as the end position of the file, etc.), which is very important for ensuring the correctness and integrity of the file.

[0054] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0055] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0056] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A compression method for reducing storage space and transmission optimization of power intelligent terminals, wherein a power intelligent terminal device is installed at a feeder terminal of a distribution network, and the power intelligent terminal device is responsible for collecting feeder current, voltage and power data and generating a recording file to be stored in a storage unit of the power intelligent terminal, characterized in that: The method comprises: Step S1: obtaining a test call request from a data receiving end, generating a first data packet based on the test call request, compressing the first data packet using different compression methods based on the data content of the recording file in the first data packet, and obtaining a compressed second data packet, wherein the first data packet includes one or more recording files corresponding to the test call request, and the test call request includes a maximum file size received by the data receiving end and a maximum data time corresponding to an existing recording file at the data receiving end; Step S2: based on the test request, obtaining a maximum file size received by the data receiving end, calculating and determining a size of the second data packet, and when the size of the second data packet is larger than the maximum file size received by the data receiving end, splitting the second data packet into a plurality of first sub-data packets; Step S3: sending the compressed second data packets one by one to the data receiving end through the wireless communication network, each data packet including label information, the label information including the total number of first sub-data packets in the second data packet and the number of each first sub-data packet; Step S4: After receiving the second data packet, the data receiving end first records the label information corresponding to the second data packet. After all the second data packets are received, the second data packet is decompressed based on the label information and stored in the storage unit of the data receiving end.

2. The method according to claim 1, characterized in that After step S4, the following steps are also included: The data receiving end determines whether the second data packet needs to be re-acquired. If it needs to be re-acquired, the call unit sends a call request including label information of the data packet that needs to be re-transmitted to the power intelligent terminal device; After receiving the retest request, the electric power intelligent terminal device sends the compressed data packet corresponding to the tag information to the data receiving end.

3. The method according to claim 1, characterized in that The step S2 further comprises: When there are multiple recording files in the second data packet, each recording file is taken as a first sub-data packet and re-compressed to obtain multiple compressed first sub-data packets. When the size of the compressed first sub-data packet is still larger than the maximum file size received by the data receiving end, the first sub-data packet is further split and compressed again until the size of the compressed first sub-data packet is less than or equal to the maximum file size received by the data receiving end. When re-compressing the recording file, the same compression method as that used to compress the first data packet is adopted.

4. The method according to claim 3, characterized in that Continuing to split the first sub-data packet means: Each of the first sub-data packets is first split into two second sub-data packets. If the size of the second sub-data packet is still larger than the maximum file size received by the data receiving end, the second sub-data packet is further split using the same splitting method as that of splitting the second sub-data packet.

5. The method according to claim 1, characterized in that In the step S1, generating the first data packet includes: All the recording files from the maximum data time to the current time corresponding to the existing recording files at the data receiving end contained in the test call request are obtained from the storage unit of the power intelligent terminal, and all the recording files are packaged to generate the first data packet.

6. The method according to claim 1, characterized in that In the step S1, the first data packet is compressed using different compression methods, including: Based on the file type of the recording file in the first data packet or the content structure of the recording file, determine whether the data in the recording file is statistical data; if so, compress the first data packet using a first compression method; otherwise, obtain part of the content in the recording file and determine whether the part of the content contains a timestamp; if so, compress the first data packet using a second compression method; otherwise, compress the first data packet using a third compression method and obtain a compressed second data packet.

7. The method according to claim 6, characterized in that The method further comprises: compressing the first data packet using a first compression method; Converting the original data sequence in each wave recording file in the first data packet into a binary representation; retaining the i-th original data value in the original data sequence, and calculating an XOR value between the i-th original data value and an adjacent i+1-th original data value as an i+1-th XOR value, where i represents a positive integer greater than or equal to 1; The i-th original data value and all XOR values ​​calculated in sequence are stored as a compressed wave recording file, and a plurality of compressed wave recording files are packaged as a compressed second data packet.

8. The method according to claim 7, characterized in that The method further comprises: compressing the first data packet using a second compression method; Using the first original data value in the original data sequence in each of the wave recording files in the first data packet as a reference data value; Calculate the difference between the i+1th original data value and the ith original data value in the original data sequence as a first difference, and calculate the difference between consecutive first difference values ​​and the next first difference value as a second difference; The reference data value, the first difference value and the second difference value are stored as compressed wave recording files, and a plurality of compressed wave recording files are packaged as a compressed second data packet.

9. The method according to claim 7, characterized in that: The method further comprises: compressing the first data packet using a third compression method; Based on a preset cache size, each of the recording files in the first data packet is read in sequence, and based on a preset compression level, a recording dictionary is constructed in the process of reading the recording files, the recording dictionary records specific patterns or phrases appearing in the recording files, and the higher the compression level, the more detailed the recording dictionary is constructed; Preset data length, and read the ith original data value in the original data sequence in the wave recording file based on the preset data length, search the wave recording dictionary for a character string matching the ith original data value, and when matching the ith original data value, record the position and length of the ith original data value in the wave recording dictionary, and use the position and length of the ith original data value in the wave recording dictionary as the content of the compressed wave recording file; Repeat the above steps until the compression of the partial contents of all the recording files is completed, and package the multiple compressed recording files as a compressed second data packet.

10. The method according to claim 6, characterized in that The compressed second data packet also includes: The compressed recording files are encapsulated into a second data packet starting with a 32-byte file header and ending with a tail file header. The 32-byte file headers are, in order: 6 bytes are used to identify the file type of the recording file or a specific sequence starting from the file header, 2 bytes are used to represent the version number of the recording file, the first 4 bytes are used to represent the CRC check value of the file header of the recording file, the first 8 bytes are used to represent the byte offset of the tail file header in the recording file, the second 8 bytes are used to represent the size of the tail file header, and the second 4 bytes are used to represent the CRC check value of the tail file header.

Images