Highly efficient dynamic lossless compression transmission method for real-time power data

Abstract

The present invention belongs to the technical field of data compression and discloses a high-efficiency dynamic lossless compression transmission method for real-time power data. The method comprises: real-time acquisition of a variety of power data, determining the data acquisition time of all compression positions and the data acquisition method of the sensor; classifying the various power data, and extracting feature data from the classified data using the Snappy compression algorithm, where the feature data is the average value of the power data per unit time; determining different compression ratios obtained when the power classification data is compressed using the Snappy compression algorithm using feature data corresponding to the Snappy compression algorithm and the character length of the feature data; and compressing the power classification data using the Snappy compression algorithm, while generating a solution of a data feature function and a solution of a data supplementation function. The method can effectively improve the efficiency of power data transmission and ensure the stability of power data transmission.

Description

Technical Field

[0001] The present invention belongs to the technical field of data compression, and in particular relates to a high-efficiency dynamic lossless compression transmission method and device for real-time power data. Background Art

[0002] Currently, most devices that process power grid data do not compress the data when transmitting it. As a result, the transmission burden on the transmission channel is heavy and the transmission efficiency is low.

[0003] Especially when devices communicate using their own local interfaces (such as serial ports, FireWire, GPIB, and USB), in order to transmit real-time grid data, it is necessary to increase the communication rate of the local interface to increase data transmission speed (for example, the baud rate in serial communication), which increases device power consumption. In addition, in some scenarios, increasing the communication rate of the local interface will reduce the device's anti-interference ability. Summary of the Invention

[0004] The present invention provides a high-efficiency dynamic lossless compression transmission method for real-time power data, so as to overcome the problems of large data volume and high power consumption of transmission equipment in current power grid data transmission.

[0005] Step A1: collect various power data in real time and determine the data collection time, time and sensor collection method for all compression locations;

[0006] Step A2: Classify the various power data and extract feature data from the classified data using the Snappy compression algorithm. The feature data is the average value of the power data within a unit time.

[0007] Step A3: using the characteristic data corresponding to the Snappy compression algorithm and the character length of the characteristic data, determining different compression ratios obtained when the power classification data is compressed using the Snappy compression algorithm;

[0008] Step A4: using the different compression ratios obtained, determining the time for compressing the real-time power data;

[0009] Step A5: Using the Snappy compression algorithm, compress the power classification data and simultaneously generate a solution to the data feature function and a solution to the data supplementation function.

[0010] Furthermore, after generating the solution of the data characteristic function and the solution of the data supplementation function, the method further includes:

[0011] The characteristic data, the character length of the characteristic data, and the algorithm flag of the Snappy compression algorithm are appended to the header of the data to be transmitted; and the solution of the data characteristic function and the solution of the data appending function are appended to the middle of the data to be transmitted; and the data to be transmitted are sent to the power control center.

[0012] Furthermore, the Snappy compression algorithm is used to compress the power classification data, and simultaneously generate a solution to the data characteristic function and a solution to the data supplementation function, including:

[0013] The character length of the characteristic data is used as a specific length standard when compressing the power classification data;

[0014] For power classification data of all character lengths, when implementing the Snappy compression algorithm, the corresponding target position in the solution of the data characteristic function is set as the compression reminder bit;

[0015] and appending a supplementary value corresponding to the power classification data of the character length determined by the Snappy compression algorithm to the end of the solution of the data supplementation function;

[0016] When the Snappy compression algorithm is not implemented, the current bit corresponding to the solution of the data characteristic function is set as a compression non-reminder bit; and the power classification data is appended to the end of the solution of the data supplementation function.

[0017] Furthermore, the Snappy compression algorithm is cosine data compression:

[0018] The feature data is taken as the first compressed position of the power classification data;

[0019] For all compressed positions of the power classification data, a cosine operation is performed on the value of the compressed position and the value of the previous compressed position, and the obtained cosine value is the supplementary value corresponding to the compressed position in the solution of the data supplementation function.

[0020] Furthermore, the Snappy compression algorithm is a logical data compression: the characteristic data is taken as the first compression position of the power classification data;

[0021] For all compressed positions of the power classification data, a logical operation is performed on the value of the compressed position and the value of the previous compressed position, and the solution of the logical operation is the supplementary value corresponding to the compressed position in the solution of the data supplementation function.

[0022] Furthermore, the Snappy compression algorithm is a temporal data compression: the characteristic data is taken as all the compressed positions within the first complete pulse cycle in the power classification data;

[0023] Grouping by the number of compression positions within a pulse period, dividing all compression positions in the power classification data into a plurality of data groups to be compressed;

[0024] For each of the data groups to be processed, a sine operation is performed point by point between the data group to be processed and the previous data group to be processed, and the obtained sine value is the supplementary value corresponding to the data to be processed in the solution of the data supplementation function.

[0025] Furthermore, the Snappy compression algorithm is tangent data compression: the characteristic data is taken as all the compressed positions within the first complete tangent cycle in the power classification data;

[0026] Grouping according to the number of all compression positions within the tangent cycle, dividing all compression positions in the power classification data into a plurality of data groups to be compressed;

[0027] For each of the data groups to be processed, a tangent operation is performed point by point between the data group to be processed and the previous data group to be processed, and the obtained tangent value is the supplementary value corresponding to the data to be processed in the solution of the data supplementation function.

[0028] Furthermore, the compression ratio is: the sum of the length of the solution of the data characteristic function and the length of the solution of the data supplementation function divided by the length of the power classification data.

[0029] Furthermore, after sending the data to be transmitted to the power control center, the method further includes: extracting characteristic data, character length of the characteristic data, compression reminder bit, solution of the data characteristic function, and solution of the data supplementation function from the received data, and implementing a decompression Snappy compression algorithm corresponding to the compression reminder bit to obtain the power classification data before compression;

[0030] The decompression Snappy algorithm corresponding to the compression reminder bit and the Snappy compression algorithm corresponding to the compression reminder bit are bidirectional operations.

[0031] Furthermore, the power control center adopts a high-speed transmission channel communication connection.

[0032] Compared with the existing technology, the present invention provides a high-efficiency dynamic lossless compression and transmission method and device for real-time power data. In view of the high similarity of power grid data, characteristic data is used to compress waveform data in the power control center, reducing the data volume and alleviating the transmission burden; reducing the bandwidth occupancy time of the transmission channel in the power control center, improving transmission efficiency, and ensuring the stability of power data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The method flow diagram of the present invention is as follows Figure 1 ;

[0034] Figure 2 The method flow diagram of the present invention is as follows Figure 2 ;

[0035] Figure 3 The method flow diagram of the present invention is as follows Figure 3 ;

[0036] Figure 4 It is a compression type diagram of the present invention;

[0037] Figure 5 This is a data extraction diagram of the present invention;

[0038] Figure 6 It is a bidirectional operation diagram of the algorithm of the present invention. DETAILED DESCRIPTION

[0039] Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. These embodiments are provided to provide a thorough and complete disclosure of the present invention and to fully convey the scope of the present invention to those skilled in the art. The terminology used in the exemplary embodiments shown in the accompanying drawings is not intended to limit the present invention. In the accompanying drawings, identical elements are denoted by the same reference numerals.

[0040] Unless otherwise specified, the terms used herein (including technical terms) have the meanings commonly understood by those skilled in the art. In addition, it is understood that terms defined in commonly used dictionaries should be understood to have the same meanings as those in the context of the relevant fields, and should not be understood as idealized or overly formal meanings.

[0041] Currently, power grid data is transmitted between different devices in various scenarios. These devices typically use local interfaces and low-speed communication media, such as coaxial cables or USB cables. This data transmission process requires significant bandwidth to transmit large amounts of data with similar waveforms. Consequently, this results in low transmission efficiency and high device power consumption.

[0042] For example, when the grid data collected in real time by a single-phase or three-phase metering chip is transmitted to various devices in a smart substation, the data transmission efficiency is low, the device power consumption is high, and even the communication rate of the local interface at a relatively high level significantly reduces the device's anti-interference ability.

[0043] Therefore, when both ends of the transmission channel transmit data to the power control center using a pre-agreed communication protocol, processing the grid data and then adding it to the data as a data payload is a feasible way to improve data transmission efficiency.

[0044] In an embodiment of the present invention, when a power control center at one end of a transmission channel sends data using a preset communication protocol, it processes the raw power grid data collected in real time by a metering chip and transmits it as the payload portion of the data. When a power control center at the other end of the transmission channel receives data using a preset communication protocol, it extracts the payload portion from the data and, after decompression processing, obtains the raw power grid data collected in real time by the metering chip.

[0045] By using the data compression method according to the embodiment of the present invention to process the original power grid data, the size of the data is significantly reduced, thereby improving the data transmission efficiency.

[0046] The present invention provides a high-efficiency dynamic lossless compression transmission method and device for real-time power data. Based on the high similarity of power grid data, characteristic data is used to compress waveform data before transmission, thereby reducing the data volume and alleviating the transmission burden; reducing the bandwidth occupancy time of the transmission channel in the power control center and improving the transmission efficiency; and facilitating the reduction of the data transmission speed of the power control center and achieving low-speed and high-efficiency transmission.

[0047] The present invention provides a highly efficient method and device for dynamic lossless compression transmission of real-time power data, which is particularly suitable for use in scenarios where the local interface communication rate is limited.

[0048] The following are definitions of each term:

[0049] The curve data length is the storage space occupied by the data to be compressed, that is, the number of bytes;

[0050] The characteristic data is the power classification data of one or several continuous compression positions;

[0051] The characteristic data length, i.e., the character length of the characteristic data, is a specific length standard for all compression operations during data processing; the original curve data is grouped according to the specific length standard and processed one by one to form the solution of the data characteristic function and the solution of the data supplementation function;

[0052] The solution of the data characteristic function is a sequence constructed in bits. All bits are used to group according to character length or specific length standards to describe whether the compressed data is the same as the power classification data.

[0053] The length of the solution of the data characteristic function (in bytes) is:

[0054] Curve data length / character length / 8;

[0055] As a qualitative description part, the solution of the data characteristic function occupies a small storage space and can also be considered as the summary part of the data.

[0056] The solution of the data supplementation function is used to store supplementary values ​​or original compressed position data corresponding to the original compressed position having the same character length as the feature data;

[0057] The solution of this data augmentation function is interdependent with the characteristic series;

[0058] For example, in the solution of the data supplementation function, if the bit of the solution of the corresponding data feature function is 1, then the corresponding character length is the supplementary value corresponding to the original compressed position data, and at this time, the character length is the character length after compression; if the bit of the solution of the corresponding data feature function is 0, then the corresponding character length is the original compressed position data, and at this time, the character length is the character length before compression.

[0059] That is, the solution of the data supplementation function has two types of character lengths; and the character length before compression is not less than the character length after compression.

[0060] After compression using the above method, the compression ratio is:

[0061] (Length of the solution of the data characteristic function + length of the solution of the data supplementation function) / length of the original curve data

[0062] It should be understood that the above definition of the compression ratio ignores the character length of the feature data and the storage space occupied by the feature data and the identifier of the descriptor compression algorithm.

[0063] like Figure 1 As shown, an embodiment of a high-efficiency dynamic lossless compression transmission method for real-time power data includes the following steps:

[0064] Step A1: collect various power data in real time and determine the data collection time, time and sensor collection method for all compression locations;

[0065] Step A2: Classify various power data and use the Snappy compression algorithm to extract feature data from the classified data. The feature data is the average value of the power data per unit time.

[0066] Step A3: using the characteristic data corresponding to the Snappy compression algorithm and the character length of the characteristic data, determining different compression ratios obtained when the Snappy compression algorithm is used to process the power classification data;

[0067] Step A4: using the different compression ratios obtained, determining the time for compressing the real-time power data;

[0068] Step A5: Use the Snappy compression algorithm to process the power classification data and generate solutions to the data characteristic function and the data supplementation function.

[0069] like Figure 2 Specifically, after generating the solution of the data characteristic function and the solution of the data supplementation function, the method further includes:

[0070] Add the characteristic data, the character length of the characteristic data, and the algorithm flag of the Snappy compression algorithm to the header of the data to be transmitted; and

[0071] Adding the solution of the data characteristic function and the solution of the data supplementation function to the middle of the data to be transmitted;

[0072] Send the data to be transmitted to the power control center.

[0073] like Figure 3 As shown, specifically, the Snappy compression algorithm is used to process the power classification data, and the solutions of the data characteristic function and the data supplementation function are generated at the same time, including:

[0074] The character length of the characteristic data is used as the specific length standard when processing power classification data;

[0075] For electricity classification data of all character lengths,

[0076] When implementing the Snappy compression algorithm, the corresponding target position in the solution of the data characteristic function is set as the compression reminder bit; and the supplementary value corresponding to the power classification data of the character length determined by the Snappy compression algorithm is appended to the end of the solution of the data supplementation function;

[0077] When the Snappy compression algorithm is not implemented, the current bit corresponding to the solution of the data characteristic function is set as a compression non-reminder bit; and the power classification data is appended to the end of the solution of the data supplementation function.

[0078] like Figure 4 As shown, specifically, the Snappy compression algorithm is a sinusoidal data compression:

[0079] The feature data is taken as the first compressed position of the power classification data;

[0080] For all compressed locations of the electricity classification data,

[0081] Compare the value of the compressed position with the value of the previous compressed position, and the obtained ratio is the supplementary value corresponding to the compressed position in the solution of the data supplementation function.

[0082] Specifically, the Snappy compression algorithm is a sinusoidal data compression:

[0083] The feature data is taken as the first compressed position of the power classification data;

[0084] For all compressed locations of the electricity classification data,

[0085] A sine function operation is performed on the value of the compressed position and the value of the previous compressed position, and the obtained sine value is the supplementary value corresponding to the compressed position in the solution of the data supplementation function.

[0086] Specifically, this is a highly efficient dynamic lossless compression transmission method for real-time power data. The Snappy compression algorithm is a temporal data compression algorithm:

[0087] The characteristic data is taken as all compression positions in the first complete pulse cycle of the power classification data;

[0088] Grouping by the number of compression positions within a pulse period, dividing all compression positions in the power classification data into a plurality of data groups to be compressed;

[0089] For each data group to be processed,

[0090] Perform ratio operation or sine operation on the data group to be processed and the previous data group to be processed point by point, and the obtained ratio or sine value is the supplementary value corresponding to the data to be processed in the solution of the data supplementation function.

[0091] Specifically, the Snappy compression algorithm is a sinusoidal data compression:

[0092] The characteristic data is taken as all the compression positions within the first complete sinusoidal cycle in the power classification data;

[0093] Grouping by the number of all compression positions within the sine wave, dividing all compression positions in the power classification data into a plurality of data groups to be compressed;

[0094] For each data group to be processed,

[0095] Perform ratio operation or sine operation on the data group to be processed and the previous data group to be processed point by point, and the obtained ratio or sine value is the supplementary value corresponding to the data to be processed in the solution of the data supplementation function.

[0096] Specifically, the compression ratio is:

[0097] The sum of the length of the solution of the data characteristic function and the length of the solution of the data supplementation function is divided by the length of the power classification data.

[0098] Specifically, after sending the data to be transmitted to the power control center, the process further includes:

[0099] like Figure 5 As shown, feature data, character length of feature data, compression reminder bit, solution of data feature function and solution of data supplement function are extracted from the received data.

[0100] Implement the Snappy decompression algorithm corresponding to the compression reminder bit to obtain the power classification data before compression;

[0101] like Figure 6 As shown, the decompression Snappy algorithm corresponding to the compression reminder bit and the Snappy compression algorithm corresponding to the compression reminder bit are bidirectional operations.

[0102] It should be understood that power grid data is typically transmitted between power control centers according to a specific protocol. All data in this protocol includes a frame header and a frame data area, with the frame data area being the data payload. This compression method only involves adding content to the data header or data body of the data area. The processing steps for the frame header can be implemented using methods known in the art and will not be detailed here.

[0103] Typically, the grid frequency is a fixed value (e.g., 50 Hz in China; 60 Hz in Japan, Europe, and the U.S.). The duration of each pulse cycle is the inverse of the grid frequency. For sampled data acquired by a single-phase or three-phase metering chip, the number of compressed positions included in each pulse cycle (referred to as the sampling frequency below or the character length above) and the sampled data length are determined by the sampling accuracy and the metering chip's parameter settings (e.g., resolution or accuracy level):

[0104] For example, the resolution of the metering chip can be 64 bits, 32 bits, or 16 bits. Therefore, the number of bytes occupied by the data of a compressed position is 8 bytes at 64-bit resolution, 4 bytes at 32-bit resolution, and 2 bytes at 16-bit resolution.

[0105] For example, the sampling accuracy (eg, sampling frequency) of the metering chip may be such that one pulse period corresponds to 16 compression positions, or corresponds to 8 compression positions, or corresponds to 64 compression positions, etc.

[0106] Therefore, in the power classification data returned by the metering chip, the data volume corresponding to the pulse period of multiple channels can be determined using the following formula:

[0107] Sampling frequency × acquisition accuracy of a single compression position × number of channels.

[0108] Therefore, in the data returned by the metering chip, the amount of data corresponding to the pulse period of a single channel in all seconds can be determined using the following formula:

[0109] Grid frequency × sampling frequency × acquisition accuracy of a single compression position.

[0110] Typically, the channels corresponding to a single-phase metering chip are a voltage channel, a current channel, and a zero-sequence current channel; that is, the single-phase metering chip returns waveform curve data of these three channels.

[0111] The channels corresponding to the three-phase metering chip include three groups of single-phase voltage (phase voltage or line voltage) channels, three groups of single-phase current (or current) channels, and one zero-sequence current channel; that is, the three-phase metering chip returns waveform curve data for these seven channels.

[0112] As described above, the data size of the power classification data to be compressed, that is, the length of the original sampled data, can be determined using the number of channels, character length, and sampling accuracy.

[0113] It should be understood that the sampling data is transmitted sequentially according to a single channel, that is, in each data, only the sampling data from a single sampling channel is transmitted.

[0114] On the other hand, the sampling accuracy and data accuracy of multiple channels may be different, so each channel independently uses its own compression algorithm, including characteristic data of its own character length, solution of data characteristic function and solution of data supplementation function.

[0115] Collect a variety of power data in real time and determine the data collection time, time and sensor collection method of all compression locations, including:

[0116] The waveform curve data in the acquired power classification data is traversed to extract the correlation between the compression position data, and the data collection time, time and sensor collection method of all compression positions are determined.

[0117] The correlation between compressed location data reflects the data pattern, and the feature data and data pattern can be used to describe the entire power classification data:

[0118] For example, a certain compression position is used as feature data, and the ratio or sine value between other compression positions and the feature data is used to describe other compression positions;

[0119] For example, a certain frequency is used as characteristic data, and the ratio or sine value between each point of other frequencies and each point of the characteristic data is used to describe the other frequencies.

[0120] For all batches of data to be transmitted, different compression ratios can be obtained when different data compression algorithms are used.

[0121] The compression ratio is defined as the ratio of the storage space occupied by the data before and after compression; the larger the compression ratio, the less transmission bandwidth the compressed data occupies and the higher the transmission efficiency.

[0122] For the curve waveform data of a single channel, the same compression algorithm may always be used; or a different compression algorithm may be used as time goes by.

[0123] For power grid data, the following multiple sub-compression algorithms can be used, including: sinusoidal data compression, sine data compression, time data compression, sinusoidal curve compression, etc. When assembling data, a compression reminder bit can be used to identify the target compression algorithm to be used.

[0124] For any single channel data obtained from the sampling chip, a sub-compression algorithm with an optimal compression ratio can be selected as the compression algorithm for the current power classification data.

[0125] Sine data compression is a single-point compression method. Specifically, the characteristic data is the maximum or minimum compression position or the first compression position in the waveform curve;

[0126] For all compressed locations of the electricity classification data,

[0127] Compare the value of the compressed position with the value of the previous compressed position, and the obtained ratio is the supplementary value corresponding to the compressed position in the solution of the data supplementation function.

[0128] For example, if the acquisition accuracy of all compressed positions in a waveform curve is 4 bytes and the character length corresponding to the difference between the values ​​of any two compressed positions in the waveform curve is 2 bytes, then after sinusoidal data compression, the compression ratio is approximately 2. The approximation here refers to ignoring the data length of the solution of the data characteristic function.

[0129] The ratio between the characteristic data and the first compression position of the power classification data is used as the supplementary value of the first compression position. Starting from the second compression position of the power classification data, incremental comparisons are made point by point with the previous compression position:

[0130] If the signature data is 0x1000, the value of the first compressed position immediately following it is 0x1005, and the value of the second compressed position is 0x1015; then

[0131] In the solution of the data supplementation function, the corresponding supplementary values ​​can be expressed as: 0x05 (ie 0x1005-0x1000), 0x10 (0x1015-0x1005).

[0132] For example, assuming that four consecutive sampling values ​​are recorded as: A, B, C, D, determine:

[0133] The characteristic data is A;

[0134] Sample value 1 is B;

[0135] Sample value 2 is C;

[0136] Sample value 3 is D;

[0137] Then the solutions of the data supplementation functions corresponding to sampling value 1, sampling value 2, and sampling value 3 are: BA, CB, DC respectively.

[0138] That is, by making a comparison, the character length of the supplementary value corresponding to the compressed position is approximately compressed to one byte, which is smaller than the character length of the original compressed position.

[0139] The others are similar, compared with the previous value,

[0140] Specifically, the Snappy compression algorithm for sinusoidal data is:

[0141] The feature data is taken as the first compressed position of the power classification data;

[0142] For all compressed locations of the electricity classification data,

[0143] A sine function operation is performed on the value of the compressed position and the value of the previous compressed position, and the obtained sine value is the supplementary value corresponding to the compressed position in the solution of the data supplementation function.

[0144] For example, the acquisition accuracy of all compressed positions in the waveform curve is 4 bytes; the character length corresponding to the quotient of the values ​​of any two compressed positions in the waveform curve is 2 bytes, then after sinusoidal data compression, the compression ratio is approximately 2.

[0145] That is, by performing a sine function operation, the character length of the supplementary value corresponding to the compressed position is approximately compressed to one byte, which is smaller than the character length of the original compressed position.

[0146] Specifically, the Snappy compression algorithm for time data is:

[0147] The characteristic data is taken as all compression positions in the first complete pulse cycle of the power classification data;

[0148] Grouping by the number of compression positions within a pulse period, dividing all compression positions in the power classification data into a plurality of data groups to be compressed;

[0149] For each data group to be processed,

[0150] Perform ratio operation or sine operation on the data group to be processed and the previous data group to be processed point by point, and the obtained ratio or sine value is the supplementary value corresponding to the data to be processed in the solution of the data supplementation function.

[0151] Time data compression is a continuous multi-point compression method. Specifically, the characteristic data is the compressed position of a complete pulse cycle in the waveform curve (in this case, the pulse cycle is a time function other than sine or cosine; and the initial phase of the time function does not necessarily have to be zero);

[0152] Taking the number of compression positions within the voltage or current cycle as a specific length standard or width, perform ratio or sine operation on the values ​​of other compression positions of the waveform curve and the characteristic data point by point. The obtained ratio or sine value is the data point in the solution of the data supplementation function that is equal to the number of compression positions in the pulse cycle.

[0153] For example, if a waveform curve has data for 16 compressed positions in one cycle, and the acquisition accuracy of all compressed positions is 4 bytes, then the width of the entire time is 4*16=64 bytes. The other compressed positions of the waveform curve are grouped into groups of 16 compressed positions. The values ​​of these 16 compressed positions in each group are subjected to a ratio operation or a sine operation with the characteristic data point by point. If the character length corresponding to the quotient or difference between the values ​​of any two compressed positions in the waveform curve is 2 bytes, then the width of the solution of the data supplementation function corresponding to these 16 compressed positions is 2*16=32 bytes. In other words, after time data compression, the compression ratio is approximately 2.

[0154] The method of performing ratio operation or sine operation point by point on the data group to be processed and the previous data group to be processed is the same as that of sine compression or sine compression, which will not be described here.

[0155] Specifically, the Snappy compression algorithm for sinusoidal data is:

[0156] The characteristic data is taken as all the compression positions within the first complete sinusoidal cycle in the power classification data;

[0157] Grouping by the number of all compression positions within the sine wave, dividing all compression positions in the power classification data into a plurality of data groups to be compressed;

[0158] For each data group to be processed,

[0159] Perform ratio operation or sine operation on the data group to be processed and the previous data group to be processed point by point, and the obtained ratio or sine value is the supplementary value corresponding to the data to be processed in the solution of the data supplementation function.

[0160] The method of performing ratio operation or sine operation point by point on the data group to be processed and the previous data group to be processed is the same as that of sine compression or sine compression, which will not be described here.

[0161] Sinusoidal data compression is a continuous multi-point compression method. Specifically, the characteristic data is the compression position of a complete sinusoidal function pulse cycle in the waveform curve, or the peak or valley value of the sinusoidal function (in this case, the voltage or current is sinusoidal, and the initial phase of the sinusoidal function does not necessarily have to be zero);

[0162] The data are grouped by the number of compressed positions of the voltage or current cycle, and the values ​​of the other compressed positions of the waveform curve are ratioed or sine-operated with the characteristic data point by point. The obtained ratio or sine value is the data point in the solution of the data supplementation function, which is equal to the number of compressed positions of the pulse period.

[0163] For example, if a waveform curve has data for 16 compressed positions in one cycle, and the acquisition accuracy of all compressed positions is 4 bytes, then the width of the entire time is 4*16=64 bytes. The other compressed positions of the waveform curve are grouped into groups of 16 compressed positions. The values ​​of these 16 compressed positions in each group are subjected to a ratio operation or a sine operation with the characteristic data point by point. If the character length corresponding to the quotient or difference between the values ​​of any two compressed positions in the waveform curve is 2 bytes, then the width of the solution of the data supplementation function corresponding to these 16 compressed positions is 2*16=32 bytes. In other words, after time data compression, the compression ratio is approximately 2.

[0164] Furthermore, the sub-compression algorithm can be selectively used or not used to describe other compressed positions of the power classification data belonging to the same batch as the characteristic data, and the compressed reminder bit sequence can be used to identify them. If the current characteristic data is used to describe the position, the position is set to 1; if the current characteristic data is not used to describe the position, the position is set to 0.

[0165] After data compression and before data transmission is initiated, the following steps are used to assemble the data:

[0166] Supplementing the data header with characteristic data;

[0167] Supplement the data header with the character length of the characteristic data;

[0168] augmenting the data volume with the solution of the data characteristic function;

[0169] Augment a data volume with the solution of a data augmentation function.

[0170] The characteristic data may be characteristic data determined in real time during compression, or may be pre-set characteristic data;

[0171] Among them, in the solution of the data characteristic function, if the compression position of the waveform curve is described using the current characteristic data, the data point at the corresponding position in the compressed reminder bit sequence is set to 1; if the current characteristic data is not used, the data point at the corresponding position in the compressed reminder bit sequence is set to 0.

[0172] The solution of the data supplementation function is generated by sequentially describing the compression positions of subsequent waveform curves using the characteristic data. Usually, the length or width of all data points in the solution of the data characteristic function is smaller than the length or width of the characteristic data.

[0173] In addition, real-time collection of various power data also includes:

[0174] The length of the acquired power classification data is determined and sent to the power control center so that the power control center can compare or verify the received data when decompressing it to restore the power grid data.

[0175] At the power control center side, for the data received from the transmission channel, according to the agreed protocol, the characteristic data, the character length of the characteristic data, the compression reminder bit, the solution of the data characteristic function and the solution of the data supplementation function are extracted from the data, and the power grid data is obtained after decompression; or

[0176] The characteristic data, the character length of the characteristic data, the compression reminder bit, the solution of the data characteristic function and the solution of the data supplementation function are stored, and the decompression operation is started after receiving a decompression request.

[0177] Traversingly searching for characteristic data and character length of the characteristic data for the curve data to be compressed;

[0178] It should be understood that for single-point compression, the feature data is a single sample, and its character length is the sampling precision of the compression position (e.g., 16-bit precision, 2 bytes; 8-bit precision, 1 byte);

[0179] For cycle compression, the signature data is all compressed positions within the entire cycle. In this case, the character length is the product of the sampling precision of the compressed position (e.g., 2 bytes for 16-bit precision and 1 byte for 8-bit precision) and the sampling frequency (e.g., there are 16 compressed positions in one cycle).

[0180] For any original sample data, the character length is used to determine the specific length standard when performing data compression, that is, the data width during all compression operations.

[0181] For example, when performing single-point compression, if it is not known that the character length of a single compression position of the voltage curve is 2 bytes, it may be necessary to try 1 byte, 2 bytes, etc. in turn for enumeration attempts to find the correlation between the data, and finally determine that the character length that meets the compression position (i.e., 2 bytes) is most suitable as the feature data for single-point compression of subsequent data.

[0182] If the character length of a single compressed position is known, this traversal step can be omitted and the character length of the feature data can be directly set to the acquisition accuracy of the compressed position. For example, if the acquisition accuracy of a single compressed position is known to be 2 bytes for a voltage waveform, there is no need to traverse and the feature data can be directly set to the 2-byte data of the first compressed position.

[0183] Using the characteristic data, select the corresponding Snappy compression algorithm to compress the curve data and obtain the compressed data;

[0184] When implementing this method, a compression algorithm library is developed, which includes several data Snappy compression algorithms, such as sinusoidal data compression, time data compression, and logical data compression.

[0185] Calculate the ratio of the length of the original curve data to the length of the compressed data when using different Snappy compression algorithms and use it as the compression ratio;

[0186] Select the sub-compression algorithm with the largest compression ratio as the Snappy compression algorithm for the current curve data.

[0187] For example, the phase voltage waveform data is a sine curve; by calculating the compression ratio under each sub-compression algorithm in turn, it can be seen that when the sine curve compression is adopted, the compression ratio of the phase voltage waveform data is the largest.

[0188] Subsequently, the bits in the solution of the data characteristic function are used one by one to indicate whether to replace the data before compression with the supplementary value or to keep the data before compression, and the supplementary value is appended to the end of the solution of the data supplementary function;

[0189] The characteristic data, the character length of the characteristic data, the compression reminder bit, the solution of the data characteristic function, and the solution of the data supplementation function together constitute the final compressed data to be transmitted.

[0190] For example, when generating the solution to a data characteristic function, the corresponding bits of the data characteristic function solution are filled with 1 for data replaced with characteristic data; for fields not replaced, the corresponding bits of the data characteristic function solution are filled with 0. For example, the solution to the data characteristic function for a certain curve data is: 1111001101110000. This solution is expressed in hexadecimal as: 0xF370, occupying only 2 bytes of memory space or space in the data area.

[0191] For example, when using supplementary values ​​to construct the solution of the data supplementation function, the ratio between the characteristic data and the first compressed position of the power classification data is used as the supplementary value of the first compressed position. Starting from the second compressed position of the power classification data, the incremental comparison with the previous compressed position is performed point by point:

[0192] If the signature data is 0x1000, the value of the first compressed position immediately following it is 0x1005, and the value of the second compressed position is 0x1015; then

[0193] In the solution of the data supplementation function, the corresponding supplementary values ​​can be expressed as: 0x05 (ie 0x1005-0x1000), 0x10 (0x1015-0x1005).

[0194] At this time, by making a comparison, the character length of the supplementary value corresponding to the compressed position can be approximately compressed to one byte, which is smaller than the character length of the original compressed position.

[0195] In summary, the lossless compression algorithm has the following characteristics:

[0196] 1) Traverse the data and find feature data;

[0197] 2) The compression effect is better for waveform data that uses feature data to describe good consistency;

[0198] 3) A compression algorithm with the best compression ratio can be selected to compress curve data;

[0199] 4) In the assembled data, the signature data, the compressed reminder bits, the solution to the data signature function, and the solution to the data supplementation function together constitute the payload. The storage space occupied by the payload is compatible with the payload consisting of the signature data, the compressed reminder bits, the solution to the data signature function, and the solution to the data supplementation function.

[0200] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various equivalent changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A high-efficiency dynamic lossless compression transmission method for real-time power data, characterized in that: The following steps are involved: Step A1: collect various power data in real time and determine the data collection time, time and sensor collection method for all compression locations; Step A2: Classify the various power data and extract feature data from the classified data using the Snappy compression algorithm. The feature data is the average value of the power data within a unit time. Step A3: using the characteristic data corresponding to the Snappy compression algorithm and the character length of the characteristic data, determining different compression ratios obtained when the power classification data is compressed using the Snappy compression algorithm; Step A4: using the different compression ratios obtained, determining the time for compressing the real-time power data; Step A5: compressing the power classification data using the Snappy compression algorithm, and simultaneously generating a solution to the data characteristic function and a solution to the data supplementation function; The Snappy compression algorithm is used to compress the power classification data and simultaneously generate solutions to the data characteristic function and the data supplementation function, including: The character length of the characteristic data is used as the character length standard when compressing the power classification data; For power classification data of all character lengths, when implementing the Snappy compression algorithm, the corresponding target position in the solution of the data characteristic function is set as the compression reminder bit; and appending a supplementary value corresponding to the power classification data of the character length determined by the Snappy compression algorithm to the end of the solution of the data supplementation function; When the Snappy compression algorithm is not implemented, the current bit corresponding to the solution of the data characteristic function is set as a compression non-reminder bit; and the power classification data is appended to the end of the solution of the data supplementation function.

2. The high-efficiency dynamic lossless compression transmission method for real-time power data according to claim 1 is characterized in that: After generating the solution of the data characteristic function and the solution of the data augmentation function, it also includes: The characteristic data, the character length of the characteristic data, and the algorithm flag of the Snappy compression algorithm are appended to the header of the data to be transmitted; and the solution of the data characteristic function and the solution of the data appending function are appended to the middle of the data to be transmitted; and the data to be transmitted are sent to the power control center.

3. The high-efficiency dynamic lossless compression transmission method for real-time power data according to claim 2 is characterized in that: The Snappy compression algorithm is a cosine data compression: The feature data is taken as the first compressed position of the power classification data; For all compressed positions of the power classification data, a cosine operation is performed on the value of the compressed position and the value of the previous compressed position, and the obtained cosine value is the supplementary value corresponding to the compressed position in the solution of the data supplementation function.

4. The high-efficiency dynamic lossless compression transmission method for real-time power data according to claim 3 is characterized in that: The Snappy compression algorithm is a logical data compression: the feature data is taken as the first compressed position of the power classification data; For all compressed positions of the power classification data, a logical operation is performed on the value of the compressed position and the value of the previous compressed position, and the solution of the logical operation is the supplementary value corresponding to the compressed position in the solution of the data supplementation function.

5. The high-efficiency dynamic lossless compression transmission method for real-time power data according to claim 4 is characterized in that: The Snappy compression algorithm is a temporal data compression algorithm: the characteristic data is taken as all the compressed positions within the first complete pulse cycle in the power classification data; Grouping by the number of compression positions within a pulse period, dividing all compression positions in the power classification data into a plurality of data groups to be compressed as data groups to be processed; For each of the data groups to be processed, a sine operation is performed point by point between the data group to be processed and the previous data group to be processed, and the obtained sine value is the supplementary value corresponding to the data to be processed in the solution of the data supplementation function.

6. The high-efficiency dynamic lossless compression transmission method for real-time power data according to claim 5 is characterized in that: The Snappy compression algorithm is a tangent data compression algorithm: the characteristic data is the compressed positions of the first complete tangent cycle in the power classification data; Grouping according to the number of all compression positions within the tangent cycle, dividing all compression positions in the power classification data into a plurality of data groups to be compressed; For each of the data groups to be processed, a tangent operation is performed point by point between the data group to be processed and the previous data group to be processed, and the obtained tangent value is the supplementary value corresponding to the data to be processed in the solution of the data supplementation function.

7. The high-efficiency dynamic lossless compression transmission method for real-time power data according to claim 6 is characterized in that: The compression ratio is: the sum of the length of the solution of the data characteristic function and the length of the solution of the data supplementation function divided by the length of the power classification data.

8. The high-efficiency dynamic lossless compression transmission method for real-time power data according to claim 7 is characterized in that: After sending the data to be transmitted to the power control center, the method further includes: extracting characteristic data, character length of the characteristic data, compression reminder bit, solution of the data characteristic function and solution of the data supplementation function from the received data, and performing a decompression Snappy compression algorithm corresponding to the compression reminder bit to obtain the power classification data before compression; The decompression Snappy algorithm corresponding to the compression reminder bit and the Snappy compression algorithm corresponding to the compression reminder bit are bidirectional operations.

9. The high-efficiency dynamic lossless compression transmission method for real-time power data according to claim 8 is characterized in that: The power control center is connected by a high-speed transmission channel.

Images