Data compression method and system based on adaptive revolving door algorithm

Abstract

The invention discloses a data compression method and system based on an adaptive revolving door algorithm. The method comprises the following steps: decomposing collected power system data into an effective data component and a noise data component through an empirical mode decomposition algorithm; performing lossless compression on the effective data component through Huffman coding to obtain a lossless compression result; carrying out lossy compression on the noise data components through a preset self-adaptive revolving door algorithm until all data components in the noise data components meet compression requirements, and obtaining a lossy compression result; and obtaining a data compression result based on the lossless compression result and the lossy compression result. The data compression rate and the compression precision of the power system can be maintained at the same time.

Description

Technical Field

[0001] The present application relates to the technical field of data compression, and in particular to a data compression method and system based on an adaptive revolving door algorithm. Background Art

[0002] Power system measurement is crucial to the stable operation of the power system, but this will generate a large amount of data. In order to reliably record all dynamic phenomena, it is necessary to use a higher sampling frequency and wave recording accuracy to continuously record a large amount of power data. When transmitting huge data from field equipment to the dispatching center, it not only takes a long time to occupy the channel, but also easily causes data channel blockage, which makes the storage and transmission of power system data face huge challenges. Therefore, it is necessary to compress large amounts of data by compression.

[0003] Currently, data compression technologies for power systems are divided into lossy compression and lossless compression. Lossy compression reduces the size of data by discarding non-core redundant data during the data compression process. It has low compression accuracy but high compression rate. Lossless compression reduces data volume by eliminating redundant information in the data. Its compression rate is relatively low.

[0004] Application Contents

[0005] The present application provides a data compression method based on an adaptive revolving door algorithm to simultaneously maintain the power system data compression rate and compression accuracy.

[0006] In a first aspect, the present application provides a data compression method based on an adaptive revolving door algorithm, comprising:

[0007] Decomposing the collected power system data into effective data components and noise data components by using the empirical mode decomposition algorithm;

[0008] Performing lossless compression on the effective data component by Huffman coding to obtain a lossless compression result;

[0009] Performing lossy compression on the noise data component by using a preset adaptive revolving door algorithm until all data components in the noise data component meet the compression requirement, thereby obtaining a lossy compression result;

[0010] A data compression result is obtained based on the lossless compression result and the lossy compression result.

[0011] The embodiment of the present application decomposes the collected power system data into effective data components and noise data components through an empirical mode decomposition algorithm, so as to facilitate subsequent compression processing of different categories for the effective data components and the noise data components; the effective data components are losslessly compressed through Huffman coding, so that the effective data components can be losslessly compressed while preserving the information of the effective data components, thereby improving the compression accuracy; the noise data components are lossily compressed through a preset adaptive revolving door algorithm, so that part of the redundant data can be discarded during the data compression process to reduce the size of the data and improve the compression efficiency; finally, based on the lossless compression result and the lossy compression result, a data compression result is obtained, and the lossless compression can be combined with the lossy compression to simultaneously maintain the power system data compression rate and compression accuracy.

[0012] Furthermore, the collected power system data is decomposed into effective data components and noise data components by using an empirical mode decomposition algorithm, specifically:

[0013] Analyze the power system data to obtain the maximum value and the minimum value corresponding to each sampling time point;

[0014] Connecting the plurality of maximum values ​​and the plurality of minimum values ​​corresponding to all sampling time points respectively by using a cubic spline interpolation function to obtain a lower envelope and an upper envelope respectively;

[0015] Compare all power system data corresponding to each sampling time point with the envelope average value to obtain a first data component corresponding to each sampling time point, wherein the envelope average value is the average value of the lower envelope and the upper envelope;

[0016] Based on all power system data corresponding to each sampling time point and the first data component corresponding to each sampling time point, obtain a first residual component corresponding to each sampling time point, and decompose the first residual component to obtain a second data component and a second residual component;

[0017] The data components are compared with preset correlation thresholds respectively, the data components whose data components are greater than the correlation thresholds are taken as valid data components, the data components whose data components are less than or equal to the correlation thresholds are combined with the second residual components to obtain noise data components, wherein the data components include the first data components and the second data components.

[0018] In this way, the collected power system data is decomposed into effective data components and noise data components through the empirical mode decomposition algorithm, which facilitates the subsequent compression processing of different categories for the effective data components and the noise data components.

[0019] Furthermore, the effective data component is losslessly compressed by Huffman coding to obtain a lossless compression result, which is specifically:

[0020] randomly extracting a third data component from the valid data components;

[0021] Calculating the character probability of each character in the third data component, and sorting the character probabilities in descending order;

[0022] Codes are respectively assigned to the two characters with the smallest character probabilities, and the two smallest characters are merged until all characters in the third data component are assigned codes, and a lossless compression result corresponding to the valid data component is output.

[0023] In this way, the effective data components are losslessly compressed through Huffman coding, and the effective data components can be losslessly compressed while preserving the information of the effective data components, thereby improving the compression accuracy.

[0024] Furthermore, the noise data component is lossily compressed by a preset adaptive revolving door algorithm to obtain a lossy compression result, specifically:

[0025] Initialize the parameters of the adaptive revolving door algorithm;

[0026] randomly extracting a fourth data component from the noise data component, calculating a variance of data in the fourth data component, and obtaining an expected error based on the variance;

[0027] Inputting a preset compression deviation into the adaptive revolving door algorithm to obtain an actual decompression error at the current moment, and obtaining a deviation value between the expected error and the actual decompression error;

[0028] Based on the expected error, the actual decompression error and the deviation value, an output value of the adaptive revolving door algorithm at the current moment is obtained, and a parameter of the adaptive revolving door algorithm is adjusted based on the output value to obtain an adjustment parameter;

[0029] All data components in the noise data component are compressed using the adjustment parameters until the compression requirements are met, and a lossy compression result is output.

[0030] In this way, during the lossy compression of the noise data component by the preset adaptive revolving door algorithm, the deviation value between the expected error and the actual decompression error is continuously calculated, and the adaptive revolving door algorithm is continuously adjusted based on the deviation value, which can flexibly adapt to the compression requirements of different data and network bandwidth limitations.

[0031] Further, the method of compressing all data components in the noise data component by adjusting the parameters until the compression requirements are met and outputting a lossy compression result is specifically as follows:

[0032] Based on the output value of the adaptive revolving door algorithm at the current moment, calculating the output deviation value at the current moment;

[0033] The output deviation value is compared with a preset deviation threshold. If the output deviation value is less than or equal to the preset deviation threshold, it means that the compression requirement is met and a lossy compression result is output.

[0034] In this way, by calculating the output deviation value at the current moment and comparing the output deviation value with the preset deviation threshold, it can be determined whether the output compressed data at the current moment meets the compression requirement.

[0035] In a second aspect, the present application provides a data compression system based on an adaptive revolving door algorithm, comprising: a decomposition module, a first compression module, a second compression module and an output module;

[0036] The decomposition module is used to decompose the collected power system data into effective data components and noise data components by using an empirical mode decomposition algorithm;

[0037] The first compression module is used to perform lossless compression on the valid data component through Huffman coding to obtain a lossless compression result;

[0038] The second compression module is used to perform lossy compression on the noise data component by using a preset adaptive revolving door algorithm until all data components in the noise data component meet the compression requirements, thereby obtaining a lossy compression result;

[0039] The output module is used to obtain a data compression result based on the lossless compression result and the lossy compression result.

[0040] The embodiment of the present application decomposes the collected power system data into effective data components and noise data components through an empirical mode decomposition algorithm, so as to facilitate subsequent compression processing of different categories for the effective data components and the noise data components; the effective data components are losslessly compressed through Huffman coding, so that the effective data components can be losslessly compressed while preserving the information of the effective data components, thereby improving the compression accuracy; the noise data components are lossily compressed through a preset adaptive revolving door algorithm, so that part of the redundant data can be discarded during the data compression process to reduce the size of the data and improve the compression efficiency; finally, based on the lossless compression result and the lossy compression result, a data compression result is obtained, and the lossless compression can be combined with the lossy compression to simultaneously maintain the power system data compression rate and compression accuracy.

[0041] Further, the decomposition includes: an analysis unit, a connection unit, a first processing unit, a second processing unit, a third processing unit and a fourth processing unit;

[0042] The analysis unit is used to analyze the power system data to obtain the maximum value and the minimum value corresponding to each sampling time point;

[0043] The connection unit is used to connect the plurality of maximum values ​​and the plurality of minimum values ​​corresponding to all sampling time points respectively through a cubic spline interpolation function to obtain a lower envelope and an upper envelope respectively;

[0044] The first processing unit is used to compare all the power system data corresponding to each sampling time point with the envelope average value to obtain the first data component corresponding to each sampling time point, wherein the envelope average value is the average value of the lower envelope and the upper envelope;

[0045] The second processing unit is used to obtain a first residual component corresponding to each sampling time point based on all power system data corresponding to each sampling time point and the first data component corresponding to each sampling time point, and decompose the first residual component to obtain a second data component and a second residual component;

[0046] The third processing unit is used to compare the data components with preset correlation thresholds respectively, take the data components whose data components are greater than the correlation thresholds as valid data components, and combine the data components whose data components are less than or equal to the correlation thresholds with the second residual components to obtain noise data components, wherein the data components include the first data components and the second data components.

[0047] Furthermore, the first compression module includes: an extraction unit, a sorting unit and an encoding unit;

[0048] The extraction unit is used to randomly extract a third data component from the valid data component;

[0049] The sorting unit is used to calculate the character probability of each character in the third data component, and sort the character probabilities in descending order;

[0050] The encoding unit is used to assign codes to the two characters with the smallest character probabilities respectively, and merge the two smallest characters until all characters in the third data component are assigned codes, and output the lossless compression result corresponding to the valid data component.

[0051] Further, the second compression module includes: an initialization unit, a fourth processing unit, a fifth processing unit, an adjustment unit and a compression unit;

[0052] The initialization unit is used to initialize the parameters of the adaptive revolving door algorithm;

[0053] The fourth processing unit is used to randomly extract a fourth data component from the noise data component, calculate the variance of the data in the fourth data component, and obtain an expected error based on the variance;

[0054] The fifth processing unit is used to input the preset compression deviation into the adaptive revolving door algorithm to obtain the actual decompression error at the current moment, and obtain the deviation value between the expected error and the actual decompression error;

[0055] The adjustment unit is used to obtain an output value of the adaptive revolving door algorithm at a current moment based on the expected error, the actual decompression error and the deviation value, and adjust a parameter of the adaptive revolving door algorithm based on the output value to obtain an adjustment parameter;

[0056] The compression unit is used to compress all data components in the noise data component by using the adjustment parameter until the compression requirement is met, and output a lossy compression result.

[0057] Furthermore, the compression unit is specifically:

[0058] Based on the output value of the adaptive revolving door algorithm at the current moment, calculating the output deviation value at the current moment;

[0059] The output deviation value is compared with a preset deviation threshold. If the output deviation value is less than or equal to the preset deviation threshold, it means that the compression requirement is met and a lossy compression result is output. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flowchart of an embodiment of a data compression method based on an adaptive revolving door algorithm provided by the present application;

[0061] Figure 2 It is a schematic diagram of another embodiment of the data compression method based on the adaptive revolving door algorithm provided by the present application;

[0062] Figure 3 is a schematic diagram of the structure of the adaptive revolving door compression algorithm;

[0063] Figure 4 It is a structural schematic diagram of an embodiment of a data compression system based on an adaptive revolving door algorithm provided by the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] It should be understood that the step numbers used in this article are only for the convenience of description and are not intended to limit the order in which the steps are executed.

[0066] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0067] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0068] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0069] The Empirical Mode Decomposition (EMD) algorithm is a signal decomposition method based on the time scale characteristics of the data itself. It does not require any basis functions to be set in advance, but it decomposes complex signals into a series of finite intrinsic mode functions (IMF) and a residual component in an iterative manner. Each IMF component contains the local characteristic signals of the original signal at different time scales, so that non-stationary data can be processed in a stable manner.

[0070] Power system measurement is crucial to the stable operation of the power system, but it will generate a huge amount of data. In order to fully record dynamic phenomena, it is necessary to continuously record massive data through high sampling frequency and wave recording accuracy. However, when transmitting huge amounts of data to the dispatching center, it is not only time-consuming but also easy to cause data channel blockage, storage and transmission face challenges, so data compression becomes necessary. Existing technologies are divided into lossy and lossless compression: lossy compression requires discarding non-core redundancy, with high compression rate but low accuracy; while lossless compression eliminates redundant information and has a lower compression rate.

[0071] Embodiment 1

[0072] Please refer to Figure 1 , Figure 1 is a flow chart of an embodiment of a data compression method based on an adaptive revolving door algorithm provided by the present application, comprising steps S11 to S14;

[0073] Step S11, decomposing the collected power system data into effective data components and noise data components by using an empirical mode decomposition algorithm;

[0074] It can be understood that, first, the power system data is collected in real time by a data collection device, and the power system data is analyzed to obtain the maximum value and the minimum value corresponding to each sampling time point.

[0075] It should be noted that, when collecting power system data, a sampling time point and a sampling time length may be preset to obtain power system data of a preset time period.

[0076] Secondly, the plurality of maximum values ​​and the plurality of minimum values ​​corresponding to all sampling time points are connected by a cubic spline interpolation function to obtain a lower envelope and an upper envelope respectively; for example, assuming that the minimum value is Minimum The coordinates of (x i ,y i ), where x i yes Position in time series or spatial coordinates, y i It is the minimum point The value of , i = 1, 2, ..., n, uses the cubic spline interpolation function method to connect these minimum points. The basic idea of ​​cubic spline interpolation is to construct a cubic polynomial S(x) between every two adjacent minimum points, satisfying the following conditions:

[0077]

[0078] Among them, the cubic spline interpolation function is usually expressed as:

[0079] S i (x) = a i +b i (xx i )+c i (xx i ) 2 +d i (xx i ) 3 ;

[0080] In the formula, a i , b i , c i , d iis the unknown coefficient, which needs to be solved according to the above conditions. The minimum values ​​can be connected together by solving the cubic spline interpolation function to obtain the upper envelope

[0081] It should be noted that the basic idea of ​​cubic spline interpolation is to construct a cubic polynomial between every two adjacent minimum points, so that all the minimum points can be connected. The maximum points are also connected in the same way to obtain the lower envelope, which is recorded as

[0082] Again, the lower envelope needs to be calculated With the upper envelope The envelope average value is obtained, and all the power system data corresponding to each sampling time point are subtracted from the average value to obtain several first differences, and it is judged whether all the first differences meet the requirements of the intrinsic mode function (IMF) component. If the requirements of the intrinsic mode function component are met, the first difference is the first data component corresponding to each sampling time point. If not, the first difference is further subtracted from the envelope average value to obtain a second difference, and it is judged whether the second difference meets the requirements of the intrinsic mode function component, until all the power system data meet the requirements of the intrinsic mode function component. At this time, the first difference and the second difference are the first data components corresponding to each sampling time point.

[0083] It should be noted that there are two requirements for the intrinsic mode function (IMF) component: (1) the number of extreme points and the number of zero crossing points are equal or the difference does not exceed 1; (2) the upper and lower envelopes formed by the maximum and minimum values ​​are symmetric about the time axis. How to determine whether the requirements of the intrinsic mode function component are met is not the focus of this application, so it will not be expanded here.

[0084] Then, all the power system data corresponding to each sampling time point are subtracted from the first data component corresponding to each sampling time point to obtain the first residual component corresponding to each sampling time point. Since the first residual component still contains the initial data component of a longer period, it is necessary to decompose the first residual component until the power system data is decomposed into data components and residual components to obtain the final second data component and the second residual component; wherein the decomposition step includes obtaining the maximum and minimum points, forming upper and lower envelopes, and obtaining the data components. The specific decomposition process has been detailed above, so it will not be repeated here.

[0085] It should be noted that it is necessary to determine whether the second residual component is a monotonic function. If it becomes a monotonic function, the final residual component is obtained. If it does not become a monotonic function, the second residual component needs to be further decomposed.

[0086] Finally, all data components are compared with the preset correlation threshold, and data components whose data components are greater than the correlation threshold are taken as valid data components, and data components whose data components are less than or equal to the correlation threshold are combined with the second residual component to obtain noise data components, wherein the data components include the first data components and the second data components. The correlation threshold needs to be determined based on the correlation coefficient between the data component and the power system data, and the calculation formula of the correlation coefficient is:

[0087]

[0088] In the formula, is the correlation coefficient between the data component and the power system data, is the average value of the power system data, is the average value of the IMF data component. The calculation formula for determining the correlation threshold λ based on the correlation coefficient is:

[0089]

[0090] In the formula, λ is the correlation threshold, is the correlation coefficient between the data component and the power system data, and t is the corresponding sampling time point.

[0091] In this way, the collected power system data is decomposed into effective data components and noise data components through the empirical mode decomposition algorithm, which facilitates the subsequent compression processing of different categories for the effective data components and the noise data components.

[0092] Step S12, performing lossless compression on the valid data component by Huffman coding to obtain a lossless compression result;

[0093] It can be understood that after the power system data is decomposed into effective data components and noise data components, the effective data components can be losslessly compressed. Specifically: first, randomly extract the third data component from the effective data component; second, calculate the character probability of each character in the third data component, and sort the character probabilities in descending order; finally, assign codes to the two characters with the smallest character probabilities respectively, and merge the two smallest characters until all characters in the third data component are assigned codes, and the lossless compression result corresponding to the effective data component is output. Exemplarily, the third data component IMF is randomly extracted from the effective data component. i, and calculate the character probability of each character in the third data component, the probability is p1, ..., p j ,…,p m , p j For the IMF i The probability of the jth character in , and sort the character probabilities in descending order, find the two characters with the smallest probability, and record the probabilities as The corresponding characters are recorded as x1 and x2, and codes 0 and 1 are assigned to x1 and x2, where the character x1 with a higher probability is assigned code 1, and the character x2 with a lower probability is assigned code 0, and the two characters x1 and x2 are merged into a new character x n , the probability of a new character The probability of the new character is sorted in descending order together with the probability of the remaining characters, and the two characters with the smallest probability are found in the same way, and codes are assigned to them and merged until all characters in the third data component are encoded, and all characters in the valid data component are encoded. Returning along the encoding path, lossless compression encoding of all valid data components, that is, lossless compression result, can be obtained.

[0094] In this way, the effective data components are losslessly compressed through Huffman coding, and the effective data components can be losslessly compressed while preserving the information of the effective data components, thereby improving the compression accuracy.

[0095] Step S13, performing lossy compression on the noise data component by using a preset adaptive revolving door algorithm until all data components in the noise data component meet the compression requirements, thereby obtaining a lossy compression result;

[0096] It can be understood that after the power system data is decomposed into effective data components and noise data components, the noise data components can be lossily compressed. The structural diagram of the adaptive revolving door algorithm is shown in FIG. Figure 3 Specifically, first, the parameters of the adaptive revolving door algorithm are initialized; including initializing the parameters of the neural network PID controller, that is, setting the output value of the neural network PID controller at time zero and the initial value of each historical moment to 0, determining the topological structure of the neural network, and the specific values ​​of the learning rate η, the initial value of the allowable deviation L0, etc. Secondly, randomly extracting a fourth data component from the noise data component, calculating the variance of the data in the fourth data component, and based on the

[0097] The expected error is obtained by using the variance. in The calculation formula of (k) is: in(k) = kσ + b; where σ is the variance of the data in the fourth data component, k and b are constant coefficients, usually k is 0.01, b is 0.00001, and the specific values ​​can be freely set, which is not limited by this application. Again, the preset compression deviation ΔE is input into the adaptive revolving door algorithm to obtain the actual decompression error y at the current moment out (k), and obtain the deviation value error(k) between the expected error and the actual decompression error, wherein the preset compression deviation ΔE is the output of the BP neural network PID controller at the previous moment. Then, based on the expected error r in (k), the actual decompression error y out (k) and the deviation value error(k) are used to calculate each layer of the neural network, and then the three parameters K output by the neural network are P , K I , K D The data is sent to the conventional PID for calculation to obtain the output value u(k) of the adaptive revolving door algorithm at the current moment, and the parameters of the adaptive revolving door algorithm are adjusted based on the output value u(k) to obtain the adjustment parameters. Finally, all data components in the noise data component are compressed using the adjustment parameters u(k) until all data components in the noise data component meet the compression requirements, and the lossy compression result is output.

[0098] In this way, during the lossy compression of the noise data component by the preset adaptive revolving door algorithm, the deviation value between the expected error and the actual decompression error is continuously calculated, and the adaptive revolving door algorithm is continuously adjusted based on the deviation value, which can flexibly adapt to the compression requirements of different data and network bandwidth limitations.

[0099] When judging whether the compression requirements are met and then outputting a lossy compression result, it is necessary to calculate the output deviation value L at the current moment based on the output value u(k) of the adaptive revolving door algorithm at the current moment; compare the output deviation value L with the preset deviation threshold L0. If the output deviation value u(k) is greater than the preset deviation threshold L, it means that the data component at the current moment does not meet the compression requirements and needs to continue to be compressed. If the output deviation value u(k) is less than or equal to the preset deviation threshold L, it means that the data component at the current moment meets the compression requirements and outputs a lossy compression result.

[0100] In this way, by calculating the output deviation value at the current moment and comparing the output deviation value with the preset deviation threshold, it can be determined whether the output compressed data at the current moment meets the compression requirement.

[0101] It should be noted that when one of the data components meets the compression requirements, it is necessary to save the weighted coefficients of the output layer and the hidden layer at the current moment and the previous three moments as the initial values ​​of the system at time zero and each historical moment when the next group of data is compressed; save the output value u of the neural network PID controller at the current moment and the previous five moments as the initial value of the controller output at each moment when the next group of data is compressed, and use the saved parameters of the controller to compress the next group of data until all data components in the noise data component are compressed and the lossy compression result is output.

[0102] It should be noted that the general steps of the standard spiral gate algorithm for lossy compression of a set of noise data components are as follows: First, the compression deviation of the spiral gate algorithm is preset to ΔE, where the time points corresponding to the data to be compressed are t0 to t n . Start from the numerical point corresponding to time point t0 and set A0 and A1 as the starting points of the revolving door algorithm. The distance between these two points and the origin P0 is ΔE. Next, start from the numerical point P1 corresponding to time point t1 and calculate the slope between this point and the two pivot points A0 and A1, which are recorded as S 上 and S 下 , where S 上 is the upward slope, S 下 is a downward slope. When the two slopes S 上 and S 下 When they are equal, S 上 =S 下 , save the data of the point before this time point, and replace the saved point with P0 to continue testing, otherwise continue to the next step. Again, test the slope of the next point with A0 and A1 in turn. If the newly calculated upper slope is Greater than the previously saved upslope Then save the current point and update the upper slope. Similarly, if the newly calculated lower slope Greater than the previously saved upslope The current point is saved and the down slope is updated. When all the numerical points corresponding to all time points have been detected, the algorithm terminates and outputs the final saved point set and up and down slopes. The compression deviation ΔE in the standard spiral door compression algorithm is a fixed value and needs to be set manually. It usually takes many attempts or a certain amount of experience to get a suitable compression deviation ΔE value to meet the required compression effect. The compression effect of the revolving door algorithm is seriously affected by the inability to find a compression deviation value that matches the characteristics of the field data.

[0103] It should be noted that the adaptive revolving door algorithm is based on the standard revolving door algorithm, combined with the feedback control idea, and a control system with compression accuracy as the core is established; at the same time, in order to achieve adaptive adjustment of different data compression processes, a BP-PID controller is designed in combination with the BP neural network to adjust the compression parameters online, and a genetic algorithm is used to optimize the BP-PID controller to further improve the efficiency of the controller.

[0104] Step S14: Obtain a data compression result based on the lossless compression result and the lossy compression result.

[0105] It can be understood that after obtaining the lossless compression result and the lossy compression result, the two results are integrated to obtain the data compression result.

[0106] It should be noted that the two results can be integrated through existing technologies. The result integration process is not the focus of this application and will not be expanded here.

[0107] The embodiment of the present application decomposes the collected power system data into effective data components and noise data components through an empirical mode decomposition algorithm, so as to facilitate subsequent compression processing of different categories for the effective data components and the noise data components; the effective data components are losslessly compressed through Huffman coding, so that the effective data components can be losslessly compressed while preserving the information of the effective data components, thereby improving the compression accuracy; the noise data components are lossily compressed through a preset adaptive revolving door algorithm, so that part of the redundant data can be discarded during the data compression process to reduce the size of the data and improve the compression efficiency; finally, based on the lossless compression result and the lossy compression result, a data compression result is obtained, and the lossless compression can be combined with the lossy compression to simultaneously maintain the power system data compression rate and compression accuracy.

[0108] Embodiment 2

[0109] For easier understanding, please refer to Figure 2 , Figure 2 It is a flowchart of another embodiment of a data compression method based on an adaptive revolving door algorithm provided by the present application;

[0110] Step S21, collecting power system data in real time and inputting it into an empirical mode decomposition (EMD) algorithm;

[0111] Step S22, decomposing the power system data into effective data components and noise data components by using an empirical mode decomposition (EMD) algorithm;

[0112] Step S23, performing lossless compression on the valid data component by Huffman coding to obtain a lossless compression result;

[0113] Step S24, performing lossy compression on the noise data component by using a preset adaptive revolving door algorithm to obtain a lossy compression result;

[0114] Step S25: Obtain a data compression result based on the lossless compression result and the lossy compression result.

[0115] It should be noted that the specific details have been elaborated in detail in the first embodiment and will not be repeated here.

[0116] Embodiment 3

[0117] Please refer to Figure 4 , Figure 4 It is a structural schematic diagram of an embodiment of a data compression system based on an adaptive revolving door algorithm provided by the present application, comprising: a decomposition module 01, a first compression module 02, a second compression module 03 and an output module 04;

[0118] The decomposition module 01 is used to decompose the collected power system data into effective data components and noise data components by using an empirical mode decomposition algorithm;

[0119] Specifically, the decomposition includes: an analysis unit, a connection unit, a first processing unit, a second processing unit and a third processing unit; the analysis unit is used to analyze the power system data to obtain the maximum value and the minimum value corresponding to each sampling time point respectively; the connection unit is used to connect several of the maximum values ​​and several of the minimum values ​​corresponding to all sampling time points respectively through a cubic spline interpolation function to obtain a lower envelope and an upper envelope respectively; the first processing unit is used to compare all the power system data corresponding to each sampling time point with the envelope average value to obtain the first data component corresponding to each sampling time point, wherein the envelope average value is the difference between the lower envelope and the upper envelope. The average value of the envelope; the second processing unit is used to obtain the first residual component corresponding to each sampling time point based on all the power system data corresponding to each sampling time point and the first data component corresponding to each sampling time point, and decompose the first residual component to obtain the second data component and the second residual component; the third processing unit is used to compare the data components with the preset relevant thresholds, take the data components whose data components are greater than the relevant thresholds as the valid data components, and combine the data components whose data components are less than or equal to the relevant thresholds with the second residual component to obtain the noise data components, wherein the data components include the first data components and the second data components.

[0120] It can be understood that, first, the power system data is collected in real time by a data collection device, and the power system data is analyzed to obtain the maximum value and the minimum value corresponding to each sampling time point.

[0121] It should be noted that, when collecting power system data, a sampling time point and a sampling time length may be preset to obtain power system data of a preset time period.

[0122] Secondly, the plurality of maximum values ​​and the plurality of minimum values ​​corresponding to all sampling time points are connected by a cubic spline interpolation function to obtain a lower envelope and an upper envelope respectively; for example, assuming that the minimum value is Minimum The coordinates of (x i ,y i ), where x i yes Position in time series or spatial coordinates, y i It is the minimum point The value of , i = 1, 2, ..., n, uses the cubic spline interpolation function method to connect these minimum points. The basic idea of ​​cubic spline interpolation is to construct a cubic polynomial S(x) between every two adjacent minimum points, satisfying the following conditions:

[0123]

[0124] Among them, the cubic spline interpolation function is usually expressed as:

[0125] S i (x) = a i +b i (xx i )+c i (xx i ) 2 +d i (xx i ) 3 ;

[0126] In the formula, a i , b i , c i , d i is the unknown coefficient, which needs to be solved according to the above conditions. The minimum values ​​can be connected together by solving the cubic spline interpolation function to obtain the upper envelope

[0127] It should be noted that the basic idea of ​​cubic spline interpolation is to construct a cubic polynomial between every two adjacent minimum points, so that all the minimum points can be connected. The maximum points are also connected in the same way to obtain the lower envelope, which is recorded as

[0128] Again, the lower envelope needs to be calculated With the upper envelope The envelope average value is obtained, and all the power system data corresponding to each sampling time point are subtracted from the average value to obtain several first differences, and it is judged whether all the first differences meet the requirements of the intrinsic mode function (IMF) component. If the requirements of the intrinsic mode function component are met, the first difference is the first data component corresponding to each sampling time point. If not, the first difference is further subtracted from the envelope average value to obtain a second difference, and it is judged whether the second difference meets the requirements of the intrinsic mode function component, until all the power system data meet the requirements of the intrinsic mode function component. At this time, the first difference and the second difference are the first data components corresponding to each sampling time point.

[0129] It should be noted that there are two requirements for the intrinsic mode function (IMF) component: (1) the number of extreme points and the number of zero crossing points are equal or the difference does not exceed 1; (2) the upper and lower envelopes formed by the maximum and minimum values ​​are symmetric about the time axis. How to determine whether the requirements of the intrinsic mode function component are met is not the focus of this application, so it will not be expanded here.

[0130] Then, all the power system data corresponding to each sampling time point are subtracted from the first data component corresponding to each sampling time point to obtain the first residual component corresponding to each sampling time point. Since the first residual component still contains the initial data component of a longer period, it is necessary to decompose the first residual component until the power system data is decomposed into data components and residual components to obtain the final second data component and the second residual component; wherein the decomposition step includes obtaining the maximum and minimum points, forming upper and lower envelopes, and obtaining the data components. The specific decomposition process has been detailed above, so it will not be repeated here.

[0131] It should be noted that it is necessary to determine whether the second residual component is a monotonic function. If it becomes a monotonic function, the final residual component is obtained. If it does not become a monotonic function, the second residual component needs to be further decomposed.

[0132] Finally, all data components are compared with the preset correlation threshold, and data components whose data components are greater than the correlation threshold are taken as valid data components, and data components whose data components are less than or equal to the correlation threshold are combined with the second residual component to obtain noise data components, wherein the data components include the first data components and the second data components. The correlation threshold needs to be determined based on the correlation coefficient between the data component and the power system data, and the calculation formula of the correlation coefficient is:

[0133]

[0134] In the formula, is the correlation coefficient between the data component and the power system data, is the average value of the power system data, is the average value of the IMF data component. The calculation formula for determining the correlation threshold λ based on the correlation coefficient is:

[0135]

[0136] In the formula, λ is the correlation threshold, is the correlation coefficient between the data component and the power system data, and t is the corresponding sampling time point.

[0137] In this way, the collected power system data is decomposed into effective data components and noise data components through the empirical mode decomposition algorithm, which facilitates the subsequent compression processing of different categories for the effective data components and the noise data components.

[0138] The first compression module 02 is used to perform lossless compression on the valid data component through Huffman coding to obtain a lossless compression result;

[0139] Specifically, the first compression module 02 includes: an extraction unit, a sorting unit and an encoding unit; the extraction unit is used to randomly extract a third data component from the valid data component; the sorting unit is used to calculate the character probability of each character appearing in the third data component, and sort the character probabilities in descending order; the encoding unit is used to assign codes to the two characters with the smallest character probabilities respectively, and merge the two smallest characters until all characters in the third data component are assigned codes, and output the lossless compression result corresponding to the valid data component.

[0140] It can be understood that after the power system data is decomposed into effective data components and noise data components, the effective data components can be losslessly compressed. Specifically: first, randomly extract the third data component from the effective data component; second, calculate the character probability of each character in the third data component, and sort the character probabilities in descending order; finally, assign codes to the two characters with the smallest character probabilities respectively, and merge the two smallest characters until all characters in the third data component are assigned codes, and the lossless compression result corresponding to the effective data component is output. Exemplarily, the third data component IMF is randomly extracted from the effective data component. i , and calculate the character probability of each character in the third data component, the probability is p1, ..., p j ,…,p m , p j For the IMFi The probability of the jth character in , and sort the character probabilities in descending order, find the two characters with the smallest probability, and record the probabilities as The corresponding characters are recorded as x1 and x2, and codes 0 and 1 are assigned to x1 and x2, where the character x1 with a higher probability is assigned code 1, and the character x2 with a lower probability is assigned code 0, and the two characters x1 and x2 are merged into a new character x n , the probability of a new character The probability of the new character is sorted in descending order together with the probability of the remaining characters, and the two characters with the smallest probability are found in the same way, and codes are assigned to them and merged until all characters in the third data component are encoded, and all characters in the valid data component are encoded. Returning along the encoding path, lossless compression encoding of all valid data components, that is, lossless compression result, can be obtained.

[0141] In this way, the effective data components are losslessly compressed through Huffman coding, and the effective data components can be losslessly compressed while preserving the information of the effective data components, thereby improving the compression accuracy.

[0142] The second compression module 03 is used to perform lossy compression on the noise data component by using a preset adaptive revolving door algorithm until all data components in the noise data component meet the compression requirements, thereby obtaining a lossy compression result;

[0143] Specifically, the second compression module 03 includes: an initialization unit, a fourth processing unit, a fifth processing unit, an adjustment unit and a compression unit; the initialization unit is used to initialize the parameters of the adaptive revolving door algorithm; the fourth processing unit is used to randomly extract a fourth data component from the noise data component, calculate the variance of the data in the fourth data component, and obtain the expected error based on the variance; the fifth processing unit is used to input a preset compression deviation into the adaptive revolving door algorithm to obtain the actual decompression error at the current moment, and obtain the deviation value of the expected error and the actual decompression error; the adjustment unit is used to obtain the output value of the adaptive revolving door algorithm at the current moment based on the expected error, the actual decompression error and the deviation value, and adjust the parameters of the adaptive revolving door algorithm based on the output value to obtain the adjustment parameters; the compression unit is used to compress all data components in the noise data component using the adjustment parameters until the compression requirements are met, and output a lossy compression result.

[0144] It can be understood that after the power system data is decomposed into effective data components and noise data components, the noise data components can be lossily compressed. The structural diagram of the adaptive revolving door algorithm is shown in FIG. Figure 3As shown. Specifically, first, initialize the parameters of the adaptive revolving door algorithm; including initializing the various parameters of the neural network PID controller, that is, setting the output value of the neural network PID controller at time zero and the initial value of each historical moment to 0, determining the topological structure of the neural network, and the specific values ​​of the learning rate η, the initial value of the allowable deviation L0, etc. Secondly, randomly extract a fourth data component from the noise data component, calculate the variance of the data in the fourth data component, and obtain the expected error based on the variance. Wherein, the expected error r in The calculation formula of (k) is: in (k) = kσ + b; where σ is the variance of the data in the fourth data component, k and b are constant coefficients, usually k is 0.01, b is 0.00001, and the specific values ​​can be freely set, which is not limited by this application. Again, the preset compression deviation ΔE is input into the adaptive revolving door algorithm to obtain the actual decompression error y at the current moment out (k), and obtain the deviation value error(k) between the expected error and the actual decompression error, wherein the preset compression deviation ΔE is the output of the BP neural network PID controller at the previous moment. Then, based on the expected error r in (k), the actual decompression error y out (k) and the deviation value error(k) are used to calculate each layer of the neural network, and then the three parameters K output by the neural network are P , K I , K D The data is sent to the conventional PID for calculation to obtain the output value u(k) of the adaptive revolving door algorithm at the current moment, and the parameters of the adaptive revolving door algorithm are adjusted based on the output value u(k) to obtain the adjustment parameters. Finally, all data components in the noise data component are compressed using the adjustment parameters u(k) until all data components in the noise data component meet the compression requirements, and the lossy compression result is output.

[0145] In this way, during the lossy compression of the noise data component by the preset adaptive revolving door algorithm, the deviation value between the expected error and the actual decompression error is continuously calculated, and the adaptive revolving door algorithm is continuously adjusted based on the deviation value, which can flexibly adapt to the compression requirements of different data and network bandwidth limitations.

[0146] Specifically, the compression unit is specifically: based on the output value of the adaptive revolving door algorithm at the current moment, calculate the output deviation value at the current moment; compare the output deviation value with a preset deviation threshold, if the output deviation value is less than or equal to the preset deviation threshold, it means that the compression requirements are met, and output a lossy compression result.

[0147] When judging whether the compression requirements are met and then outputting a lossy compression result, it is necessary to calculate the output deviation value L at the current moment based on the output value u(k) of the adaptive revolving door algorithm at the current moment; compare the output deviation value L with the preset deviation threshold L0. If the output deviation value u(k) is greater than the preset deviation threshold L, it means that the data component at the current moment does not meet the compression requirements and needs to continue to be compressed. If the output deviation value u(k) is less than or equal to the preset deviation threshold L, it means that the data component at the current moment meets the compression requirements and outputs a lossy compression result.

[0148] In this way, by calculating the output deviation value at the current moment and comparing the output deviation value with the preset deviation threshold, it can be determined whether the output compressed data at the current moment meets the compression requirement.

[0149] It should be noted that when one of the data components meets the compression requirements, it is necessary to save the weighted coefficients of the output layer and the hidden layer at the current moment and the previous three moments as the initial values ​​of the system at time zero and each historical moment when the next group of data is compressed; save the output value u of the neural network PID controller at the current moment and the previous five moments as the initial value of the controller output at each moment when the next group of data is compressed, and use the saved parameters of the controller to compress the next group of data until all data components in the noise data component are compressed and the lossy compression result is output.

[0150] It should be noted that the general steps of the standard spiral gate algorithm for lossy compression of a set of noise data components are as follows: First, the compression deviation of the spiral gate algorithm is preset to ΔE, where the time points corresponding to the data to be compressed are t0 to t n . Start from the numerical point corresponding to time point t0 and set A0 and A1 as the starting points of the revolving door algorithm. The distance between these two points and the origin P0 is ΔE. Next, start from the numerical point P1 corresponding to time point t1 and calculate the slope between this point and the two pivot points A0 and A1, which are recorded as S 上 and S 下 , where S 上 is the upward slope, S 下 is a downward slope. When the two slopes S 上 and S 下 When they are equal, S 上 =S 下 , save the data of the point before this time point, and replace the saved point with P0 to continue testing, otherwise continue to the next step. Again, test the slope of the next point with A0 and A1 in turn. If the newly calculated upper slope is Greater than the previously saved upslope Then save the current point and update the upper slope. Similarly, if the newly calculated lower slope Greater than the previously saved upslope The current point is saved and the down slope is updated. When all the numerical points corresponding to all time points have been detected, the algorithm terminates and outputs the final saved point set and up and down slopes. The compression deviation ΔE in the standard spiral door compression algorithm is a fixed value and needs to be set manually. It usually takes many attempts or a certain amount of experience to get a suitable compression deviation ΔE value to meet the required compression effect. The compression effect of the revolving door algorithm is seriously affected by the inability to find a compression deviation value that matches the characteristics of the field data.

[0151] It should be noted that the adaptive revolving door algorithm is based on the standard revolving door algorithm, combined with the feedback control idea, and a control system with compression accuracy as the core is established; at the same time, in order to achieve adaptive adjustment of different data compression processes, a BP-PID controller is designed in combination with the BP neural network to adjust the compression parameters online, and a genetic algorithm is used to optimize the BP-PID controller to further improve the efficiency of the controller.

[0152] The output module 04 is used to obtain a data compression result based on the lossless compression result and the lossy compression result.

[0153] It can be understood that after obtaining the lossless compression result and the lossy compression result, the two results are integrated to obtain the data compression result.

[0154] It should be noted that the two results can be integrated through existing technologies. The result integration process is not the focus of this application and will not be expanded here.

[0155] The embodiment of the present application decomposes the collected power system data into effective data components and noise data components through an empirical mode decomposition algorithm, so as to facilitate subsequent compression processing of different categories for the effective data components and the noise data components; the effective data components are losslessly compressed through Huffman coding, so that the effective data components can be losslessly compressed while preserving the information of the effective data components, thereby improving the compression accuracy; the noise data components are lossily compressed through a preset adaptive revolving door algorithm, so that part of the redundant data can be discarded during the data compression process to reduce the size of the data and improve the compression efficiency; finally, based on the lossless compression result and the lossy compression result, a data compression result is obtained, and the lossless compression can be combined with the lossy compression to simultaneously maintain the power system data compression rate and compression accuracy.

[0156] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the scope of protection of the present application. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A data compression method based on adaptive revolving door algorithm, It is characterized in that include: Decomposing the collected power system data into effective data components and noise data components by using the empirical mode decomposition algorithm; Performing lossless compression on the effective data component by Huffman coding to obtain a lossless compression result; Performing lossy compression on the noise data component by using a preset adaptive revolving door algorithm until all data components in the noise data component meet the compression requirement, thereby obtaining a lossy compression result; A data compression result is obtained based on the lossless compression result and the lossy compression result.

2. The data compression method based on the adaptive revolving door algorithm according to claim 1, It is characterized in that The collected power system data is decomposed into effective data components and noise data components by using the empirical mode decomposition algorithm, specifically: Analyze the power system data to obtain the maximum value and the minimum value corresponding to each sampling time point; Connecting the plurality of maximum values ​​and the plurality of minimum values ​​corresponding to all sampling time points respectively by using a cubic spline interpolation function to obtain a lower envelope and an upper envelope respectively; Compare all power system data corresponding to each sampling time point with the envelope average value to obtain a first data component corresponding to each sampling time point, wherein the envelope average value is the average value of the lower envelope and the upper envelope; Based on all power system data corresponding to each sampling time point and the first data component corresponding to each sampling time point, obtain a first residual component corresponding to each sampling time point, and decompose the first residual component to obtain a second data component and a second residual component; The data components are compared with preset correlation thresholds respectively, the data components whose data components are greater than the correlation thresholds are taken as valid data components, the data components whose data components are less than or equal to the correlation thresholds are combined with the second residual components to obtain noise data components, wherein the data components include the first data components and the second data components.

3. The data compression method based on the adaptive revolving door algorithm according to claim 1, It is characterized in that The effective data component is losslessly compressed by Huffman coding to obtain a lossless compression result, specifically: randomly extracting a third data component from the valid data components; Calculating the character probability of each character in the third data component, and sorting the character probabilities in descending order; Codes are respectively assigned to the two characters with the smallest character probabilities, and the two smallest characters are merged until all characters in the third data component are assigned codes, and a lossless compression result corresponding to the valid data component is output.

4. The data compression method based on the adaptive revolving door algorithm according to claim 1, It is characterized in that The noise data component is lossily compressed by a preset adaptive revolving door algorithm to obtain a lossy compression result, specifically: Initialize the parameters of the adaptive revolving door algorithm; randomly extracting a fourth data component from the noise data component, calculating a variance of data in the fourth data component, and obtaining an expected error based on the variance; Inputting a preset compression deviation into the adaptive revolving door algorithm to obtain an actual decompression error at the current moment, and obtaining a deviation value between the expected error and the actual decompression error; Based on the expected error, the actual decompression error and the deviation value, an output value of the adaptive revolving door algorithm at the current moment is obtained, and a parameter of the adaptive revolving door algorithm is adjusted based on the output value to obtain an adjustment parameter; All data components in the noise data component are compressed using the adjustment parameters until the compression requirements are met, and a lossy compression result is output.

5. The data compression method based on the adaptive revolving door algorithm according to claim 4, It is characterized in that The method of compressing all data components in the noise data component by adjusting the parameters until the compression requirements are met and outputting a lossy compression result is specifically as follows: Based on the output value of the adaptive revolving door algorithm at the current moment, calculating the output deviation value at the current moment; The output deviation value is compared with a preset deviation threshold. If the output deviation value is less than or equal to the preset deviation threshold, it means that the compression requirement is met and a lossy compression result is output.

6. A data compression system based on adaptive revolving door algorithm, It is characterized in that include: A decomposition module, a first compression module, a second compression module and an output module; The decomposition module is used to decompose the collected power system data into effective data components and noise data components by using an empirical mode decomposition algorithm; The first compression module is used to perform lossless compression on the valid data component through Huffman coding to obtain a lossless compression result; The second compression module is used to perform lossy compression on the noise data component by using a preset adaptive revolving door algorithm until all data components in the noise data component meet the compression requirements, thereby obtaining a lossy compression result; The output module is used to obtain a data compression result based on the lossless compression result and the lossy compression result.

7. The data compression system based on the adaptive revolving door algorithm according to claim 6, It is characterized in that The decomposition includes: an analysis unit, a connection unit, a first processing unit, a second processing unit and a third processing unit; The analysis unit is used to analyze the power system data to obtain the maximum value and the minimum value corresponding to each sampling time point; The connection unit is used to connect the plurality of maximum values ​​and the plurality of minimum values ​​corresponding to all sampling time points respectively through a cubic spline interpolation function to obtain a lower envelope and an upper envelope respectively; The first processing unit is used to compare all the power system data corresponding to each sampling time point with the envelope average value to obtain the first data component corresponding to each sampling time point, wherein the envelope average value is the average value of the lower envelope and the upper envelope; The second processing unit is used to obtain a first residual component corresponding to each sampling time point based on all power system data corresponding to each sampling time point and the first data component corresponding to each sampling time point, and decompose the first residual component to obtain a second data component and a second residual component; The third processing unit is used to compare the data components with preset correlation thresholds respectively, take the data components whose data components are greater than the correlation thresholds as valid data components, and combine the data components whose data components are less than or equal to the correlation thresholds with the second residual components to obtain noise data components, wherein the data components include the first data components and the second data components.

8. The data compression system based on the adaptive revolving door algorithm according to claim 6, It is characterized in that The first compression module includes: an extraction unit, a sorting unit and an encoding unit; The extraction unit is used to randomly extract a third data component from the valid data component; The sorting unit is used to calculate the character probability of each character in the third data component, and sort the character probabilities in descending order; The encoding unit is used to assign codes to the two characters with the smallest character probabilities respectively, and merge the two smallest characters until all characters in the third data component are assigned codes, and output the lossless compression result corresponding to the valid data component.

9. The data compression system based on the adaptive revolving door algorithm according to claim 6, It is characterized in that The second compression module includes: an initialization unit, a fourth processing unit, a fifth processing unit, an adjustment unit and a compression unit; The initialization unit is used to initialize the parameters of the adaptive revolving door algorithm; The fourth processing unit is used to randomly extract a fourth data component from the noise data component, calculate the variance of the data in the fourth data component, and obtain an expected error based on the variance; The fifth processing unit is used to input the preset compression deviation into the adaptive revolving door algorithm to obtain the actual decompression error at the current moment, and obtain the deviation value between the expected error and the actual decompression error; The adjustment unit is used to obtain an output value of the adaptive revolving door algorithm at a current moment based on the expected error, the actual decompression error and the deviation value, and adjust a parameter of the adaptive revolving door algorithm based on the output value to obtain an adjustment parameter; The compression unit is used to compress all data components in the noise data component by using the adjustment parameter until the compression requirement is met, and output a lossy compression result.

10. The data compression system based on the adaptive revolving door algorithm according to claim 9, It is characterized in that The compression unit is specifically: Based on the output value of the adaptive revolving door algorithm at the current moment, calculating the output deviation value at the current moment; The output deviation value is compared with a preset deviation threshold. If the output deviation value is less than or equal to the preset deviation threshold, it means that the compression requirement is met and a lossy compression result is output.

Images