Cloud collaborative self-adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters

Abstract

The application discloses a cloud collaborative self-adaptive divide-and-conquer method for high-frequency data compression and reconstruction of intelligent electric meters, and comprises the following steps: using an event detection method of non-intrusive load identification to divide load power consumption data into event sections and non-event sections; using a fluctuation detection method based on speech endpoint detection to locate load power fluctuation sections for the data of the non-event sections, and the rest sections are load power smooth sections; based on the CS theory and a cloud collaborative framework, data compression and pre-reconstruction are carried out on the load power fluctuation section data in the terminal; if the pre-reconstruction error is greater than a set threshold, the data is taken as reconstruction data, and the data is stored in a fluctuation database at the same time; a sparse basis and an observation matrix are updated through dictionary learning; otherwise, the OMP algorithm is used for reconstruction; an improved SAX method is used for load power smooth section data compression and reconstruction; event section data and reconstructed data are spliced according to the order of appearance in original data, and complete load power consumption reconstruction data is obtained.

Description

Technical Field

[0001] This invention relates to a method for compressing electricity data, and more particularly to a cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters. Background Technology

[0002] With the implementation of advanced measurement systems and the development of information and communication technologies, electricity consumption data sensing devices, represented by smart meters, have been widely installed and deployed. [1] The resulting large amount of detailed monitoring data enables the analysis and identification of user electricity consumption behavior, customer segmentation, load forecasting, and power system demand response. [2] Advanced applications become possible. Due to the limited computing resources of edge devices, relevant big data and artificial intelligence analysis engines need to be deployed on cloud platforms. However, transmitting massive amounts of raw smart meter data back to the main station places a significant burden on system data communication. Therefore, an effective data compression method is urgently needed to achieve efficient data transmission with limited communication and resource requirements. This allows for the reconstruction of complete electricity consumption scenario data at the cloud main station, ensuring the application value of the data and maximizing the super-metering function of smart meters.

[0003] Data compression methods are mainly divided into two categories: lossless compression and lossy compression. [3] Lossy compression removes unimportant details from the data source, retaining only the critical information needed for data application, and is fast and efficient. If a suitable trade-off between data compression and information loss can be accepted, lossy compression methods are suitable for widespread application.

[0004] In the area of ​​lossy compression of power data, some scholars have already conducted relevant work. To improve compression efficiency, some scholars have used the Symbolic Aggregation Approximation (SAX) method. [4] Symbolic representation of the average load data within a fixed time window is used, but many key features of the electricity consumption data are lost.

[0005] Piecewise function approximation is a widely used compression method. First, the time series is divided into segments according to a certain rule, and then each segment is approximated by a function. [5] The optimal function is selected from the candidate function group to represent each segment, which improves the compression performance. However, the candidate function is still selected from the general function.

[0006] Electrical equipment has several fixed operating modes. Based on this characteristic, some scholars have proposed template-based load data compression schemes. One approach uses labeled electrical data segments as seeds to cluster the segmented training data to obtain templates for power consumption patterns, which are then used to represent the segmented data to be compressed. [6]This method performs well in scenarios with only a single appliance, but it requires a large amount of labeled training data for pattern discovery, resulting in slightly insufficient generalization ability.

[0007] Compressed sensing (CS) can sample and recover sparsifiable signals at a rate much lower than that required by the Nyquist sampling theorem, thus achieving signal dimensionality reduction. [7] A CS-based high-frequency load data compression method was proposed, which uses a random Gaussian matrix and a discrete Fourier basis as the measurement matrix and a sparse basis, and uses DFT coefficients containing 99.99% energy to estimate the sparsity, thereby adaptively determining the degree of compression, achieving good results, but the processing time is long.

[0008] [References]

[0009] [1] F.Jman, M.Abujubbeh, IoT-enabled smart grid via SM: An overview[J]. Future Generation Computer Systems, vol.96, pp.579-590, 2019.

[0010] [2]Y.Wang,Q.Chen,T.Hong,et al.,Review of smart meter data analytics:Applications,methodologies,and challenges[J].IEEE Transactions on Smart Grid,vol.10,no.3,pp.3125-3148,2018.

[0011] [3] CEShannon, "A Mathematical Theory of Communication," The BellSystem Technical Journal, vol.27, no.4, pp.623-656, 1948.

[0012] [4] J.Lin, E.Keogh, S.Lonardi, B.Chiu, "A symbolic representation of timeseries, with implications for streaming algorithms," in Proceedings of the 8thACM SIGMOD workshop on Research issues in data mining and knowledgediscovery, June.2003, pp.2-11.

[0013] [5] J.Qi, R.Zhang, K.Ramamohanarao, et al., "Indexable online time series segmentation with error bound guarantee," World Wide Web, vol.18, no.2, pp.359-401, 2015.

[0014] [6] K. Jia, G. Guo, J. Xiao, et al., "Data compression approach for the homeenergy management system," Applied Energy, 2019, 247: 643-656.

[0015] [7] S. Tripathi and S. De, "An Efficient Data Characterization and Reduction Scheme for Smart Metering Infrastructure," IEEETrans.Industr.Inform., vol.14, no.10, pp.4300-4308, Oct.2018. Summary of the Invention

[0016] In response to the aforementioned existing technologies, this invention provides a cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters. This method enables effective data transmission with limited communication and resource requirements, reduces the communication pressure on power companies caused by data collection, and restores complete electricity consumption scenario data through data reconstruction to ensure the application value of the data and maximize the super-metering function of smart meters.

[0017] To address the aforementioned technical problems, this invention proposes a cloud-based collaborative method for compressing and sensing load power fluctuation data, which mainly includes the following steps:

[0018] Step 1: Use a non-intrusive load identification event detection method to divide the high-frequency load power consumption data collected by the smart meter into event segments and non-event segments; for the data in the event segments, proceed to step 5; for the data in the non-event segments, proceed to step 2.

[0019] Step 2: Use the fluctuation detection method based on voice endpoint detection to locate the load power fluctuation segment. The remaining segments are the load power stability segments. Process the data of the load power fluctuation segment according to Step 3; process the data of the load power stability segment according to Step 4.

[0020] Step 3: Based on compressed sensing (CS) theory and cloud-based collaborative framework, perform load power fluctuation segment data compression and pre-reconstruction at the terminal, and determine whether the pre-reconstruction error is greater than a set threshold. When the pre-reconstruction error is greater than the set threshold, use the load power fluctuation segment data before compression as the reconstruction data, and execute Step 5. Furthermore, store the load power fluctuation segment data in the fluctuation database, update the sparse basis and observation matrix through dictionary learning, and use the updated sparse basis and observation matrix for subsequent load power fluctuation segment data compression and pre-reconstruction in the next iteration of Step 3. When the pre-reconstruction error is not greater than the set threshold, reconstruct the compressed load power fluctuation segment data using the orthogonal matching pursuit (OMP) algorithm, and then execute Step 5.

[0021] Step 4: Process the data using the improved Symbolic Aggregation Approximation (SAX) method. First, select compression parameters using the Rectangular Partitioning (DIRECT) algorithm. Then, partition and symbolically represent the load power consumption stability period data according to the selected compression parameters. After partitioning, the amount of information contained in each time partition is equal. Finally, reconstruct the data based on the mean, standard deviation, symbolic representation, and time partition breakpoint sequence of the load power consumption stability period data. Then, execute Step 5 sequentially.

[0022] Step 5: Combine the event segment data obtained in Step 1, the reconstructed load power fluctuation segment data obtained in Step 3, and the reconstructed load power stability segment data obtained in Step 4 according to their order of appearance in the load power data to obtain complete load power reconstruction data.

[0023] Furthermore, in the method described in this invention:

[0024] In step 1, a non-intrusive load identification event detection method is used to divide the load power consumption data into event segments and non-event segments. This includes: using a smart meter as a terminal, sampling the load power consumption data at a sampling frequency; using the non-intrusive load identification event detection method at the terminal to detect event segments in the data; each event segment corresponds to a transition process in the operating state of an appliance, including the appliance's on / off state and the switching between non-zero power operating states; other segments outside the event segments are non-event segments. The data of the divided event segments is transmitted back to the remote master station via a communication network, and the master station directly reads the data as the reconstructed data for that event segment.

[0025] In step 2, the steps for locating load power fluctuation segments using the fluctuation detection method based on voice endpoint detection are as follows:

[0026] 2-1) Perform a differential analysis on the non-event segment data after the division in step 1 and then connect them to obtain the differential signal. Among them, P S (i) For the i-th non-event segment in the load power consumption data, ΔP S (i) For P S (i) The difference sequence;

[0027] 2-2) Apply a sliding window of a certain length to the differential signal ΔP from step 2-1). S Perform a non-overlapping scan and calculate the fluctuation characteristics within the window according to Equation (1). When Equation (1) is true, the window is determined to be a fluctuation window.

[0028]

[0029] Among them, E j J represents the energy value within the j-th sliding window of the differential signal. W R is the length of the sliding window. j Range(·) represents the maximum value of the power range of the original non-event segment contained in the window. Range(·) indicates taking the range function. λ1 and λ2 are the corresponding judgment thresholds, which are the judgment thresholds of the energy value and the maximum power range within the sliding window, respectively.

[0030] 2-3) For the fluctuation window in step 2-2), connect adjacent windows with a spacing of less than a certain distance to form a load power fluctuation segment.

[0031] In step 3, based on compressed sensing theory and a cloud-based collaborative framework, load fluctuation data is compressed at the terminal, and then the compressed data is transmitted to the backend for reconstruction via a communication network, including:

[0032] 3-1) When compressing the data of the first fluctuation segment, the compressed sensing parameters are first initialized: the measurement matrix Φ is initialized as a random Gaussian matrix, and the sparse basis Ψ is initialized as a discrete Fourier transform matrix; then a fluctuation database is created in the background; next, the data of this fluctuation segment enters step 2) for processing.

[0033] 3-2) For the load power fluctuation segment data f that needs to be processed, it is compressed based on the compressed sensing theory to obtain low-dimensional data y, as shown in equation (2);

[0034] y=Φf (2)

[0035] Then, the low-dimensional data y is pre-reconstructed at the terminal. That is, based on the current measurement matrix Φ and sparse basis Ψ, the orthogonal matching pursuit (OMP) algorithm is used to recover the load power fluctuation segment data f from the low-dimensional data y, thus obtaining the reconstructed load power fluctuation segment data. And calculate the pre-reconstruction error E r , as in equation (3)

[0036]

[0037] Where L is the total length of the load power fluctuation segment data f, f l and These are the standardized results of the original load power fluctuation segment data and the reconstructed load power fluctuation segment data, respectively.

[0038] 3-3) When the pre-reconstruction error obtained in step 3-2) is greater than the given threshold, the load power fluctuation segment data f is sent back to the background and the process proceeds to step 3-4); otherwise, the compressed low-dimensional data y is sent back to the background and the process proceeds to step 3-5.

[0039] 3-4) Receive load power fluctuation segment data f in the background and supplement it into the fluctuation database. Use the load power fluctuation segment data samples in the fluctuation database to obtain a new sparse basis Ψ' and a new measurement matrix Φ' through dictionary learning and matrix optimization. Then assign them to the sparse basis Ψ and the measurement matrix Φ, and send them to the terminal as compressed sensing parameters when processing the next load power fluctuation segment. In addition, use the load power fluctuation segment data f as reconstruction data and proceed to step 5.

[0040] 3-5) In the background, orthogonal matching pursuit (OMP) is used to recover the load power fluctuation segment data f from the low-dimensional data y, and it is used as the reconstructed data before proceeding to step 5.

[0041] In step 4, based on the improved Symbolic Aggregation Approximation (SAX) method, the load power consumption stability data obtained in step 2 is compressed, and then the compressed data is transmitted to the backend via a communication network for reconstruction, including:

[0042] 4-1) First, parameter selection is performed: the rectangular partitioning (DIRECT) algorithm is adopted, that is, the candidate parameters, including the number of time partitions and the number of symbol representations, are used as variables, the reconstruction error function is used as the objective function, and the variable values ​​when the error function obtains the global optimal solution are the final selected parameter values;

[0043] 4-2) Based on the parameter values ​​selected in step 4-1), and using the improved symbolic aggregation approximation method, the load power stability segment data to be compressed is symbolically represented on the device to complete the data compression; the specific process is as follows:

[0044] For the original time series of the stable power consumption period of the load to be compressed, its time axis is first partitioned: the cumulative distribution function (CDF) of the original data difference sequence needs to be constructed, and the projections of its equal quantiles onto the time axis form the time partition breakpoint sequence <c1,c2,…,c z ,…,c Z >. Then partition its amplitude axis: divide the Gaussian distribution into equal probability quantile sequences <β1,β2,…,β n ,…,β N-1 > As a sequence of amplitude partitioning points, the amplitude intervals are represented by symbols 1, 2, ..., N;

[0045] During symbolization, the power mean in each time partition is calculated and symbolized according to the position of the power mean in the amplitude interval, as shown in equation (4);

[0046]

[0047] Among them, c z-1 and c z These are the (z-1)th and zth time partition breakpoints, respectively. It is a time partition breakpoint c z-1 to c z The power mean within the time partition between them It is a time partition breakpoint c z-1 to c z The symbolic representation of the load data within the time partition, β n-1 and β n These are the (n-1)th and nth amplitude partitioning breakpoints, respectively;

[0048] 4-3) The compressed data is transmitted to the background for reconstruction, which involves reversing the sign values ​​in each time partition to restore the data values. The specific process is as follows:

[0049] For any stable segment, calculate the reconstructed load data value in the z-th time partition according to formula (5). Time partition breakpoint c z-1 To cz The reconstructed load power value at each sampling point is As shown in equation (5),

[0050]

[0051] in, and The power symbol for the z-th time partition represents the lower and upper bounds of that interval. and All are obtained based on the amplitude partition breakpoint sequence. σ and σ are the mean and standard deviation of the load power consumption data during the stable period, respectively.

[0052] Compared with the prior art, the beneficial effects of the present invention are:

[0053] Existing electricity consumption information collection systems or metering automation systems can typically only reliably collect minute-level measurement data from the user side, making it difficult to acquire fine-grained data at higher frequencies (such as second-level) from the user side. This significantly limits the realization of advanced smart grid functions supported by the advanced measurement system in the vision design. To address this, this invention, based on the "divide and conquer" approach, employs a fluctuation segment detection and localization method based on voice endpoint detection, combined with an event detection algorithm. It segments the load electricity consumption data according to its change patterns, and then applies different processing methods for compression and reconstruction. This approach, even with limited remote communication resources, minimizes or limits the amount of data transmitted back to the master station while retaining more important information contained in the high-frequency data, ultimately enabling the master station layer to acquire high-frequency measurement data from smart meters. Specifically, on the one hand, the cloud-based collaborative fluctuation data compression and reconstruction method based on compressed sensing can adaptively update scenario-specific compressed sensing parameters when encountering unfamiliar scenarios, i.e., when the pre-reconstruction error is large, ensuring high reconstruction accuracy of fluctuation segment data under different scenarios. On the other hand, the SAX-based method for compressing and reconstructing stationary segment data combines the DIRECT method and the irregular time partitioning method. It can find symbolic representation parameters that minimize reconstruction error and time partition breakpoints that can better track the changing trend of stationary segment data, thus improving compression efficiency while retaining more data features. Attached Figure Description

[0054] Figure 1 This is a flowchart of the cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters according to the present invention.

[0055] Figure 2 This is a schematic diagram of the segmented load power consumption data;

[0056] Figure 3 This is a flowchart of data compression and reconstruction for load power fluctuation segments;

[0057] Figure 4 This is a schematic diagram of the power segment during a period of stable power consumption for a certain load.

[0058] Figure 5 These are load power reconfiguration data curves of the method of this invention and existing compressed sensing methods. Detailed Implementation

[0059] The design concept of the cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters proposed in this invention is as follows: The raw load power consumption data is divided into event segments and non-event segments using an event detection method. Then, a fluctuation detection method is used to locate the fluctuation segments, while the remaining segments are considered stable segments. The event segment data is transmitted back to the remote master station as is; during reconstruction, only the transmitted data needs to be read without further processing. For the fluctuation segment data, load power consumption fluctuation segment data compression and pre-reconstruction are performed at the terminal based on compressed sensing (CS) theory. When the pre-reconstruction error is large, the load power consumption fluctuation segment data is transmitted back to the backend to update the sparse basis and observation matrix through dictionary learning. Simultaneously, the transmitted load power consumption fluctuation segment data is used as the reconstruction data; otherwise, the compressed data is used. The data is transmitted to the backend and reconstructed using the Orthogonal Matching Pursuit (OMP) algorithm. For the stable load segments, an improved Symbolic Aggregation Approximation (SAX) method is used. First, the DIRECT algorithm is used to select compression parameters. Then, based on the selected parameters, the stable load consumption data is partitioned and symbolically represented. Periods with larger fluctuations in the appliance operating power curve are partitioned more densely, while periods with smaller fluctuations are partitioned more sparsely. The mean, standard deviation, symbolic representation, and time-partition breakpoint sequence of the stable load consumption data are then transmitted back to the backend for reconstruction. Finally, the reconstructed data segments are concatenated in the backend according to their order of appearance in the original load consumption data to obtain the complete reconstructed load consumption data.

[0060] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention.

[0061] like Figure 1 As shown, the cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters includes the following steps:

[0062] Step 1: Divide the load power consumption data into event segments and non-event segments using a non-intrusive load identification event detection method. This includes: using a smart meter as a terminal, sampling the load power consumption data at a sampling frequency; using a non-intrusive load identification event detection method at the terminal to detect event segments in the data. These event segments correspond to the transition process of appliance operating state changes, including the turning on, turning off, and switching between non-zero power operating states of appliances, such as... Figure 2This diagram illustrates that event segments are specially marked, and other segments outside the event segment markings are non-event segments. For event segment data, step 5 is executed, which involves transmitting the divided event segment data back to the remote master station via the communication network. The master station directly reads the data as the reconstructed data for that event segment. For non-event segment data, step 2 is executed.

[0063] Step 2: Use a voice endpoint detection-based fluctuation detection method to locate the load power fluctuation segment; the remaining segments are the load power stability segments, such as... Figure 2 Illustration. Data for fluctuating load power consumption is processed according to step 3; data for stable load power consumption is processed according to step 4.

[0064] The steps for locating load power fluctuation segments using a voice endpoint detection-based fluctuation detection method are as follows:

[0065] 2-1) Perform a differential operation on the non-event segment data after step 1 and connect them to obtain the differential signal ΔP. S ={ΔP S (i) |i=1,2,3...}, where P S (i) For the i-th non-event segment in the load power consumption data, ΔP S (i) For P S (i) The difference sequence;

[0066] 2-2) Using a sliding window of a certain length (the basic principle for determining this length is that the length of the sliding window is determined by the amount of data expected to be processed in a single step when judging fluctuations), apply the differential signal ΔP from step 2-1). S Perform a non-overlapping scan and calculate the fluctuation characteristics within the window according to Equation (1). When Equation (1) is true, the window is determined to be a fluctuation window.

[0067]

[0068] Among them, E j J represents the energy value within the j-th sliding window of the differential signal. W R is the length of the sliding window. j Range(·) represents the maximum value of the power range of the original non-event segment contained within the window. λ1 and λ2 are the threshold values ​​for determining the energy value and the maximum power range within the sliding window, respectively, determined by the minimum fluctuation level of the data expected to be classified as a fluctuation segment.

[0069] 2-3) For the fluctuation window in step 2-2), connect adjacent windows with a spacing of less than a certain distance to form a load power fluctuation segment.

[0070] Step 3, as follows Figure 3 As shown, based on compressed sensing (CS) theory and a cloud-based collaborative framework, load fluctuation data is compressed and pre-reconstructed at the terminal. The pre-reconstruction error is determined to be greater than a set threshold, which is selected based on the maximum acceptable error during reconstruction. When the pre-reconstruction error exceeds the set threshold, the load fluctuation data before compression is used as the reconstruction data, and step 5 is executed. Furthermore, the load fluctuation data is stored in a fluctuation database, and the sparse basis and observation matrix are updated through dictionary learning. The updated sparse basis and observation matrix are then used in step 3 of the next cycle for load fluctuation data compression and pre-reconstruction. When the pre-reconstruction error is not greater than the set threshold, the compressed load fluctuation data is reconstructed using the Orthogonal Matching Pursuit (OMP) algorithm, and then step 5 is executed.

[0071] In step 3, based on compressed sensing theory and a cloud-based collaborative framework, load fluctuation data is compressed at the terminal, and then the compressed data is transmitted to the backend for reconstruction via a communication network, including:

[0072] 3-1) When compressing the data of the first fluctuation segment, the compressed sensing parameters are first initialized: the measurement matrix Φ is initialized as a random Gaussian matrix, and the sparse basis Ψ is initialized as a discrete Fourier transform matrix; then a fluctuation database is created in the background; next, the data of this fluctuation segment enters step 3-2) for processing.

[0073] 3-2) For the load power fluctuation segment data f that needs to be processed, it is compressed based on the compressed sensing theory to obtain low-dimensional data y, as shown in equation (2);

[0074] y=Φf (2)

[0075] Then, the low-dimensional data y is pre-reconstructed at the terminal. That is, based on the current measurement matrix Φ and sparse basis Ψ, the orthogonal matching pursuit (OMP) algorithm is used to recover the load power fluctuation segment data f from the low-dimensional data y, thus obtaining the reconstructed load power fluctuation segment data. And calculate the pre-reconstruction error E r , as in equation (3)

[0076]

[0077] Where L is the total length of the load power fluctuation segment data f, f l and These are the standardized results of the original load power fluctuation segment data and the reconstructed load power fluctuation segment data, respectively.

[0078] 3-3) When the pre-reconstruction error obtained in step 3-2) is greater than the given threshold, the load power fluctuation segment data f is sent back to the background and the process proceeds to step 3-4); otherwise, the compressed low-dimensional data y is sent back to the background and the process proceeds to step 3-5.

[0079] 3-4) Receive load power fluctuation segment data f in the background and supplement it into the fluctuation database. Use the load power fluctuation segment data samples in the fluctuation database to obtain a new sparse basis Ψ' and a new measurement matrix Φ' through dictionary learning and matrix optimization. Then assign them to the sparse basis Ψ and the measurement matrix Φ, and send them to the terminal as compressed sensing parameters when processing the next load power fluctuation segment. In addition, use the load power fluctuation segment data f as reconstruction data and proceed to step 5.

[0080] 3-5) In the background, orthogonal matching pursuit (OMP) is used to recover the load power fluctuation segment data f from the low-dimensional data y, and it is used as the reconstructed data before proceeding to step 5.

[0081] Step 4: Process the data using the improved Symbolic Aggregation Approximation (SAX) method. First, select compression parameters using the Rectangular Partitioning (DIRECT) algorithm. Then, partition and symbolically represent the load power consumption stability period data according to the selected compression parameters. After partitioning, the amount of information contained in each time partition is equal. Finally, reconstruct the data based on the mean, standard deviation, symbolic representation, and time partition breakpoint sequence of the load power consumption stability period data. Then, execute Step 5 sequentially.

[0082] In step 4, based on the improved Symbolic Aggregation Approximation (SAX) method, the load power consumption stability data obtained in step 2 is compressed, and then the compressed data is transmitted to the backend via a communication network for reconstruction, including:

[0083] 4-1) First, parameter selection is performed: the rectangular partitioning (DIRECT) algorithm is adopted, that is, the candidate parameters, including the number of time partitions and the number of symbol representations, are used as variables, the reconstruction error function is used as the objective function, and the variable values ​​when the error function obtains the global optimal solution are the final selected parameter values;

[0084] 4-2) Based on the parameter values ​​selected in step 4-1), and using the improved symbolic aggregation approximation method, the load power stability segment data to be compressed is symbolically represented on the device to complete the data compression; the specific process is as follows:

[0085] For the original time series of the stable power consumption period of the load to be compressed, its time axis is first partitioned: the cumulative distribution function (CDF) of the original data difference sequence needs to be constructed, and the projections of its equal quantiles onto the time axis form the time partition breakpoint sequence <c1,c2,…,c z ,…,c Z>. Then partition its amplitude axis: divide the Gaussian distribution into equal probability quantile sequences <β1,β2,…,β n ,…,β N-1 > As a sequence of amplitude partitioning points, the amplitude intervals are represented by symbols 1, 2, ..., N; Figure 4 Taking a certain load power segment as an example, a partition diagram is shown when the number of time partitions Z is 7 and the number of symbols N is 4. During symbolization, the power mean in each time partition is calculated and symbolized according to the position of the power mean in the amplitude range, as shown in equation (4);

[0086]

[0087] Among them, c z-1 and c z These are the (z-1)th and zth time partition breakpoints, respectively. It is a time partition breakpoint c z-1 to c z The power mean within the time partition between them It is a time partition breakpoint c z-1 to c z The symbolic representation of the load data within the time partition, β n-1 and β n These are the (n-1)th and nth amplitude partitioning breakpoints, respectively;

[0088] 4-3) After the load power consumption stability period data is symbolically represented, the compressed data is averaged. Standard deviation σ, sign represents the sequence The data, along with the location records of each time partition, is transmitted to the backend for reconstruction. This involves reverse-engineering the symbol values ​​within each time partition back into data values, and then proceeding to step 6. The specific process of reverse reconstruction is as follows:

[0089] For any stable segment, calculate the reconstructed load data value in the z-th time partition according to formula (5). Time partition breakpoint c z-1 To c z The reconstructed load power value at each sampling point is As shown in equation (5),

[0090]

[0091] in, and The power symbol for the z-th time partition represents the lower and upper bounds of that interval. and All are obtained based on the amplitude partition breakpoint sequence. σ and σ are the mean and standard deviation of the load power consumption data during the stable period, respectively.

[0092] Step 5: Combine the event segment data obtained in Step 1, the reconstructed load power fluctuation segment data obtained in Step 3, and the reconstructed load power stability segment data obtained in Step 4 according to their order of appearance in the load power data (i.e., the original data) to obtain complete load power reconstruction data.

[0093] The method described in this invention and an existing compressed sensing method [7] were experimentally compared to compare the data compression performance of the two methods. In step 2 of the method of this invention, when calculating the fluctuation characteristics within the window, the length of the sliding window is 20, the threshold λ1 for determining the energy value within the sliding window is 1500, and the threshold λ2 for determining the maximum power range within the sliding window is 25; for the fluctuation window, adjacent windows with a spacing of less than 10 are connected to form a load power fluctuation segment. In step 3, the threshold set for judging the pre-reconstruction error is 0.9.

[0094] The measured electricity consumption data used in the experiment were high-frequency measurement data from smart meters, collected from six different households, with a sampling frequency of 1Hz for each data point. The performance of data compression was described from two aspects: reconstruction accuracy and compression efficiency. Reconstruction accuracy refers to the difference between the reconstructed data and the original data; compression efficiency refers to the degree to which the data can be reduced through compression. Reconstruction accuracy was measured using the root mean square error (RMSE), calculated as shown in equation (6). Compression efficiency was measured using the compression ratio (CR), calculated as shown in equation (7).

[0095]

[0096] Where T equals the total length of the original load power consumption data, P t and These are the standardized results of the original load power consumption data and the reconstructed data.

[0097]

[0098] Where S0 is the amount of data to be transmitted before compression, and S is the amount of data to be transmitted after compression.

[0099] The compression performance results of the two methods are compared in Table 1. In terms of reconstruction accuracy, the method of this invention performs better overall, with an average reconstruction error of 0.18%, meaning an average reconstruction accuracy of 99.82%. Furthermore, the method of this invention exhibits more stable performance across different scenarios, with a smaller variance in RMSE. This is because, on the one hand, this paper adopts a "divide and conquer" approach, dividing the original data into finer segments and using appropriate compression and reconstruction methods based on the characteristics of different data segments. On the other hand, key parameters in this invention (such as the sparse basis and measurement matrix used for compressing fluctuating load power consumption data, and the number of time partitions and symbol representations used for compressing stable load power consumption data) can be adaptively adjusted according to changes in the scenario, ensuring high reconstruction accuracy under different scenarios. From the perspective of compression efficiency, because this invention adopts a divide-and-conquer strategy, it optimizes the compression efficiency of fluctuating segments while using an improved SAX method to process stable segments, significantly improving the compression ratio. Overall, the compression efficiency of this invention is significantly better than the comparative methods.

[0100] Table 1 Comparison of compression performance results between the two methods of this invention and reference [7]

[0101]

[0102] Figure 5 Using partial data from household number 1, the load reconfiguration data curves of the proposed method and the comparative method are presented. As can be seen from the figure, the comparative method has poor applicability to partial load data segments, especially for stable segments adjacent to step-type event segments, where the reconfiguration curve exhibits significant jitter. In contrast, the proposed method demonstrates better adaptability to various data segments.

[0103] Although the invention has been described above in conjunction with the accompanying drawings, it is not intended to limit the invention. The invention is not limited to the examples given above. Any changes, modifications, additions, or substitutions made by those skilled in the art within the scope of the invention should also be considered within the protection scope of the invention.

Claims

1. A cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters, characterized in that, Includes the following steps: Step 1: Use a non-intrusive load identification event detection method to divide the high-frequency load power consumption data collected by the smart meter into event segments and non-event segments; for the data in the event segments, proceed to step 5; for the data in the non-event segments, proceed to step 2. Step 2: Use the fluctuation detection method based on voice endpoint detection to locate the load power fluctuation segment. The remaining segments are the load power stability segments. Process the data of the load power fluctuation segment according to Step 3; process the data of the load power stability segment according to Step 4. Step 3: Based on compressed sensing (CS) theory and cloud-based collaborative framework, perform load power fluctuation data compression and pre-reconstruction at the terminal, and determine whether the pre-reconstruction error is greater than the set threshold. When the pre-reconstruction error is greater than the set threshold, the data of the previous load power fluctuation segment is used as the reconstruction data, and step 5 is executed. Furthermore, the data of the load power consumption fluctuation segment is stored in the fluctuation database, and the sparse basis and observation matrix are updated through dictionary learning. The updated sparse basis and observation matrix are then used in step 3 of the next cycle for load power consumption fluctuation segment data compression and pre-reconstruction. When the pre-reconstruction error is not greater than the set threshold, the compressed load power fluctuation segment data is reconstructed using the orthogonal matching pursuit (OMP) algorithm, and then step 5 is executed; Step 4: Process the data using the improved Symbolic Aggregation Approximation (SAX) method. First, select compression parameters using the Rectangular Partitioning (DIRECT) algorithm. Then, partition and symbolically represent the load power consumption stability period data according to the selected compression parameters. After partitioning, the amount of information contained in each time partition is equal. Finally, reconstruct the data based on the mean, standard deviation, symbolic representation, and time partition breakpoint sequence of the load power consumption stability period data. Then, execute Step 5 sequentially. Step 5: Combine the event segment data obtained in Step 1, the reconstructed load power fluctuation segment data obtained in Step 3, and the reconstructed load power stability segment data obtained in Step 4 according to their order of appearance in the load power data to obtain complete load power reconstruction data.

2. The cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters according to claim 1, characterized in that, In step 1, a non-intrusive load identification event detection method is used to divide the load power consumption data into event segments and non-event segments. This includes: using a smart meter as a terminal, sampling the load power consumption data at the terminal according to the sampling frequency, and using a non-intrusive load identification event detection method at the terminal to detect event segments in the data. The event segments correspond to the transition process of the appliance's operating state, including the appliance's on / off state and the switching between non-zero power operating states. Other segments outside the event segments are non-event segments.

3. The cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters according to claim 1, characterized in that, In step 1, the data of the divided event segments is transmitted back to the remote master station through the communication network, and the master station directly reads the data as the reconstructed data of the event segment.

4. The cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters according to claim 1, characterized in that, In step 2, the steps for locating load power fluctuation segments using the fluctuation detection method based on voice endpoint detection are as follows: Step 2-1) Perform a differential analysis on the non-event segment data after the division in Step 1 and then connect them to obtain the differential signal. ,in, For the i-th non-event segment in the load power consumption data, For the i-th non-event segment Differential sequence of load power consumption data; Step 2-2) Apply a sliding window of a certain length to the differential signal in step 2-1). Perform a non-overlapping scan and calculate the fluctuation characteristics within the window according to Equation (1). When Equation (1) is true, the window is determined to be a fluctuation window. （1）； in, Let be the energy value within the j-th sliding window of the differential signal. The length of the sliding window. This represents the maximum value of the power range of the original non-event segments contained within the window. This indicates taking the range function. and These are the threshold values ​​for determining the maximum energy value and the maximum power range within the sliding window, respectively. Step 2-3) For the fluctuation window in step 2-2), connect adjacent windows with a spacing of less than a certain distance to form a load power fluctuation segment.

5. The cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters according to claim 1, characterized in that, In step 3, based on compressed sensing theory and a cloud-based collaborative framework, load fluctuation data is compressed at the terminal, and then the compressed data is transmitted to the backend for reconstruction via a communication network, including: Step 3-1) When compressing the data of the first fluctuation segment, the compressed sensing parameters are first initialized: measurement matrix. Initialized as a random Gaussian matrix with sparse basis Initialize as a discrete Fourier transform matrix; then create a fluctuation database in the background; next, the fluctuation segment data will proceed to step 3-2) for processing; Step 3-2) For the load power fluctuation segment data f that needs to be processed, compress it based on compressed sensing theory to obtain low-dimensional data y, as shown in equation (2). （2）； Then, the low-dimensional data y is pre-reconstructed at the terminal, that is, based on the current measurement matrix. and sparse base The Orthogonal Matching Pursuit (OMP) algorithm is used to recover the load power fluctuation segment data f from the low-dimensional data y, thus obtaining the reconstructed load power fluctuation segment data. And calculate the pre-reconstruction error. , as in equation (3) （3）； Where L is the total length of the load power fluctuation segment data f. and These are the standardized results of the original load power fluctuation segment data and the reconstructed load power fluctuation segment data, respectively. Step 3-3) If the pre-reconstruction error obtained in step 3-2) is greater than the given threshold, transmit the load power fluctuation segment data f back to the backend and proceed to step 3-4); otherwise, transmit the compressed low-dimensional data y back to the backend and proceed to step 3-5). Steps 3-4) Receive load power fluctuation segment data f in the background and supplement it to the fluctuation database. Use the load power fluctuation segment data samples in the fluctuation database to obtain a new sparse basis through dictionary learning and matrix optimization. and the new measurement matrix Then assign it to the sparse base. and measurement matrix The data is sent to the terminal as compressed sensing parameters when processing the next load fluctuation segment; and the load fluctuation segment data f is used as reconstructed data and proceeds to step 5. Steps 3-5) In the background, orthogonal matching pursuit (OMP) is used to recover the load power fluctuation segment data f from the low-dimensional data y, and this data is used as the reconstructed data before proceeding to step 5.

6. The cloud-based collaborative adaptive divide-and-conquer method for high-frequency data compression and reconstruction of smart meters according to claim 1, characterized in that, In step 4, based on the improved Symbolic Aggregation Approximation (SAX) method, the load power consumption stability data obtained in step 2 is compressed, and then the compressed data is transmitted to the backend via a communication network for reconstruction, including: Step 4-1) First, select parameters: Use the rectangular partitioning (DIRECT) algorithm, that is, take the candidate parameters, including the number of time partitions and the number of symbol representations, as variables, and take the reconstruction error function as the objective function. The variable values ​​when the error function obtains the global optimal solution are the final selected parameter values. Step 4-2) Based on the parameter values ​​selected in Step 4-1), and using the improved symbolic aggregation approximation method, the load power stability segment data to be compressed is symbolically represented on the device to complete the data compression; the specific process is as follows: For the original time series of the stable power consumption period of the load to be compressed, its time axis is first partitioned: the cumulative distribution function (CDF) of the original data difference sequence needs to be constructed, and the projections of its equal quantiles onto the time axis form the time partition breakpoint sequence. Then, partition its amplitude axis: divide the Gaussian distribution into equal probability quantile sequences. , ,…, ,…, As a sequence of amplitude partitioning breakpoints, the amplitude intervals are represented by symbols 1, 2, ..., N; During symbolization, the power mean in each time partition is calculated and symbolized according to the position of the power mean in the amplitude interval, as shown in equation (4). （4）； in, and These are the (z-1)th and zth time partition breakpoints, respectively. Time partition breakpoint arrive The power mean within the time partition between them Time partition breakpoint arrive The symbolic representation of load data within the time partitions between them. and These are the (n-1)th and nth amplitude partitioning breakpoints, respectively; Step 4-3) The compressed data is transmitted to the background for reconstruction, which involves reversing the symbol values ​​in each time partition to restore the data values. The specific process is as follows: For any stable segment, calculate the reconstructed load data value in the z-th time partition according to formula (5). Time partition breakpoints to The reconstructed load power value at each sampling point is As shown in equation (5), （5）； in, and The power symbol for the z-th time partition represents the lower and upper bounds of that interval. and All are obtained based on the amplitude partition breakpoint sequence. and These are the mean and standard deviation of the load power consumption data during the stable period, respectively.

Images