A method for identifying and repairing energy consumption data based on statistical patterns
Through statistical mode-based methods, abnormal and missing data in building energy consumption data are identified and repaired, and the G-test and EM algorithms and sparse Bayesian time series method are used to solve the problem of poor data repair results in the prior art. , achieving higher repair accuracy and accuracy.
Patent Information
- Application Number
- CN202010864776.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-25
- Publication Date
- 2024-12-10
- Estimated Expiration
- 2040-08-25
AI Technical Summary
The existing building energy consumption data repair methods are due to sensor failure and network signal instability, resulting in data acquisition errors or loss, making it difficult to obtain better repair results.
Using a statistical mode-based method, the data is divided into subsets according to the data source, G-test is used to identify abnormal data and correct it through the expected maximum value algorithm. The missing data set is repaired by EM algorithm, and the data set with high missing rate is sparse. Yes time series method is repaired.
It improves the accuracy and accuracy of data repair, can effectively repair the missing data of various building energy consumption, and provides more accurate energy consumption data support for further analysis.
Smart Images
Figure CN111984626B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of building energy consumption data analysis, and in particular relates to a method for identifying and repairing energy consumption data based on statistical patterns. Background Art
[0002] Based on existing building energy consumption data, energy consumption forecasts can be easily performed, providing detailed data support for energy scheduling and achieving the goal of reducing building energy consumption. Therefore, the integrity of energy consumption data directly affects the accuracy of forecast results. However, in practice, the collection of building energy consumption data is prone to sensor failures and unstable network signals, resulting in errors or loss of operational data, which greatly inconveniences energy consumption data analysis.
[0003] Interpolation and averaging are currently common data repair methods, which simply fill in the missing data. However, due to the large differences in the amount of data and the amount of missing data collected by sensors in real-world building energy consumption data, a one-size-fits-all repair method is unlikely to achieve effective results. Summary of the Invention
[0004] In order to solve the problems existing in the prior art, the present invention provides an energy consumption data identification and repair method based on statistical patterns to solve the problem of poor repair effect caused by the existing data repair method.
[0005] The technical solution adopted in the present invention is as follows:
[0006] A method for identifying and repairing energy consumption data based on statistical patterns, comprising:
[0007] Dividing the sample data into multiple subsets according to the data source, the sample data is the building energy consumption data record within a preset time period, and the data in the subsets respectively correspond to the data collected by each data source;
[0008] Divide the data matrix of each subset into time series form, and clarify the known and unknown quantities in each subset;
[0009] The subsets are divided according to whether they contain missing data, and the subset with 0 unknowns is recorded as the non-missing data set S O , record the subset with the number of unknown numbers greater than 0 as the missing data set S d ;
[0010] For the non-missing dataset S O Perform G-test identification and record the identified abnormal data set as S′ O , and use the expected maximum algorithm to correct it;
[0011] For the missing data set Sd The missing rate is used to divide the data missing situation. For the data set with missing rate < preset value A, the EM algorithm is used to repair the energy consumption data. For the data set with missing rate ≥ preset value A, the sparse Bayesian time series method is used to repair the energy consumption data.
[0012] Preferably, the data sources include the building's lighting socket electricity consumption, air conditioning electricity consumption, power electricity consumption, special electricity consumption, temperature, humidity, relative humidity, radiation, and the number of people indoors.
[0013] Preferably, the data matrix of each subset is divided into a time series form, and the data collected by the jth group of data sources can be expressed as S j ={S 1j ,S 2j ,…,S nj}.
[0014] Preferably, the process of clarifying each known quantity and unknown number in each subset includes: assuming n is a binary variable that represents the observability of the collected data, where n represents the sequence number of the moment in the time series, n = 1, 2, ..., N; if O n =1, it means that the nth moment is observable, otherwise if O n =0, it means that the output at time n is unobservable; observable data is recorded as known quantity, and unobservable data is recorded as unknown quantity.
[0015] Preferably, for the non-missing dataset S O Perform G-test identification and record the identified abnormal data set as S′ O , and use the EM algorithm to correct it. The process is as follows:
[0016] For the non-missing dataset S O , we select the G-test identification method commonly used in the medical field, and complete the quality judgment of energy consumption data by testing whether the observable data variable values meet the theoretical expectations. By tracking the maximum confidence level of building energy consumption data, we judge whether it meets the minimum requirements. The non-missing data sets that meet the requirements are retained, and the abnormal data sets S′ that do not meet the requirements are deleted. O , and the EM algorithm is used for correction.
[0017] Preferably, for the missing data set S d The missing rate is used to divide the data missing situation. The data set with a missing rate < the preset value A is recorded as data set B′, and the data set with a missing rate ≥ the preset value A is recorded as data set B″. The missing rate of building energy consumption data is set as the ratio of the number of missing data in the data set to the total number of data in the data set.
[0018] For the dataset B′ whose missing rate is less than the preset value A, the EM algorithm is used to repair the energy consumption data, as follows:
[0019] The unknown quantity in the data set B′ is recorded as the implicit variable I n (n=1,2,…,N),I n Represents a discrete variable at time n, with the value being a data corresponding to the 1st, 2nd, ..., Nth time, and the known quantity is recorded as the observed data X i , where n represents the sequence number of the moment in the time series, n = 1, 2,…, N.
[0020] The EM algorithm iterates according to the E step and the M step to obtain a complete data set S that is filled and can be used for the next step of analysis. The specific steps of the EM algorithm are as follows:
[0021] The dataset B′ is taken as input and the complete dataset S after data repair is taken as output.
[0022] S1: Initialize dataset B′.
[0023] S2: Set the number of iterations k to 0 and perform E and M steps;
[0024] Expectation step (ie E step): Use the current parameter to estimate the approximate Q function and the estimated parameter θ, and use the implicit variable estimate value I in the data set B′ n and observation data X i To calculate the complete data set S, the expression of the log-likelihood function is as follows:
[0025] Q i (I n )=P(I n |X i θ (k) )
[0026] According to the value of θ in the kth iteration, the implicit variable I is calculated n The probability of the value taken.
[0027] Maximization step (i.e., M step): maximize the Q function to obtain a new iterative parameter, that is, solve the value of θ when the function reaches its maximum value in the following formula, and update the value of θ.
[0028] θ (k+1) =argmax θ Q i (I n )
[0029] S3: The number of iterations k is set to k+1, and the E and M steps in S3 are continued until the data converges, that is, |θ (k+1) -θ (k)| is sufficiently small. The estimated value of the implicit variable I′ at this time n and observation data X′ i To complete the dataset S.
[0030] For the data set B″ whose missing rate is ≥ the preset value A, the sparse Bayesian time series method is used to repair the building energy consumption data. The specific method is:
[0031] The time series signal of building energy consumption itself has time domain smoothness and only changes significantly at a few moments. Therefore, there are not many large data in the difference data of two adjacent samples, and most of the small differences can be ignored. Therefore, in order to obtain the difference of building energy consumption at adjacent observation points, a difference matrix M1 is designed. After completing the design of matrix M1, the j group data S″ in the data set B″ is j Projecting the M1 matrix can be expressed as:
[0032]
[0033] x j1 The elements in represent the difference in building energy consumption at adjacent observation points. j1 There are only a few large elements in , so it can be represented by the time domain smoothness matrix. Similar to the matrix M1, in order to express S″ j The acceleration matrix M2 is designed to represent the time series of acceleration changes. j Projecting the M2 matrix can be expressed as:
[0034]
[0035] x j2 Elements in (S″ ij -S″ (i-1)j )-(S″ (i+1)j -S″ ij ) can S″ j The time series represented by x is expressed as the acceleration of change. j2 Only a few elements have large values, and many elements have values that can be ignored. Then the energy consumption time series of the building can be expressed as S″ j =ψx j (j=1,2,...,K), thus obtaining the complete data set S after data recovery, ψ represents ψ1 or ψ2, where K is the number of data sources.
[0036] Compared with the prior art, the present invention has at least the following beneficial effects:
[0037] The energy consumption data identification and repair method based on statistical model of the present invention selects G-test identification method to identify the non-missing data set S O Screening is performed, and the datasets that meet the requirements are retained, and the abnormal datasets S′ that do not meet the requirements are O The EM algorithm is used for correction. Based on the G-test identification, the missing data cases are divided based on the missing rate as the criterion. The EM algorithm and the sparse Bayesian time series method are used to repair the two data sets with missing data, respectively, to improve the accuracy of data repair. In summary, the method provided by the present invention can effectively and reliably repair various cases of missing building energy consumption data, and has good repair accuracy to obtain more accurate building energy consumption data, thereby facilitating further mining and analysis of energy consumption data.
[0038] Furthermore, the sparse Bayesian time series method, leveraging the temporal smoothness inherent in building energy consumption time series signals, specifically represents energy consumption data within a sparse representation basis, thereby recovering the original data. This method is suitable for situations where large-scale building energy consumption data is missing. Even if there are missing data in a localized dataset, accurate building energy consumption data can still be recovered. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 The overall framework diagram of the energy consumption data identification and repair method based on statistical patterns of the present invention. DETAILED DESCRIPTION
[0040] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings.
[0041] Figure 1 The overall framework of the energy consumption data identification and repair method based on statistical patterns of the present invention includes the following steps:
[0042] S1. Divide sample data into multiple subsets, where the sample data is a record of building energy consumption data within a preset time period, and the data in the subsets respectively correspond to data collected by various data sources.
[0043] During the execution of S1, when collecting building energy consumption data, the various data sources usually include 9 categories, namely building lighting socket electricity, air conditioning electricity, power electricity, special electricity, temperature, humidity, relative humidity, radiation and number of indoor people. Figure 1 As shown in subset 1-subset k.
[0044] S2. Divide the data matrix of each subset into time series form, clarify each known quantity and unknown quantity, then the data collected by the jth group of data sources can be expressed as S j ={S 1j,S 2j ,…,S nj}. And let O n (n=1,2,…,N) is a binary variable, which indicates the observability of the collected data. n =1, it means that n time can be observed, otherwise if O n = 0, it means that the output at time n is unobservable. Observable data are known quantities, and unobservable data are unknown quantities.
[0045] S3, for the non-missing dataset S O Perform G-test identification and record the identified abnormal data set as S′ O , and corrected using the EM algorithm;
[0046] For the non-missing dataset S O , we select the G-test identification method commonly used in the medical field, and complete the quality judgment of energy consumption data by testing whether the observable data variable values meet the theoretical expectations. By tracking the maximum confidence level of building energy consumption data, we judge whether it meets the minimum requirements. The non-missing data sets that meet the requirements are retained, and the abnormal data sets S′ that do not meet the requirements are deleted. O , and the EM algorithm is used for correction.
[0047] S4. For missing dataset S d The missing rate is used to classify the data missing situation. The data set with a missing rate < the preset value A is recorded as data set B′, and the data set with a missing rate ≥ the preset value A is recorded as data set B″. The missing rate of building energy consumption data is set as the ratio of the number of missing data to the total amount of data.
[0048] S401: For a data set B′ whose missing rate is less than a preset value A, the EM algorithm is used to repair the energy consumption data. The details are as follows:
[0049] The unknown quantity in each subset is recorded as the implicit variable I n (n=1,2,…,N),I n Represents a discrete variable at time n, with a value of 1, 2, ..., N, and the known quantity is recorded as the observed data X i .
[0050] The EM algorithm iterates according to the E step and the M step to obtain a complete data set S that is filled and can be used for the next step of analysis. The specific steps of the EM algorithm are as follows:
[0051] The dataset B′ is taken as input and the complete dataset S after data repair is taken as output.
[0052] Step 1: Initialize the dataset B′.
[0053] Step 2: Set the number of iterations k to 0 and perform E and M steps;
[0054] Expectation step (E step): Use the current parameters to estimate the approximate Q function and the estimated parameters θ, and use the implicit variable estimate value I in the data set B′ n and observation data X i To calculate the complete data set S, the expression of the log-likelihood function is shown in (4):
[0055] Q i (I n )=P(I n |X i θ (k) ) (4)
[0056] According to the value of θ in the kth iteration, the implicit variable I is calculated n The probability of the value taken.
[0057] Maximization step (M step): Maximize the Q function to obtain a new iterative parameter, that is, solve the value of θ when the function reaches the maximum value in the following formula and update the value of θ.
[0058] θ (k+1) =argmax θ Q i (I n ) (5)
[0059] Step 3: The number of iterations k is set to k+1, and the E and M steps in Step 3 are continued until the data converges, i.e. |θ (k+1) -θ (k) | is sufficiently small. The estimated value of the implicit variable I at this time n ′ and observation data X i ’ to complete the dataset S.
[0060] S402: For the data set B″ whose missing rate is greater than or equal to the preset value A, the sparse Bayesian time series method is used to repair the building energy consumption data. The specific method is:
[0061] The time series signal of building energy consumption itself has time domain smoothness and only changes significantly at a few moments. Therefore, there are not many large data in the difference data of two adjacent samples, and most of the small differences can be ignored. Therefore, in order to obtain the difference of building energy consumption at adjacent observation points, a difference matrix M1 is designed. After completing the design of matrix M1, the j group data S″ in the data set B″ is j Projecting the M1 matrix can be expressed as:
[0062]
[0063] xj1 The elements in represent the difference in building energy consumption at adjacent observation points. j1 There are only a few large elements in , so it can be represented by the time domain smoothness matrix. Similar to the matrix M1, in order to express S″ j The acceleration matrix M2 is designed to represent the time series of acceleration changes. j Projecting the M2 matrix can be expressed as:
[0064]
[0065] x j2 Elements in (S″ ij -S″ (i-1)j )-(S″ (i+1)j -S″ ij ) can S″ j The time series represented by x is expressed as the acceleration of change. j2 Only a few elements have large values, and many elements have values that can be ignored. Then the energy consumption time series of the building can be expressed as S″ j =ψx j (j=1,2,...,K), thus obtaining the complete data set S after data recovery.
[0066] Through the above methods, a complete building energy consumption data set was obtained, which facilitates further mining and analysis of energy consumption data.
[0067] The above content is only for explaining the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.
Claims
1. A method for identifying and repairing energy consumption data based on statistical patterns, It is characterized in that The process includes the following: Divide the sample data into multiple subsets according to the data source, the sample data is the building energy consumption data record within a preset time period, and the data in the subsets respectively correspond to the data collected by each data source; Divide the data matrix of each subset into time series form, and clarify each known quantity and unknown quantity in each subset; The subsets are divided according to whether they contain missing data, and the subset with 0 unknowns is recorded as the non-missing data set S O , record the subset with the number of unknown numbers greater than 0 as the missing data set S d ; For the non-missing dataset S O Perform G-test identification and correct the identified abnormal data set using the expected maximum algorithm; For the missing data set S d , the missing data are divided according to the missing rate. For the data set with missing rate < preset value A, the EM algorithm is used to repair the energy consumption data. For the data set with missing rate ≥ preset value A, the sparse Bayesian time series method is used to repair the energy consumption data. For the data set B″ with a missing rate ≥ the preset value A, the sparse Bayesian time series method is used to repair the building energy consumption data. The process includes: The j group of data S in the data set B″ j ″For the difference matrix M 1 The matrix is projected and expressed as: xj 1 The elements in represent the difference in building energy consumption between adjacent observation points; S j ″For the acceleration matrix M 2 The matrix is projected and is expressed as: set up Then the energy consumption time series of the building can be expressed as S″ j =ψx j , thus obtaining the complete data set S after data recovery, where ψ represents ψ 1 or 2 , j = 1, 2, ..., K, where K is the number of data sources.
2. According to the method for identifying and repairing energy consumption data based on statistical patterns in claim 1, It is characterized in that The data sources include the building's lighting socket electricity consumption, air conditioning electricity consumption, power electricity consumption, special electricity consumption, temperature, humidity, relative humidity, radiation, and the number of people indoors.
3. According to the method for identifying and repairing energy consumption data based on statistical patterns in claim 1, It is characterized in that The process of clarifying each known quantity and unknown number in each subset includes: n is a binary variable, indicating the observability of the collected data, where n represents the sequence number of the time series, n = 1, 2, ..., N; if O n =1, it means that the nth moment is observable, otherwise if O n =0, it means that the output at time n is unobservable; observable data is recorded as known quantity, and unobservable data is recorded as unknown quantity.
4. The method for identifying and repairing energy consumption data based on statistical patterns according to claim 1, It is characterized in that For the non-missing dataset S O The process of performing G-test identification and correcting the identified abnormal data set using the expected maximum algorithm includes: The G-test identification method is used to complete the quality judgment of energy consumption data by checking whether the observable data variable values meet the theoretical expectations; by tracking the maximum confidence level of the data, it is judged whether the maximum confidence level meets the minimum requirements, and the non-missing data sets that meet the requirements are retained, and the abnormal data sets S′ that do not meet the requirements are retained. O , and the EM algorithm is used for correction.
5. The method for identifying and repairing energy consumption data based on statistical patterns according to claim 1, It is characterized in that The missing rate is the ratio of the number of missing data in the data set to the total number of data in the data set.
6. The method for identifying and repairing energy consumption data based on statistical patterns according to claim 1, It is characterized in that The unknown quantity in the data set B′ with missing rate < preset value A is recorded as implicit variable I n ,I n represents the discrete variable at the nth moment, and the known quantity is recorded as the observed data X i , where n represents the sequence number of the moment in the time series, n = 1, 2, ..., N; For data sets with missing rate less than the preset value A, the EM algorithm is used to repair energy consumption data. During the repair, the EM algorithm iterates according to the E step and the M step to obtain a complete data set S that is filled and can be used for the next step of analysis. The data set B′ is used as the input of the EM algorithm, and the complete data set S after data repair is used as the output, including the following steps: S1: Initialize the data set B′; S2: Set the number of iterations k to 0 and perform E and M steps; Step E: Use the current parameters to estimate the approximate Q function and the estimated parameters θ, and use the implicit variables in the data set B′ to estimate the value I n and observation data X i To calculate the complete data set S, the expression of the log-likelihood function is as follows: Q i (I n )=P(I n |X i ;θ (k) ) According to the value of θ in the kth iteration, the implicit variable I is calculated n the probability of the value taken; Step M: Maximize the Q function, obtain new iteration parameters, solve the value of θ when the function reaches the maximum value in the following equation, and update the value of θ; θ (k+1) =argmax θ Q i (I n ) S3: The number of iterations k is set to k+1, and the E and M steps in S3 are continued until the data converges, making |θ (k+1) -θ (k) | reaches the preset value; use the implicit variable estimate value I′ at this time n and observation data X′ i Go to complete the data set S and complete the repair.
7. The method for identifying and repairing energy consumption data based on statistical patterns according to claim 1, It is characterized in that The preset value A ranges from 0.4 to 0.6.