A layered cleaning method, medium and electronic equipment for building power consumption data

Abstract

The present invention proposes a layered cleaning method, medium and electronic equipment for building power consumption data, which belongs to the field of data processing. The method includes: obtaining electricity metering data and layering it into one, two and three levels according to the equipment type. For the layered data, a clustering algorithm is used to identify outliers, and false and true outliers are distinguished through inter-layer mutual verification, the former are kept unchanged, and the latter are replaced with null values. The outliers replaced with null values ​​and the original null values ​​are marked as missing values, and then divided into long-term and local missing values. A complete time series is created, and local missing values ​​are filled by interpolation. Long-term missing values ​​are predictively filled by a support vector regression algorithm, and finally the cleaned power consumption data is obtained. By introducing inter-layer mutual verification, the limitations of a single dimension are broken through, and true and false outliers are accurately distinguished through cross-level correlation, providing a reliable data basis for subsequent building energy consumption analysis and other work. It meets the needs of the building energy consumption centralized control platform and the refined management of buildings for high-quality data.

Description

Technical Field

[0001] The present invention relates to the technical field of energy consumption data processing, and in particular to a layered cleaning method, medium and electronic equipment for building power consumption data. Background Art

[0002] Energy consumption in public buildings, especially electricity consumption, is a significant source of carbon emissions. As energy management becomes increasingly sophisticated, accurate analysis of building electricity consumption data becomes crucial. However, actual building electricity consumption data collected often contains numerous outliers, missing values, and other issues, severely impacting data accuracy and reliability, and consequently, the effectiveness of energy management.

[0003] In this regard, traditional data processing methods usually use simple statistical methods, threshold comparisons, or empirical judgments to deal with outliers and missing values. However, these methods lack scientificity and systematicity, and it is difficult to effectively identify and process various complex data relationships. In particular, for building electricity consumption data, which has multi-level and multi-type characteristics, traditional processing methods often have the problem of low outlier recognition rate. In particular, building electricity consumption data is affected by many factors, such as time, weather, equipment type, etc. These factors make the data present complex nonlinear relationships, further increasing the difficulty of data processing. Therefore, a more scientific, systematic and effective building electricity consumption data cleaning method is needed to accurately identify and process outliers and missing values ​​in the data, improve data reliability, and provide support for refined energy management. Summary of the Invention

[0004] Aiming at the problems of poor quality of building sub-item metering data and low outlier recognition rate and missing value filling effect of traditional data cleaning methods that do not conform to data characteristics, the present invention systematically proposes a stratified cleaning method, medium and electronic equipment for building electricity consumption data from the aspects of data stratification, outlier recognition and inter-layer mutual verification, and missing value classification and filling, to meet the needs of building energy consumption centralized control platform and refined building management for high-quality data.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a method for layered cleaning of building power consumption data, comprising:

[0007] Obtaining building electricity metering data, and stratifying the data according to the type of building electrical equipment, including first-level stratified data, second-level stratified data, and third-level stratified data;

[0008] Clustering algorithms are used to identify outliers in each layer of data. Inter-layer verification is performed on outliers in the primary, secondary, and tertiary layered data to delineate between false and true outliers.

[0009] Keep the original data of false outliers unchanged and replace the true outliers with null values; for each layer of data, mark the null values ​​after the outliers are replaced and the null values ​​in the original data as missing values, and divide the missing values ​​into long-term missing values ​​and local missing values;

[0010] Interpolation method is used to fill local missing values, and support vector regression algorithm is used to predictively fill long-term missing values ​​to obtain the power consumption data after cleaning.

[0011] Preferably, the first-level hierarchical data includes total building electricity consumption;

[0012] The secondary hierarchical data includes electricity consumption of lighting sockets, electricity consumption of air conditioning system, electricity consumption of power system and electricity consumption of special systems;

[0013] The three-level hierarchical data subordinate to lighting socket electricity consumption includes indoor lighting electricity consumption, indoor socket electricity consumption, public area socket electricity consumption and outdoor landscape lighting electricity consumption;

[0014] The three-level hierarchical data related to indoor socket electricity consumption includes electricity consumption of cold and heat source systems, electricity consumption of air conditioning water systems, electricity consumption of air conditioning wind systems, and electricity consumption of distributed air conditioning systems;

[0015] The three-level hierarchical data pertaining to electricity consumption at public area sockets includes elevator electricity consumption, water pump electricity consumption, fan electricity consumption, and data center electricity consumption;

[0016] The three-level hierarchical data belonging to special system electricity consumption includes electricity consumption in kitchens and restaurants, electricity consumption in car charging piles, electricity consumption in mechanical garages, electricity consumption in production and operation, electricity consumption in bathing areas, and electricity consumption in rental areas.

[0017] Preferably, the clustering algorithm is used to identify outliers for each layer of data, specifically including:

[0018] Normalize the primary, secondary, and tertiary stratified data;

[0019] The time feature data and environmental feature data corresponding to the target data are used as clustering features, and are respectively combined with the first-level, second-level and third-level hierarchical data to form a first-level hierarchical clustering feature matrix, a second-level hierarchical clustering feature matrix and a third-level hierarchical clustering feature matrix; each row in the clustering feature matrix contains a sample point, and the sample point has a time feature and an environmental feature;

[0020] Clustering algorithms are used to detect outliers on the first-level, second-level, and third-level hierarchical clustering feature matrices, and the first-level, second-level, and third-level hierarchical data outliers are marked accordingly.

[0021] Preferably, the clustering algorithm is used to perform outlier detection on the primary, secondary and tertiary hierarchical clustering feature matrices respectively, and the primary, secondary and tertiary hierarchical data outliers are marked accordingly, wherein the clustering algorithm is a DBSCAN algorithm; specifically including:

[0022] Obtain sample points of the first-level, second-level, and third-level hierarchical clustering feature matrices, calculate the Euclidean distance between the sample points, and construct a distance matrix;

[0023] Based on the distance matrix, if the number of points within the neighborhood radius of the sample point is not less than the minimum number of samples in the neighborhood, the sample point is regarded as the core point;

[0024] Based on the core point, recursively search for all core points that can be reached directly or indirectly, as well as their neighboring points, to form a cluster;

[0025] According to the preset indicator function, if the sample point does not belong to the cluster, the sample point is an outlier.

[0026] Preferably, the inter-layer verification of outliers in the primary, secondary and tertiary hierarchical data to define false outliers and true outliers specifically includes:

[0027] If both the secondary stratified data and the primary stratified data at the same moment are identified as outliers, and the difference between the sum of the secondary stratified data and the primary stratified data is within the error range of the meter, then the outlier at that moment is determined to be a false outlier;

[0028] If the third-level stratified data and the second-level stratified data at the same moment are both identified as outliers, and the difference between the sum of the third-level stratified data and the second-level stratified data is within the error range of the meter, then the outlier at that moment is determined to be a false outlier;

[0029] If only one layer of the secondary stratified data and the primary stratified data at the same moment is identified as an outlier, the outlier at that moment is determined to be a true outlier;

[0030] If only one layer of the three-level stratified data and the corresponding two-level stratified data at the same moment is identified as an outlier, the outlier at that moment is determined to be a true outlier;

[0031] All other cases were considered true outliers.

[0032] Preferably, the missing values ​​are divided into long-term missing values ​​and local missing values, specifically: when the power consumption data of each layer is null for N consecutive moments or more, it is determined to be a long-term missing value; when the number of null values ​​at consecutive moments is less than N, it is determined to be a local missing value; N is a preset positive integer.

[0033] Preferably, the predictive filling of long-term missing values ​​using a support vector regression algorithm specifically includes:

[0034] According to the time points corresponding to the long-term missing values, the corresponding time features, environmental features and historical power consumption features are extracted, and the features are preprocessed;

[0035] The preprocessed features are input into the pre-trained SVR model to obtain the predicted data values ​​of each layer.

[0036] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for layered cleaning of building power consumption data described in the first aspect.

[0037] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for layered cleaning of building power consumption data described in the first aspect are implemented.

[0038] In a fourth aspect, the present invention provides a system for layered cleaning of building power consumption data, comprising:

[0039] a data stratification module configured to obtain building electricity metering data and stratify the data into first-level stratified data, second-level stratified data, and third-level stratified data according to the type of building electrical equipment;

[0040] The anomaly identification module is configured to use a clustering algorithm to identify outliers for each layer of data; perform inter-layer verification on outliers in the primary, secondary, and tertiary layered data to demarcate false outliers and true outliers;

[0041] The data partitioning module is configured to keep the original data of the false outliers unchanged and replace the true outliers with null values; for each layer of data, the null values ​​after the outliers are replaced and the null values ​​in the original data are marked as missing values, and the missing values ​​are divided into long-term missing values ​​and local missing values;

[0042] The data cleaning module is configured to use interpolation to fill in local missing values ​​and use support vector regression algorithm to predictively fill in long-term missing values ​​to obtain cleaned power consumption data.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) The present invention identifies outliers in a targeted manner based on device type, and can accurately locate problematic data. True and false outliers are distinguished through inter-layer mutual verification, avoiding single misjudgment and improving the accuracy of outlier processing. Outliers and original null values ​​are identified and classified for easy targeted processing. Interpolation is used to fill in local missing values, allowing for simple and efficient data recovery. Support vector regression algorithms are then used to predictively fill in long-term missing values, making reasonable estimates based on data patterns. Overall, the process is complete, effectively improving the quality of building power consumption data and providing reliable data support for energy consumption analysis and building management.

[0045] (2) In traditional outlier judgment methods, they are often limited to a single dimension or only analyze within the same level of data, making it difficult to fully capture the complex relationships between data. The inter-level mutual verification of the present invention breaks the convention by introducing a cross-level verification mechanism. It compares the abnormal conditions of data at different levels at the same time and makes a comprehensive judgment based on the error range of the meter. This cross-level association method fully explores the inherent logical connection between data, no longer views each level of data in isolation, and can more accurately distinguish between true and false outliers, effectively avoiding the misjudgment problem caused by simple algorithm detection or accidental errors, greatly improving the accuracy and reliability of outlier judgment.

[0046] (3) Traditional data processing methods have a low recognition rate for outliers in building power consumption data and are unable to effectively process complex data relationships. The present invention achieves high-precision recognition of outliers by adopting a clustering algorithm and combining it with an inter-layer mutual verification mechanism. The present invention can achieve a recognition rate of over 93% for building power consumption outliers, which is much higher than traditional methods, thereby ensuring the accuracy and reliability of the data.

[0047] (4) Traditional methods for addressing missing values ​​in data often lack scientificity and systematicity. This invention classifies missing values ​​into long-term missing values ​​and local missing values, and employs support vector regression and interpolation methods for predictive filling and direct filling, respectively. In particular, for long-term missing values, the invention achieves a predictive filling accuracy exceeding 95%, effectively addressing the impact of data gaps on energy management.

[0048] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their description are used to explain the present invention but do not constitute a limitation of the present invention.

[0050] Figure 1 A main flow chart of a method for layered cleaning of building power consumption data provided by an embodiment of the present invention;

[0051] Figure 2 Data acquisition and layering flow chart provided for an embodiment of the present invention;

[0052] Figure 3 A flowchart of outlier identification provided by an embodiment of the present invention;

[0053] Figure 4 A flowchart of filling in vacancies provided by an embodiment of the present invention;

[0054] Figure 5 A diagram showing the effect of filling local vacancy values ​​provided by an embodiment of the present invention;

[0055] Figure 6 This is a diagram showing the effect of long-term vacancy value prediction and filling provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0057] Example 1

[0058] like Figure 1 As shown, this embodiment discloses a method for layered cleaning of building power consumption data, comprising the following steps:

[0059] S1: Obtain building electricity metering data, and stratify the data according to the type of building electrical equipment, including first-level stratified data, second-level stratified data, and third-level stratified data;

[0060] S2: For each layer of data, clustering algorithms are used to identify outliers; outliers in the first-level, second-level, and third-level layered data are cross-checked between layers to delineate between false outliers and true outliers;

[0061] S3: Keep the original data of false outliers unchanged and replace the true outliers with null values; for each layer of data, mark the null values ​​after the outliers are replaced and the null values ​​in the original data as missing values, and divide the missing values ​​into long-term missing values ​​and local missing values;

[0062] S4: Use interpolation to fill in local missing values, and use support vector regression algorithm to predictively fill in long-term missing values ​​to obtain the cleaned power consumption data.

[0063] Next, combine Figure 1 , a layered cleaning method for building power consumption data disclosed in this embodiment is described in detail.

[0064] In S1, Figure 2As shown, building electricity metering data is obtained, and the data is divided into first-level hierarchical data, second-level hierarchical data and third-level hierarchical data according to the type of building electrical equipment. The data are all directly measured data by meters.

[0065] Specifically, the first-level hierarchical data is the total electricity consumption of the building;

[0066] Secondary hierarchical data include electricity consumption of lighting sockets, electricity consumption of air conditioning systems, electricity consumption of power systems and electricity consumption of special systems;

[0067] Each type of secondary hierarchical data has subordinate tertiary hierarchical data based on the actual electrical equipment usage of the building.

[0068] As a specific implementation, the three-level hierarchical data related to lighting socket electricity consumption includes indoor lighting electricity consumption, indoor socket electricity consumption, public area socket electricity consumption, and outdoor landscape lighting electricity consumption;

[0069] The three-level hierarchical data related to indoor socket electricity consumption includes electricity consumption of cold and heat source systems, electricity consumption of air conditioning water systems, electricity consumption of air conditioning wind systems, and electricity consumption of distributed air conditioning systems;

[0070] The three-level hierarchical data pertaining to electricity consumption at public area sockets includes elevator electricity consumption, water pump electricity consumption, fan electricity consumption, and data center electricity consumption;

[0071] The three-level hierarchical data belonging to special system electricity consumption includes electricity consumption in kitchens and restaurants, electricity consumption in car charging piles, electricity consumption in mechanical garages, electricity consumption in production and operation, electricity consumption in bathing areas, and electricity consumption in rental areas.

[0072] Before identifying outliers in power consumption data, this embodiment first divides complex building power consumption data into different levels according to equipment type. Data at different levels correspond to different power consumption systems, which can conduct targeted anomaly detection and missing value processing to avoid confusion and help accurately locate outliers and missing values.

[0073] In S2, Figure 3 As shown in the figure, outlier identification is performed on each layer of data, specifically including:

[0074] S201: normalizing the primary, secondary and tertiary stratified data;

[0075] S202: Using the time feature data and the environmental feature data corresponding to the target data as cluster features, and forming a first-level hierarchical cluster feature matrix, a second-level hierarchical cluster feature matrix, and a third-level hierarchical cluster feature matrix with the first-level, second-level, and third-level hierarchical data, respectively; each row in the cluster feature matrix contains a sample point, and the sample point has a time feature and an environmental feature;

[0076] S203: using a clustering algorithm to perform outlier detection on the first-level, second-level, and third-level hierarchical clustering feature matrices, and marking outliers of the first-level, second-level, and third-level hierarchical data.

[0077] In S201, the normalization method is specifically as follows:

[0078]

[0079] Where, is the normalized value, is the original layered data, and Indicates the maximum and minimum values ​​in the original layer data.

[0080] In S202, the time feature data and environmental feature data corresponding to the target data are used as clustering features, and are respectively combined with the first-level, second-level, and third-level hierarchical data to form a first-level hierarchical clustering feature matrix (F1), a second-level hierarchical clustering feature matrix (F2), and a third-level hierarchical clustering feature matrix (F3).

[0081] For example, suppose we want to analyze a shopping mall's building electricity consumption data at different times of the day for a month. We select a specific week's electricity consumption data as the focus of data cleaning analysis, and the building electricity metering data for that week serves as the "target data." When performing outlier detection on this target data, we use its corresponding temporal feature data (such as the specific day of the week and the number of hours per day) and environmental feature data (such as the daily outdoor temperature and weather conditions) to form a clustering feature matrix with the primary, secondary, and tertiary hierarchical data. This matrix is ​​then used to perform subsequent outlier identification and other operations.

[0082] Among them, the feature matrix can be expressed as:

[0083] X=[month, week, day, hour, outdoor temperature, indoor temperature, solar radiation intensity, sunny / cloudy / rainy conditions, stratified power consumption at the previous moment, stratified power consumption at the previous three moments, stratified power consumption at the corresponding moment of the previous day, stratified power consumption at the corresponding moment of the previous three days, stratified power consumption at the corresponding moment of the previous seven days].

[0084] Furthermore, the specific representation of the F1, F2, and F3 matrices is: each row contains the time features and environmental features of a sample.

[0085] For example, the time characteristic data adopts the month, week, day, and hour corresponding to each layered data; the environmental characteristic data adopts the outdoor temperature, indoor temperature, solar radiation intensity, and sunny / cloudy / rainy conditions.

[0086] In S203, the DBSCAN clustering algorithm is used to perform outlier detection on the F1, F2, and F3 matrices respectively, and the outliers of the first-level, second-level, and third-level hierarchical data are marked accordingly.

[0087] Specifically, the specific method of using the DBSCAN algorithm to identify outliers in the F1, F2, and F3 matrices is as follows:

[0088] For each pair of samples in the F1, F2, and F3 matrices, the Euclidean distance between them is calculated according to the following formula to form a distance matrix:

[0089]

[0090] Where p and q are two n-dimensional sample points, and are their values ​​in the i-th dimension respectively.

[0091] For each sample point, if Neps(p) ≥ min_samples, then point p is a core point. Neps(p) represents the number of points in the neighborhood radius (eps) of point p, and min_samples is the minimum number of samples in the neighborhood.

[0092] Starting from a core point p, recursively find all core points that can be reached directly or indirectly, as well as the neighborhood points of these core points, to form a cluster.

[0093] by , ,…, Represents all clusters in the data set, each cluster Is a set of points. Define an indicator function I(p∈Ci), if point p belongs to cluster Ci, then I(p∈Ci)=1; otherwise I(p∈Ci)=0.

[0094] If ∀i∈{1,2,…,k},I(p∈Ci)=0, then p is an outlier.

[0095] Furthermore, the hyperparameters of the DBSCAN algorithm, including the eps value and min_samples value, were optimized based on the F1, F2, and F3 matrices. During this process, at least 20 typical outliers were manually identified for each matrix based on actual conditions. A grid search technique was then used to iteratively optimize the hyperparameter values ​​until all typical outliers were identified by manual judgment and clustering results, resulting in the optimal hyperparameter values. The optimal hyperparameters were then used to identify all outliers in the primary, secondary, and tertiary stratified data.

[0096] In this embodiment, the DBSCAN algorithm is used to detect outliers in the stratified building electricity consumption data. This algorithm can effectively identify clusters of any shape, conform to the complex and changeable distribution characteristics of building electricity consumption data, and accurately identify outliers. Secondly, there is no need to pre-specify the number of clusters, which avoids the problem of subjectively setting the number of clusters and affecting the detection results. Furthermore, by setting hyperparameters such as the neighborhood radius and the minimum number of samples, the sensitivity of the detection can be flexibly adjusted according to the actual situation. In addition, in the process of determining the hyperparameters, iterative optimization with the help of grid search technology can ensure the accuracy of the detection and effectively identify all outliers in the stratified data at all levels, providing a reliable data foundation for subsequent data cleaning and building energy consumption analysis.

[0097] As an implementation method, inter-layer verification is performed on outliers in the primary, secondary, and tertiary hierarchical data to delineate between false outliers and true outliers, specifically including:

[0098] S211: If both the secondary stratified data and the primary stratified data at the same moment are identified as outliers, and the difference between the sum of the secondary stratified data and the primary stratified data is within the error range of the meter, then the outlier at that moment is determined to be a false outlier;

[0099] S212: If the third-level stratified data and the corresponding second-level stratified data at the same moment are both identified as outliers, and the difference between the sum of the third-level stratified data and the corresponding second-level stratified data is within the error range of the meter, then the outlier at that moment is determined to be a false outlier;

[0100] S213: If only one layer of the secondary layered data and the primary layered data at the same moment is identified as an outlier, the outlier at that moment is determined to be a true outlier;

[0101] S214: If only one layer of the three-level hierarchical data and the corresponding two-level hierarchical data at the same moment is identified as an outlier, the outlier at that moment is determined to be a true outlier;

[0102] S215: All other cases are determined to be true outliers.

[0103] This embodiment performs inter-layer verification of outliers at each level, breaking through the limitations of traditional single-dimensional outlier determination. Based on the characteristics of stratified building electricity consumption data, a multi-level correlation verification mechanism is established. By comparing anomalies between data at different levels and combining them with the error range of meter instruments, true and false outliers can be accurately distinguished. This approach fully considers the inherent logical relationships between data, effectively avoiding misjudgments caused by simple algorithmic detection or accidental errors, and improving the accuracy and reliability of outlier determination.

[0104] In S3, the original data of false outliers are kept unchanged, and true outliers are replaced with null values.

[0105] As a specific implementation method, replacing true outliers with null values ​​includes the following steps:

[0106] Create a DataFrame containing timestamps and values ​​based on Python;

[0107] Using the above inter-layer mutual verification, we define a judgment function to determine true outliers. This is to express the inter-layer mutual verification rule in the form of a judgment function. We use the apply method to apply the judgment function to each layer of the DataFrame and create a new column to store the result.

[0108] Finally, we use the loc method and Boolean indexing to find all rows marked as true outliers and replace the values ​​of each stratified data column of these rows with NaN. We use the drop method to delete the auxiliary column used to mark true outliers.

[0109] Furthermore, true outliers that are replaced with null values ​​are identified as missing values.

[0110] As a specific implementation method, marking the null values ​​in the original data as missing values ​​specifically includes:

[0111] Create a complete time series. First, create a complete time index based on the time range and appropriate time intervals (such as days, hours, etc.) to ensure the continuity of the time series. Then, realign the target data to the complete time series based on the time index. Use the reindex() method in Python to match the timestamps of the original data to the complete time axis. For unmatched time points, the data is filled with missing values ​​(NaN).

[0112] This example creates a complete time series and realigns the target data with it, allowing for a comprehensive analysis of null values ​​within the target data within a unified time dimension. This unified time framework provides a solid foundation for accurately identifying null values ​​and avoids misidentification of null values ​​due to time discontinuities or inconsistencies. Based on accurately identified null values, subsequent processing operations such as classification and filling of missing values ​​can be performed more accurately, effectively improving data quality and the reliability of subsequent analysis results.

[0113] Furthermore, the missing values ​​are divided into two categories: long-term missing values ​​and local missing values.

[0114] When the power consumption data of each layer is null for N or more consecutive moments, it is determined to be a long-term missing value; when the number of consecutive null values ​​is less than N, it is determined to be a local missing value; where N is a preset positive integer. In this embodiment, N=5 is preferred.

[0115] In S4, Figure 4As shown, the interpolation method is used to fill the local missing values, and the support vector regression algorithm is used to predictively fill the long-term missing values ​​to obtain the power consumption data after cleaning.

[0116] In this embodiment, local missing values ​​are filled using the following cubic spline interpolation method:

[0117]

[0118] Among them, f(x) is the data value after interpolation, , , , is the interpolation coefficient solved by the least squares method, is the position of each known layered data point, and x is the interpolation point. The effect of filling local vacancy values ​​is as follows Figure 5 shown.

[0119] The support vector regression algorithm is used to predictively fill long-term missing values. The specific process is as follows:

[0120] The time characteristic data, environmental characteristic data and historical power consumption data of each layer corresponding to the target data are used to form a first-level hierarchical predictive filling feature matrix X1, a second-level hierarchical predictive filling feature matrix X2 and a third-level hierarchical predictive filling feature matrix X3; wherein the historical power consumption data are the first-level, second-level and third-level hierarchical data of the previous moment, the previous three moments, the corresponding moment of the previous day, the corresponding moment of the previous three days and the corresponding moment of the previous seven days.

[0121] For each hierarchical predictive filling feature matrix, an SVR model is constructed respectively, and the objective function of SVR is:

[0122]

[0123] Among them, w is the weight vector; b is the bias term; is a linear model; C is a regularization parameter used to control the complexity and fault tolerance of the model. yes Insensitive loss function, It is a sample The corresponding true value is calculated as follows:

[0124]

[0125] Map the input data to a high-dimensional space through a kernel function (such as a Gaussian kernel or a polynomial kernel):

[0126]

[0127] in, is a kernel function. Optional kernel functions include linear kernel, polynomial kernel and radial basis function (RBF) kernel. Those skilled in the art can select according to actual needs, and the present invention does not impose any limitation on this. 、 is the Lagrange multiplier; b is the bias term; are support vectors.

[0128] Furthermore, the SVR model is trained using the X1, X2, and X3 matrices, and the optimal regularization parameter C and kernel function are selected through cross-validation.

[0129] For the missing power consumption data of each layer, the trained SVR model is used to make predictions as follows:

[0130] Extract the corresponding time features, environmental features, and historical power consumption features based on the time points corresponding to the missing data. Normalize and encode the extracted features to meet the model input requirements.

[0131] The preprocessed features are input into the SVR model to obtain the predicted data values ​​of each layer;

[0132] Fill the missing positions of each layer data with the predicted values. The long-term predictive filling effect is as follows: Figure 6 shown.

[0133] In this embodiment, the number of consecutive null values ​​is used as the criterion for classification, making the missing value classification more consistent with actual data characteristics and providing a reasonable basis for subsequent processing. For locally missing values, cubic spline interpolation is used, which accurately fits the known data point locations, smoothly filling in the gaps while preserving the local characteristics of the data. For long-term missing values, a support vector regression algorithm is used to construct a model that integrates temporal, environmental, and historical electricity consumption characteristics. Parameter optimization through cross-validation effectively uncovers underlying patterns in the data and enables accurate predictive filling. This combination improves the accuracy and adaptability of data filling, providing a high-quality data foundation for the analysis and application of building electricity consumption data.

[0134] In terms of outlier identification, this specific embodiment is based on the DBSCAN clustering algorithm and combines time and environmental features to construct a cluster feature matrix. It can effectively adapt to the complex distribution of building electricity consumption data and accurately identify outliers, which is different from the traditional single-dimensional detection method. In terms of inter-layer mutual verification, true and false outliers are defined based on the relationship between data at different levels and the error range of the meter, breaking through the traditional single judgment mode, fully considering the inherent logic of the data, reducing the misjudgment rate, and providing a reliable basis for subsequent processing. In terms of missing value classification, the number of consecutive null values ​​is used as the standard to divide them into long-term and local missing values, making the filling strategy more targeted. Interpolation and support vector regression algorithms are used for different types of missing values, respectively, to improve the accuracy and rationality of data filling. The overall solution provides a systematic and innovative method for cleaning building electricity consumption data, which is of great significance to building energy consumption management.

[0135] Example 2

[0136] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method for layered cleaning of building power consumption data as described in the first embodiment above are implemented.

[0137] Example 3

[0138] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for layered cleaning of building power consumption data as described in the first embodiment above are implemented.

[0139] Example 4

[0140] This embodiment provides a system for cleaning building power consumption data in layers, including:

[0141] a data stratification module configured to obtain building electricity metering data and stratify the data into first-level stratified data, second-level stratified data, and third-level stratified data according to the type of building electrical equipment;

[0142] The anomaly identification module is configured to use a clustering algorithm to identify outliers for each layer of data; perform inter-layer verification on outliers in the primary, secondary, and tertiary layered data to demarcate false outliers and true outliers;

[0143] The data partitioning module is configured to keep the original data of the false outliers unchanged and replace the true outliers with null values; for each layer of data, the null values ​​after the outliers are replaced and the null values ​​in the original data are marked as missing values, and the missing values ​​are divided into long-term missing values ​​and local missing values;

[0144] The data cleaning module is configured to use interpolation to fill in local missing values ​​and use support vector regression algorithm to predictively fill in long-term missing values ​​to obtain cleaned power consumption data.

[0145] The steps or modules involved in Examples 2 to 4 above correspond to those in Example 1. For detailed implementations, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media that includes one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and cause the processor to perform any method of the present invention.

[0146] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A layered cleaning method for building power consumption data, characterized in that: include: Obtaining building electricity metering data, and stratifying the data according to the type of building electrical equipment, including first-level stratified data, second-level stratified data, and third-level stratified data; For each layer of data, a clustering algorithm is used to identify outliers; inter-layer mutual verification is performed on the outliers in the first-level, second-level and third-level layered data, and by comparing the abnormal conditions of the data of different levels at the same time, and combining the error range of the meter to make a comprehensive judgment, false outliers and true outliers are delineated. Specifically, if the second-level layered data and the first-level layered data at the same time are both identified as outliers, and the difference between the sum of the second-level layered data and the first-level layered data is within the error range of the meter, then the outlier at that moment is determined to be a false outlier; if the third-level layered data and the second-level layered data to which it belongs are both identified as outliers, and the difference between the sum of the third-level layered data and the second-level layered data to which it belongs is within the error range of the meter, then the outlier at that moment is determined to be a false outlier; if only one layer of the second-level layered data and the first-level layered data at the same time is identified as an outlier, then the outlier at that moment is determined to be a true outlier; if only one layer of the third-level layered data and the second-level layered data to which it belongs is identified as an outlier at the same time, then the outlier at that moment is determined to be a true outlier; all other cases are determined to be true outliers; Keep the original data of false outliers unchanged and replace the true outliers with null values; for each layer of data, mark the null values ​​after the outliers are replaced and the null values ​​in the original data as missing values, and divide the missing values ​​into long-term missing values ​​and local missing values; Interpolation is used to fill in local missing values, and support vector regression algorithm is used to predictively fill in long-term missing values ​​to obtain the power consumption data after cleaning; the local missing values ​​are filled using the following cubic spline interpolation method: ； Among them, f(x) is the data value after interpolation, , , , is the interpolation coefficient solved by the least squares method, is the position of each known layered data point, and x is the interpolation point; The support vector regression algorithm is used to predictively fill in long-term missing values, specifically including: extracting corresponding time features, environmental features and historical power consumption features according to the time points corresponding to the long-term missing values, and preprocessing the features; the historical power consumption data is the first-level, second-level and third-level stratified data of the previous moment, the previous three moments, the corresponding moment of the previous day, the corresponding moment of the previous three days, and the corresponding moment of the previous seven days; the preprocessed features are input into a pre-trained SVR model to obtain predicted data values ​​for each layer.

2. A layered cleaning method for building power consumption data according to claim 1, characterized in that: The first-level hierarchical data includes the total electricity consumption of the building; The secondary hierarchical data includes electricity consumption of lighting sockets, electricity consumption of air conditioning system, electricity consumption of power system and electricity consumption of special systems; The three-level hierarchical data subordinate to lighting socket electricity consumption includes indoor lighting electricity consumption, indoor socket electricity consumption, public area socket electricity consumption and outdoor landscape lighting electricity consumption; The three-level hierarchical data related to indoor socket electricity consumption includes electricity consumption of cold and heat source systems, electricity consumption of air conditioning water systems, electricity consumption of air conditioning wind systems, and electricity consumption of distributed air conditioning systems; The three-level hierarchical data pertaining to electricity consumption at public area sockets includes elevator electricity consumption, water pump electricity consumption, fan electricity consumption, and data center electricity consumption; The three-level hierarchical data belonging to special system electricity consumption includes electricity consumption in kitchens and restaurants, electricity consumption in car charging piles, electricity consumption in mechanical garages, electricity consumption in production and operation, electricity consumption in bathing areas, and electricity consumption in rental areas.

3. A layered cleaning method for building power consumption data according to claim 1, characterized in that: For each layer of data, a clustering algorithm is used to identify outliers, specifically including: Normalize the primary, secondary, and tertiary stratified data; The time feature data and environmental feature data corresponding to the target data are used as clustering features, and are respectively combined with the first-level, second-level and third-level hierarchical data to form a first-level hierarchical clustering feature matrix, a second-level hierarchical clustering feature matrix and a third-level hierarchical clustering feature matrix; each row in the clustering feature matrix contains a sample point, and the sample point has a time feature and an environmental feature; Clustering algorithms are used to detect outliers on the first-level, second-level, and third-level hierarchical clustering feature matrices, and the first-level, second-level, and third-level hierarchical data outliers are marked accordingly.

4. A layered cleaning method for building power consumption data according to claim 3, characterized in that: The clustering algorithm is used to perform outlier detection on the first-level, second-level, and third-level hierarchical clustering feature matrices, and the first-level, second-level, and third-level hierarchical data outliers are marked accordingly, wherein the clustering algorithm is the DBSCAN algorithm; specifically, the following steps are included: Obtain sample points of the first-level, second-level, and third-level hierarchical clustering feature matrices, calculate the Euclidean distance between the sample points, and construct a distance matrix; Based on the distance matrix, if the number of points within the neighborhood radius of the sample point is not less than the minimum number of samples in the neighborhood, the sample point is regarded as the core point; Based on the core point, recursively search for all core points that are directly or indirectly reached, as well as their neighboring points, to form a cluster; According to the preset indicator function, if the sample point does not belong to the cluster, the sample point is an outlier.

5. A layered cleaning method for building power consumption data according to claim 1, characterized in that: The missing values ​​are divided into long-term missing values ​​and local missing values. Specifically, when the power consumption data of each layer is null for N or more consecutive moments, it is determined to be a long-term missing value; when the number of null values ​​at consecutive moments is less than N, it is determined to be a local missing value; N is a preset positive integer.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for layered cleaning of building power consumption data according to any one of claims 1 to 5 are implemented.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the method for layered cleaning of building power consumption data according to any one of claims 1 to 5 are implemented.

8. A building power consumption data layered cleaning system, based on the building power consumption data layered cleaning method according to claim 1, characterized in that: include: a data stratification module configured to obtain building electricity metering data and stratify the data into first-level stratified data, second-level stratified data, and third-level stratified data according to the type of building electrical equipment; The anomaly identification module is configured to use a clustering algorithm to identify outliers for each layer of data; perform inter-layer verification on outliers in the primary, secondary, and tertiary layered data to demarcate false outliers and true outliers; The data partitioning module is configured to keep the original data of the false outliers unchanged and replace the true outliers with null values; for each layer of data, the null values ​​after the outliers are replaced and the null values ​​in the original data are marked as missing values, and the missing values ​​are divided into long-term missing values ​​and local missing values; The data cleaning module is configured to use interpolation to fill in local missing values ​​and use support vector regression algorithm to predictively fill in long-term missing values ​​to obtain cleaned power consumption data.

Images