A method for completing missing values ​​in daily load data of electricity users

Abstract

This invention relates to the field of power load technology and discloses a method for completing missing values ​​in daily load data of power users. The method includes acquiring meteorological data and date information data of each user in the same user cluster; integrating the meteorological data, date information data, historical electricity consumption data, and feature vector data of each user to form a training dataset corresponding to the user cluster; training a data completion model corresponding to the user cluster based on the training dataset; outputting the corresponding data completion model and associating it with the corresponding user cluster after training; inputting the meteorological information data, date information data, and feature vector data of a user in the user cluster into the data completion model associated with the user cluster, outputting the missing values ​​of that user, and filling the missing values ​​into the user's historical electricity consumption data to form the filled historical electricity consumption data.

Description

Technical Field

[0001] This invention relates to the field of power load technology, and more specifically to a method for completing missing values ​​in daily load data of power users. Background Technology

[0002] In the power industry, accurate and complete electricity consumption data is of immeasurable value to the operation and planning of electricity retailers. Based on detailed historical electricity consumption data, retailers need to accurately predict future electricity demand, thereby enabling them to rationally formulate marketing strategies, optimize power resource allocation, and control costs.

[0003] However, the collection of electricity consumption data from power users currently faces numerous severe challenges. On the one hand, many power users use outdated metering devices that cannot achieve automatic data collection. This makes data acquisition reliant on manual operation, which is not only inefficient but also prone to human error, leading to incomplete and delayed data collection. On the other hand, data omissions occur frequently, further exacerbating the problem of missing electricity consumption data.

[0004] To address the issue of missing historical electricity load data for power users, existing technologies often employ similar-day methods or predictive algorithms for imputation on a per-user basis. However, the similar-day method has significant drawbacks. It struggles to effectively capture the changing trends of electricity consumption data within a short period before and after the missing date, resulting in imputed data that fails to accurately reflect actual electricity consumption. This, in turn, affects the accuracy of electricity retailers' forecasts of future electricity demand, leading to poor data imputation results when faced with excessively large missing data or data scarcity for a single user. Conversely, predictive algorithms, applied only to individual users, are also susceptible to data shortages or excessively large missing data for those users, resulting in significant errors during imputation, even after successful completion.

[0005] Therefore, there is an urgent need for a method to complete the missing values ​​of daily load data for electricity users, which can accurately and reliably fill in the missing values ​​of the electricity load data corresponding to users. Summary of the Invention

[0006] One of the objectives of this invention is to provide a method for completing missing values ​​in daily load data of electricity users, which can accurately and reliably fill in the missing values ​​in the electricity load data corresponding to users.

[0007] To achieve the above objectives, a method for completing missing values ​​in daily load data of electricity users is provided, including the following steps:

[0008] S1. Obtain historical electricity consumption data for each user within a specific historical time period from the power trading center;

[0009] S2. Perform data cleaning and standardization operations on the historical electricity consumption data corresponding to each user in sequence;

[0010] S3. Based on the historical electricity consumption data corresponding to each user after standardization, extract the features corresponding to each user and generate feature vector data corresponding to each user. The feature vector data includes the user's data domain feature data, time domain feature data and frequency domain feature data.

[0011] S4. Based on the data domain feature data, time domain feature data and frequency domain feature data corresponding to each user, cluster each user according to the preset feature clustering strategy to form multiple user clusters;

[0012] S5. Obtain the meteorological data and date information data corresponding to each user in the same user aggregation class, and integrate the meteorological data and date information data corresponding to each user with the historical electricity consumption data and feature vector data corresponding to the user to form the training dataset data corresponding to the user aggregation class.

[0013] S6. Based on the training dataset data corresponding to the same user aggregation class, and based on the pre-built data completion model and the model training strategy corresponding to the data completion model, train the data completion model corresponding to the user aggregation class. After training, output the corresponding data completion model and associate it with the corresponding user aggregation class.

[0014] S7. Take the meteorological information data, date information data and feature vector data of a user in S3 as input data, input them into the data completion model associated with the user's aggregation class, output the missing value corresponding to the user, and fill the missing value into the user's historical electricity consumption data to form the filled historical electricity consumption data.

[0015] The technical principles and effects of this solution: This solution obtains historical electricity consumption data from the power trading center. This is the foundation of the entire solution; comprehensive and accurate data acquisition provides the raw materials for subsequent analysis. The electricity consumption data of different users over a certain historical period constitutes a complex dataset, which contains useful information but may also contain noise and incomplete parts.

[0016] The acquired historical electricity consumption data is cleaned to remove errors, duplicates, and outliers, ensuring data quality. Standardization transforms the data to a uniform scale, eliminating the influence of different dimensions and making subsequent data analysis and model training more accurate and effective.

[0017] Based on the standardized data, specific strategies are used to extract user features. Through these feature extractions, the raw electricity consumption data is transformed into more representative feature vector data, providing more effective input for subsequent clustering and model training.

[0018] By utilizing the feature vector data of each user and applying a specific clustering algorithm, users with similar characteristics are grouped together to form multiple user clusters. The purpose of this is to group users with similar electricity consumption behaviors together, enabling more accurate training and application of data completion models tailored to the characteristics of different user categories.

[0019] For each user cluster, relevant meteorological data (such as temperature, humidity, precipitation, etc.) and date information (year, month, day of the week, whether it is a holiday, etc.) are obtained. This external data may be related to the user's electricity consumption behavior. For example, in hot weather, residential users' air conditioning electricity consumption may increase; during holidays, commercial users' electricity consumption patterns may differ from weekdays. Integrating this meteorological and date information data with the user's historical electricity consumption data enriches the feature dimensions of the training dataset, enabling the model to learn more factors affecting electricity consumption data, thereby improving the accuracy of data completion.

[0020] The system takes meteorological information, date information, and feature vector data of a specific user from a user cluster as input and feeds them into a data completion model associated with that user cluster. Based on patterns learned during previous training, the model predicts missing electricity consumption data and outputs the missing values. Finally, these missing values ​​are filled into the user's historical electricity consumption data, completing the data completion process.

[0021] This solution categorizes users in the transaction center by extracting features and clustering their historical electricity consumption data, grouping users with similar electricity consumption behaviors into the same cluster. This allows the model to learn and predict based on the commonalities of similar user groups during subsequent data completion, significantly improving the accuracy of missing value prediction compared to the single-user similarity day method. The completion model built using clustered multi-user data further enhances the robustness and completion effect of the data completion model.

[0022] Meteorological and date information data for each user within the same user cluster are integrated with historical electricity consumption data and feature vector data to form a training dataset. Meteorological and date information significantly influences user electricity consumption behavior. Incorporating this multivariate data into the training allows the data completion model to learn more comprehensive influencing factors, thereby more accurately predicting missing values.

[0023] Based on the training dataset corresponding to user aggregation classes, a pre-built data completion model and model training strategy are used for training. Through extensive data training and optimization, the data completion model can adapt to the electricity consumption characteristics of different user aggregation classes, improving the model's generalization ability and stability. Compared with existing technologies, this method reduces inaccurate completion caused by individual abnormal data or single user characteristics through multi-user aggregation and extensive data training, ensuring the reliability of data completion. It also reduces errors during data completion by combining the data characteristics of a user class. In short, it achieves accurate and reliable completion of missing values ​​in the electricity load data corresponding to users.

[0024] Furthermore, S2 includes:

[0025] S20. Based on the preset row-by-row traversal strategy, perform row-by-row traversal of each electricity consumption data in the historical electricity consumption data corresponding to each user.

[0026] The preset row-by-row traversal strategy is as follows:

[0027] Each row of electronic data is identified and judged to determine whether there are any missing data in each row. If so, the starting index of the missing value is recorded. When the next row of non-missing data is encountered, the previously recorded starting index is combined with the previous index of the current non-missing data row to form a missing interval, which is recorded. The starting index is then reset, and the search for the next missing interval continues until all electronic data is identified and judged, and the missing interval set corresponding to the user is output.

[0028] Based on the missing interval set corresponding to the user, obtain the electricity consumption data of the previous day and the next day for each missing interval, calculate the electricity consumption ratio between the electricity consumption data of the next day of the missing interval and the electricity consumption data of the previous day of the missing interval, and determine whether the electricity consumption ratio is greater than the preset ratio. If it is, it is determined that there is data stacking in the missing interval; otherwise, it is determined that there is no data stacking in the missing interval.

[0029] S21. When the judgment result indicates that there is data stacking in the missing interval, the corresponding missing interval is cleaned to remove the data stacking.

[0030] S22. After cleaning the data stacking, outliers are eliminated from each user's electronic data.

[0031] S23. Standardize the electronic data of each user after excluding outliers.

[0032] Beneficial Effects: A pre-defined row-by-row traversal strategy allows for meticulous review of historical electricity consumption data. This strategy accurately identifies missing values ​​within data rows and precisely locates missing intervals through a clever indexing method. This precise location provides a reliable foundation for subsequent analysis of data before and after the missing interval, avoiding erroneous judgments due to inaccurate missing value location. For example, in complex electricity consumption datasets, it can accurately identify the start and end positions of consecutive missing values, providing an accurate data range for judging data stacking. After determining the missing interval, calculating the ratio of electricity consumption data for the day before and after the missing interval and comparing it with a pre-defined ratio can effectively determine whether data stacking exists. Data stacking can severely affect the accuracy of data and the reliability of analysis results. Accurately identifying data stacking in this way allows for timely detection and handling of such data problems, ensuring data accuracy. For instance, if the electricity consumption is abnormally high the day after a missing interval, comparing the calculated ratio with a pre-defined ratio can determine whether it is due to data stacking, avoiding the use of such erroneous data in subsequent analysis.

[0033] After cleaning the data stack, outliers are removed from the electricity consumption data of each user. Removing outliers makes the data distribution more reasonable and enhances data stability.

[0034] Furthermore, the preset feature clustering strategy is as follows:

[0035] Step 1: Based on the data domain feature data, time domain feature data, and frequency domain feature data corresponding to each user, integrate the feature vector data corresponding to each user to form the feature matrix X corresponding to all users;

[0036]

[0037] In the formula, [x i1 ,x i2 ,…,x in [x] represents the feature vector corresponding to user i. in This represents the dimension value corresponding to the nth dimension of user i in the corresponding feature vector data;

[0038] Step 2: Determine the number of clusters K corresponding to this clustering;

[0039] Step 3: Based on the feature matrix corresponding to all users, and based on the number of clusters K, randomly select K users from the m users as the cluster centers for this clustering;

[0040] Step 4: Based on the feature matrix corresponding to the remaining users, and using the preset user power distance calculation formula, calculate the user power distance between the remaining users and the users corresponding to the K cluster centers.

[0041] The preset formula for calculating the user power distance is:

[0042]

[0043] In the formula, d (x,u) x represents the power distance between the user and the user corresponding to the cluster center. ij Let u be the dimension value corresponding to the j-th dimension of user i. j This is the dimension value corresponding to the j-th dimension of the user corresponding to the cluster center;

[0044] Step 5: Based on the user power distance between each remaining user and the user corresponding to each cluster center, compare the user power distance between the remaining user and the user corresponding to each cluster center, and classify the user corresponding to the cluster center with the smallest user power distance into the same category as the remaining user, forming the initial aggregation class corresponding to each cluster center.

[0045] Step 6: Based on the power distance of each user in the initial aggregation class, calculate the arithmetic mean of each dimension of the user in each initial aggregation class, obtain the center point of each initial aggregation class, calculate and compare the power distance between each user in each initial aggregation class and the center point of each initial aggregation class, and classify each user into the aggregation class corresponding to the center point with the smallest power distance.

[0046] Step 7: After completing the user adjustment for all cluster aggregation classes, output the users of the current aggregation class corresponding to each cluster center to form the user aggregation class corresponding to each cluster center;

[0047] Step 8: After completing the formation of the user aggregation class corresponding to each cluster center in this clustering, perform the next round of clustering and update the number of clusters corresponding to the next round of clustering. Repeat step 3 until the number of clusters reaches the preset threshold.

[0048] Step 9: Based on the user aggregation class corresponding to each cluster center under each number of clusters, calculate the loss function value corresponding to each number of clusters according to the preset loss function calculation formula, and determine and output the optimal number of clusters and the user aggregation class corresponding to each cluster center under the optimal number of clusters.

[0049] Beneficial effects: By calculating the distance between users' electricity consumption, users with the lowest similarity are grouped together with the cluster center. This allows for the reasonable classification of users with similar electricity consumption behaviors into the same category, accurately identifying different user groups. For example, in an electricity consumption dataset containing users from various industries, this method can accurately distinguish between industrial and commercial users because their electricity consumption characteristics differ in the feature matrix, and they can be correctly classified through similarity calculation.

[0050] By continuously updating the number of clusters and performing multiple rounds of clustering, and evaluating the clustering effect using a loss function calculation formula, the optimal number of clusters and the corresponding user clusters can be found. The loss function can quantify the quality of the clustering results. By iteratively adjusting the number of clusters, the model can continuously optimize the clustering results, making them more consistent with the inherent structure of the data. For example, in the analysis of electricity user behavior, the optimal clustering results found in this way make the electricity consumption behavior of users within each cluster more similar, while making the differences in electricity consumption behavior between different clusters more obvious, thereby improving the performance of subsequent data analysis and model training based on the clustering results.

[0051] In the initial clustering stage, although user groups are divided by randomly selecting cluster centers and calculating based on user electricity distance, the randomness of cluster center selection and the complexity of actual electricity consumption data inevitably lead to some users being misclassified. For example, some users' electricity consumption behavior may be similar to that of a certain cluster center in some characteristics, but from the perspective of overall multi-dimensional features, they are more suitable to belong to other clusters. Without user removal and secondary partitioning, these misclassified users will interfere with the consistency of data within clusters, making the clustering results unable to accurately reflect the users' true electricity consumption behavior patterns.

[0052] By generating centroids by taking the arithmetic mean of the feature vectors of all users within the current cluster, the algorithm can comprehensively reflect the overall behavioral patterns of users in that cluster. The centroids, acting as virtual users, have features that are the average of the features of all users in the cluster, thus offering greater universality. Through multiple rounds of iterative adjustments to the centroids, the algorithm can progressively optimize the clustering results, reducing the local optima problem caused by initially randomly selecting cluster centers.

[0053] Furthermore, S6 includes:

[0054] S60. Divide the dataset information corresponding to the same user aggregation class into a training set and a validation set according to a preset ratio;

[0055] S61. Based on the pre-built data completion model, determine the parameter search range corresponding to each hyperparameter of the data completion model;

[0056] S61. Based on the parameter search range corresponding to each hyperparameter, randomly generate several hyperparameter groups to form a data completion model corresponding to each hyperparameter group.

[0057] S62. Take the meteorological data, date information data and feature vector data in the training set as input data, input them into the data completion model corresponding to each hyperparameter group, predict the prediction data corresponding to each hyperparameter group as the prediction value, and use the corresponding electronic data in the training set as the target value.

[0058] S63. Based on the predicted values ​​and target values ​​corresponding to each hyperparameter group, and using the preset loss value calculation formula, calculate the loss value of the data completion model corresponding to each hyperparameter group.

[0059] The preset formula for calculating the loss value is as follows:

[0060]

[0061] In the formula, MSE is the loss value, and y i For the target value, The value is the predicted value, and n is the number of samples in the dataset.

[0062] S64. Based on the loss value of the data completion model corresponding to each hyperparameter group, determine whether each loss value is less than the preset first loss threshold. If not, mutate all hyperparameter groups to form multiple new hyperparameter groups and corresponding data completion models, and re-execute S62.

[0063] If so, select the corresponding hyperparameter set, mutate the hyperparameter set to form multiple new hyperparameter sets and corresponding data completion models, re-execute S62 to S63, and determine whether the loss value calculated in S63 is less than the loss value corresponding to the hyperparameter set. If not, the hyperparameter set is the optimal hyperparameter set, and the data completion model corresponding to the hyperparameter set is output; if so, the corresponding new hyperparameter set is the optimal hyperparameter set, and the data completion model corresponding to the new hyperparameter set is output.

[0064] S65. Take the meteorological data, date information data and feature vector data obtained by the user in S3 as input data, input them into the data completion model corresponding to the corresponding hyperparameter group of the output, output the corresponding prediction data, and calculate the corresponding prediction error based on the user sub-data in the validation set. Determine whether the prediction error is less than the preset error threshold. If it is, the data completion model corresponding to the corresponding hyperparameter group of the output is a feasible model. Otherwise, it is an infeasible model, and S61 is executed again.

[0065] Beneficial effects: After initially calculating the loss value of the data-completed model corresponding to each hyperparameter group, if no loss value is less than the preset first loss threshold, it indicates that the current hyperparameter group may not have reached an optimal state. Mutating all hyperparameter groups at this point can generate entirely new hyperparameter combinations. Each hyperparameter may generate new values ​​within its search range, and the recombination of different hyperparameter values ​​greatly expands the hyperparameter search space. What might have been limited to finding optimal solutions in certain local regions can be explored through mutation, allowing access to more previously unexplored hyperparameter combination regions, thus significantly increasing the likelihood of finding the optimal hyperparameter group that minimizes the model's loss value.

[0066] Even if a hyperparameter set has a loss value lower than a preset first loss threshold, it is still essential to mutate it. This is because the initially found seemingly optimal hyperparameter set may only be a local optimum, performing well within a certain local region of the model's parameter space, but not globally optimal. Mutating this hyperparameter set can break the current local optimum and explore other potentially better parameter combinations. This allows the model to escape the limitations of local optima and adapt to more complex and varied data features and patterns. Attached Figure Description

[0067] Figure 1 This is a flowchart of the method for completing missing values ​​in daily load data of power users in Embodiment 1 of the present invention;

[0068] Figure 2 This is a graph showing the change in the number of clusters and the value of the loss function in Embodiment 1 of the present invention.

[0069] Figure 3 This is a diagram showing the effect of missing value completion in Embodiment 1 of the present invention. Detailed Implementation

[0070] The following detailed description illustrates the specific implementation method:

[0071] Example 1

[0072] A method for completing missing values ​​in daily load data of electricity users, basically as follows: Figure 1 As shown, it includes the following steps:

[0073] S1. Obtain historical electricity consumption data for each user within a certain historical period from the power trading center; for example, obtain historical electricity consumption data for users from the trading center, with a total of 1147 user numbers and a time period from January 1, 2023 to October 31, 2024.

[0074] S2. Perform data cleaning and standardization operations on the historical electricity consumption data corresponding to each user in sequence;

[0075] S2 includes:

[0076] S20. Based on the preset row-by-row traversal strategy, perform row-by-row traversal of each electricity consumption data in the historical electricity consumption data corresponding to each user.

[0077] The preset row-by-row traversal strategy is as follows:

[0078] Each row of electronic data is identified and judged to determine whether there are any missing data in each row. If so, the starting index of the missing value is recorded. When the next row of non-missing data is encountered, the previously recorded starting index is combined with the previous index of the current non-missing data row to form a missing interval, which is recorded. The starting index is then reset, and the search for the next missing interval continues until all electronic data is identified and judged, and the missing interval set corresponding to the user is output.

[0079] Based on the missing interval set corresponding to the user, obtain the electricity consumption data of the previous day and the next day for each missing interval, calculate the electricity consumption ratio between the electricity consumption data of the next day of the missing interval and the electricity consumption data of the previous day of the missing interval, and determine whether the electricity consumption ratio is greater than the preset ratio. If it is, it is determined that there is data stacking in the missing interval; otherwise, it is determined that there is no data stacking in the missing interval.

[0080] S21. When the judgment result indicates that there is data stacking in the missing interval, the corresponding missing interval is cleaned to remove the data stacking.

[0081] S22. After cleaning the data stacking, outliers are eliminated from each user's electronic data.

[0082] S23. Standardize the electronic data of each user after excluding outliers.

[0083] S3. Based on the standardized historical electricity consumption data of each user, extract the features corresponding to each user to generate feature vector data for each user. The feature vector data includes user data domain feature data, time domain feature data, and frequency domain feature data. The data domain feature data includes data on the user's district / county, industrial and commercial information, and the ratio data between demand-based billing and capacity-based billing. Demand-based billing is an electricity billing method based on the user's actual maximum electricity demand. Capacity-based billing is an electricity billing method based on the rated capacity of the user's transformer. During feature extraction, some raw data is first converted into numerical labels that can characterize the user's electricity consumption behavior for subsequent machine learning analysis. The name of the user's district / county is mapped to a unique code through address classification, serving as a feature of district / county information. User types are defined, with 1 representing commercial electricity consumption and 2 representing industrial electricity consumption to represent industrial and commercial types. Billing methods are defined, with 3 representing demand-based billing and 4 representing capacity-based billing.

[0084] The time-domain feature data includes the user's corresponding maximum electricity consumption, maximum absolute electricity consumption, minimum electricity consumption, average electricity consumption, peak-to-peak electricity consumption, absolute square value of electricity consumption, root mean square value of electricity consumption, electricity amplitude, standard deviation of electricity consumption, kurtosis of electricity consumption, skewness of electricity consumption, margin index, impulse index, and peak value index; the calculation formulas corresponding to the time-domain feature data are shown in Table 1:

[0085]

[0086]

[0087] Table 1

[0088] In this embodiment, under stable power consumption conditions, the maximum value, maximum absolute value (which can also be considered as peak value), or minimum value varies little and is basically stable below a threshold. However, once the maximum value, maximum absolute value, or minimum value abnormally increases or decreases, it can be generally considered that the user's power consumption has changed. A change to a certain extent indicates some kind of statistical error. The mean reflects the change in the vibration signal caused by changes in power consumption during the user's power consumption process. The root mean square (RMS), also known as the effective value, reflects the energy intensity and stability of the vibration signal. Power load forecasting typically focuses on this indicator; if it becomes abnormally large, it indicates that the power consumption characteristics are strongly interfered with by some external factor. Kurtosis reflects the impact characteristics of the vibration signal. Kurtosis is sensitive to impact; generally, the kurtosis value should be around 3 because the kurtosis of a normal distribution is equal to 3. If it deviates too much from 3, it indicates that the power user's power consumption habits are easily interfered with by external factors. Skewness reflects the asymmetry of the vibration signal. Normally, the vibration signal is symmetrical about the x-axis, in which case the skewness should approach 0. If the periodic average value of electricity consumption shifts upward or downward, the skewness increases. Margin indicators are used to measure the rate of change of periodic electricity consumption. Impulse indicators and peak indicators are both used to detect the presence of impulses in a signal.

[0089] The frequency domain feature data includes the power consumption centroid frequency, the root mean square frequency of power consumption, the average frequency of power consumption, and the variance of power consumption. The calculation formulas for each data point corresponding to the frequency domain feature data are shown in Table 2.

[0090]

[0091] Table 2

[0092] In the table, f k Let S(k) be the frequency value of the k-th component, and S(k) be the weight value of the k-th component.

[0093] S4. Based on the data domain feature data, time domain feature data and frequency domain feature data corresponding to each user, cluster each user according to the preset feature clustering strategy to form multiple user clusters;

[0094] The preset feature clustering strategy is as follows:

[0095] Step 1: Based on the data domain feature data, time domain feature data, and frequency domain feature data corresponding to each user, integrate the feature vector data corresponding to each user to form the feature matrix X corresponding to all users;

[0096]

[0097] In the formula, [x i1 ,x i2,…,x in [x] represents the feature vector corresponding to user i. in This represents the dimension value corresponding to the nth dimension of user i in the corresponding feature vector data. In this embodiment, the dimensions of the data corresponding to the domain feature data, time domain feature data, and frequency domain feature data are determined. For example, the data of the user's district / county is the first dimension, and industrial electricity consumption is the second dimension.

[0098] Step 2: Determine the number of clusters K corresponding to this clustering;

[0099] Step 3: Based on the feature matrix corresponding to all users, and based on the number of clusters K, randomly select K users from the m users as the cluster centers for this clustering;

[0100] Step 4: Based on the feature matrix corresponding to the remaining users, and using the preset user power distance calculation formula, calculate the user power distance between the remaining users and the users corresponding to the K cluster centers.

[0101] The preset formula for calculating the user power distance is:

[0102]

[0103] In the formula, d (x,u) x represents the power distance between the user and the user corresponding to the cluster center. ij Let u be the dimension value corresponding to the j-th dimension of user i. j This is the dimension value corresponding to the j-th dimension of the user corresponding to the cluster center;

[0104] Step 5: Based on the user power distance between each remaining user and the user corresponding to each cluster center, compare the user power distance between the remaining user and the user corresponding to each cluster center, and classify the user corresponding to the cluster center with the smallest user power distance into the same category as the remaining user, forming the initial aggregation class corresponding to each cluster center.

[0105] Step 6: Based on the power distance of each user in the initial aggregation class, calculate the arithmetic mean of each dimension of the user in each initial aggregation class, obtain the center point of each initial aggregation class, calculate and compare the power distance between each user in each initial aggregation class and the center point of each initial aggregation class, and classify each user into the aggregation class corresponding to the center point with the smallest power distance.

[0106] Step 7: After completing the user adjustment for all cluster aggregation classes, output the users of the current aggregation class corresponding to each cluster center to form the user aggregation class corresponding to each cluster center;

[0107] Step 8: After completing the formation of the user aggregation class corresponding to each cluster center in this clustering, perform the next round of clustering and update the number of clusters corresponding to the next round of clustering. Repeat step 3 until the number of clusters reaches the preset threshold.

[0108] Step 9: Based on the user aggregation class corresponding to each cluster center under each number of clusters, calculate the loss function value corresponding to each number of clusters according to the preset loss function calculation formula, and determine and output the optimal number of clusters and the user aggregation class corresponding to each cluster center under the optimal number of clusters.

[0109] The preset loss function calculation formula is as follows:

[0110]

[0111] In the formula, J is the loss function value, and c (d) For data point x (d) The cluster category, where d ranges from 1 to m, corresponds to m data points, and each data point is assigned to one of K clusters, c (d) This is the label used to identify the cluster to which the data point belongs; m is the total number of data points, which in this clustering scenario is the number of all user feature vectors participating in the clustering. It is used as a divisor when calculating the average loss value to measure the overall data size. μ1,…,μ K This is the center vector of each of the K clusters, μ K (k = 1, 2, ..., K) represents the center of the k-th cluster, which is the mean of the feature vectors of all data points within that cluster, reflecting the central tendency of the data in that cluster. (d) Let be the feature vector of the i-th data point, corresponding to the integrated feature vector of each user. It contains multi-dimensional information such as data domain features, time domain features, and frequency domain features. For data point x (d) To its cluster center The square of the Euclidean distance. In this embodiment, the elbow method is used to determine the optimal number of clusters in the classification of different clusters, that is, the number of clusters is determined by the elbow point, such as... Figure 2 As shown, the number of clusters K is represented by the x-axis, and the loss function value is represented by the y-axis. Among the classification parameters of different clusters, an "elbow" appears at K=5, where the rate of decrease in the sum of squares within the cluster slows down significantly. Therefore, the optimal number of clusters for K-means clustering is determined to be 5.

[0112] For example, five clusters are formed, and the user information corresponding to each cluster is shown in Table 3:

[0113] cluster Total number of users Complete data user count Number of users with missing values 1 261 198 63 2 644 473 171 3 341 289 52 4 284 261 23 5 596 440 156

[0114] Table 3

[0115] S5. Obtain meteorological data and date information data corresponding to each user in the same user aggregation class, and integrate the meteorological data and date information data corresponding to each user with the historical electricity consumption data and feature vector data corresponding to the user to form the training dataset data corresponding to the user aggregation class; In this embodiment, the meteorological data includes temperature, humidity and precipitation information corresponding to the user's location information, and the date information data includes year, month, week number, day of the month, day of the week, whether it is a statutory holiday, and whether it is a holiday adjusted by work.

[0116] S6. Based on the training dataset data corresponding to the same user aggregation class, and based on the pre-built data completion model and the model training strategy corresponding to the data completion model, train the data completion model corresponding to the user aggregation class. After training, output the corresponding data completion model and associate it with the corresponding user aggregation class.

[0117] S6 includes:

[0118] S60. Divide the dataset information corresponding to the same user aggregation class into a training set and a validation set according to a preset ratio;

[0119] S61. Based on the pre-built data completion model, determine the parameter search range corresponding to each hyperparameter of the data completion model;

[0120] S61. Based on the parameter search range corresponding to each hyperparameter, randomly generate several hyperparameter groups to form a data completion model corresponding to each hyperparameter group.

[0121] S62. The meteorological data, date information data and feature vector data in the training set are used as input data and input into the data completion model corresponding to each hyperparameter group. The predicted data corresponding to each hyperparameter group is predicted as the predicted value, and the corresponding electronic data in the training set is used as the target value.

[0122] S63. Based on the predicted values ​​and target values ​​corresponding to each hyperparameter group, and using the preset loss value calculation formula, calculate the loss value of the data completion model corresponding to each hyperparameter group.

[0123] The preset formula for calculating the loss value is as follows:

[0124]

[0125] In the formula, MSE is the loss value, and y i For the target value, The value is the predicted value, and n is the number of samples in the dataset.

[0126] S64. Based on the loss value of the data completion model corresponding to each hyperparameter group, determine whether each loss value is less than the preset first loss threshold. If not, mutate all hyperparameter groups to form multiple new hyperparameter groups and corresponding data completion models, and re-execute S62.

[0127] If so, select the corresponding hyperparameter set, mutate the hyperparameter set to form multiple new hyperparameter sets and corresponding data completion models, re-execute S62 to S63, and determine whether the loss value calculated in S63 is less than the loss value corresponding to the hyperparameter set. If not, the hyperparameter set is the optimal hyperparameter set, and the data completion model corresponding to the hyperparameter set is output; if so, the corresponding new hyperparameter set is the optimal hyperparameter set, and the data completion model corresponding to the new hyperparameter set is output.

[0128] S65. Take the meteorological data, date information data and feature vector data obtained by the user in S3 as input data, input them into the data completion model corresponding to the corresponding hyperparameter group of the output, output the corresponding prediction data, and calculate the corresponding prediction error based on the user sub-data in the validation set. Determine whether the prediction error is less than the preset error threshold. If it is, the data completion model corresponding to the corresponding hyperparameter group of the output is a feasible model. Otherwise, it is an infeasible model, and S61 is executed again.

[0129] S7. Taking the meteorological information data, date information data, and feature vector data corresponding to a user in the user aggregation class as input data, inputting them into the data completion model associated with that user aggregation class, outputting the missing values ​​corresponding to that user, and filling the missing values ​​into the user's historical electricity consumption data to form the filled historical electricity consumption data. For example... Figure 3 As shown, users with complete data from June 1, 2024 to October 31, 2024 were used as the validation set, and a portion of users were selected as the test set. Missing values ​​were randomly placed, and after being filled in through the above steps, the results were merged and compared with the validation set. R = ratio (true value / test value). In the algorithm's filling results, the deviation rate was controlled within ±2%.

[0130] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, based on the guidance provided in this application, improve and implement this solution in combination with their own capabilities. Some typical well-known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for completing missing values of daily load data of an electric power user, characterized by: The method comprises the following steps: S1, obtaining historical power consumption data corresponding to each user in a certain historical time period from a power transaction center; S2, sequentially performing data cleaning operation and standardization processing on the obtained historical power consumption data corresponding to each user; S3, extracting features corresponding to each user according to the historical power consumption data corresponding to each user after standardization processing, generating feature vector data corresponding to each user, wherein the feature vector data comprises data field feature data, time domain feature data and frequency domain feature data of the user; S4, clustering each user based on a preset feature clustering strategy according to the data field feature data, time domain feature data and frequency domain feature data corresponding to each user, forming a plurality of user aggregation classes; S5, obtaining meteorological data and date information data corresponding to each user in the same user aggregation class, and integrating the meteorological data and date information data corresponding to each user with the historical power consumption data and feature vector data corresponding to the user, forming training data set data corresponding to the user aggregation class; S6, training the data completion model corresponding to the same user aggregation class based on the pre-constructed data completion model and the model training strategy corresponding to the data completion model, and outputting the corresponding data completion model after training and associating with the corresponding user aggregation class; The model training strategy corresponding to the data completion model comprises varying the hyperparameters to form a plurality of new hyperparameter groups and corresponding data completion models; S7, inputting the meteorological information data, date information data and feature vector data obtained by the user in S3 into the data completion model associated with the user aggregation class, outputting the missing values corresponding to the user, and filling the missing values into the historical power consumption data of the user to form the filled historical power consumption data.

2. The method of claim 1, wherein the method comprises: The S2 comprises: S20, based on a preset row-by-row traversal strategy, performing row-by-row traversal on each power consumption data in the historical power consumption data corresponding to each user; The preset row-by-row traversal strategy is: performing row-by-row identification and judgment on each power consumption data, judging whether there is a missing value in each power consumption data in the data row, if yes, recording the starting index of the missing value, when encountering the next non-missing value data row, forming a missing interval with the previous recorded starting index and the index of the previous data row of the current non-missing value, and recording it, and resetting the starting index to continue searching for the next missing interval, until the identification and judgment of all power consumption data are completed, and outputting the missing interval set corresponding to the user; According to the missing interval set corresponding to the user, the power consumption data of the previous day and the next day corresponding to each missing interval is obtained, the power consumption ratio between the power consumption data of the next day and the power consumption data of the previous day of the missing interval is calculated, and it is judged whether the power consumption ratio is greater than a preset ratio. If yes, it is judged that the missing interval has data stacking condition, otherwise, it is judged that the missing interval does not have data stacking condition; S21, when the judgment result is that the missing interval has data stacking condition, the corresponding missing interval is cleaned up for data stacking condition; S22, after the data stacking condition is cleaned up, the abnormal value of each user corresponding to each power consumption data is excluded; S23, the power consumption data corresponding to each user after the abnormal value is excluded is standardized.

3. The method of claim 2, wherein the method further comprises: The preset feature clustering strategy is: Step 1, according to the data field characteristic data corresponding to each user, time domain characteristic data and frequency domain characteristic data, the characteristic vector data corresponding to each user is integrated to form the characteristic matrix corresponding to all users ; In the formula, is a feature vector corresponding to the user i, is a dimension value corresponding to the n-th dimension in the corresponding feature vector data of the user i; Step 2, determine the number of clusters corresponding to this clustering K; Step 3, according to the feature matrix corresponding to all users, based on the cluster number K, randomly select K from the m users as the cluster center corresponding to this clustering; Step 4, according to the feature matrix corresponding to the remaining users, based on the preset user power consumption distance degree calculation formula, the user power consumption distance degree between the remaining users and the users corresponding to the K cluster centers is calculated; The preset user power consumption distance degree calculation formula is: wherein, is a user distance measure for the user and the user corresponding to the cluster center, is a dimension value corresponding to the jth dimension corresponding to the user i, is a dimension value corresponding to the jth dimension corresponding to the user corresponding to the cluster center. Step 5, according to the user power consumption distance degree between each remaining user and the user corresponding to each cluster center, compare the user power consumption distance degree between the remaining users and the users corresponding to each cluster center, divide the user with the smallest user power consumption distance degree corresponding to the cluster center and the remaining user into the same class, and form the initial aggregation class corresponding to each cluster center; Step 6, according to each user power consumption distance degree in the initial aggregation class, the arithmetic mean value of each dimension corresponding to each user in the corresponding initial aggregation class is calculated, and the center point of each initial aggregation class is obtained. The user power consumption distance degree between each user in each initial aggregation class and the center point of each initial aggregation class is calculated and compared, and each user is divided into the aggregation class corresponding to the center point with the smallest user power consumption distance degree in turn; Step 7, after completing the user adjustment of all cluster aggregation classes, output the user of this round aggregation class corresponding to each cluster center, and form the user aggregation class corresponding to each cluster center; Step 8, after completing the formation of the user aggregation class corresponding to each cluster center corresponding to this clustering, the next round of clustering is carried out and the cluster number corresponding to the next round of clustering is updated, and step 3 is reexecuted until the members in the cluster no longer change; Step 9, according to the user aggregation class corresponding to each cluster center corresponding to each cluster number, based on the preset loss function calculation formula, the loss function value corresponding to each cluster number is calculated, and the optimal cluster number and the user aggregation class corresponding to each cluster center corresponding to the optimal cluster number are determined and output.

4. The method of claim 3, wherein the method further comprises: The S6 includes: S60, the data set information corresponding to the same user aggregation class is divided into training set and validation set according to the preset proportion; S61, based on the pre-constructed data completion model, determine the parameter search range corresponding to each hyperparameter corresponding to the data completion model; according to the parameter search range corresponding to each hyperparameter, randomly generate a plurality of hyperparameter groups to form the data completion model corresponding to each hyperparameter group; S62, the meteorological data, date information data and feature vector data in the training set are input into the data completion model corresponding to each hyperparameter group as input data, and the predicted data corresponding to each hyperparameter group is predicted as a predicted value, and the corresponding electronic data in the training set is used as a target value; S63, according to the predicted value and the corresponding target value corresponding to each hyperparameter group, based on the preset loss value calculation formula, the loss value of the data completion model corresponding to each hyperparameter group is calculated; The preset loss value calculation formula is: wherein is a loss value, is a target value, is a prediction value, is a number of samples in the dataset information; S64, according to the loss value of the data completion model corresponding to each hyperparameter group, it is judged whether there is a loss value less than a preset first loss threshold, if not, all hyperparameter groups are mutated to form a plurality of new hyperparameter groups and corresponding data completion models, and S62 is re-executed; If yes, the corresponding hyperparameter group is selected, and the hyperparameter group is mutated to form a plurality of new hyperparameter groups and corresponding data completion models, S62 to S63 are re-executed, and it is judged whether the loss value calculated in S63 exists less than the loss value corresponding to the hyperparameter group, if not, the hyperparameter group is the optimal hyperparameter group, and the data completion model corresponding to the hyperparameter group is output; if yes, the corresponding new hyperparameter group is the optimal hyperparameter group, and the data completion model corresponding to the new hyperparameter group is output; S65, the meteorological data, date information data and feature vector data obtained by the user in S3 in the verification set are input into the data completion model corresponding to the output corresponding hyperparameter group as input data, and the corresponding predicted data is output, and the corresponding prediction error is calculated based on the user sub data in the verification set, it is judged whether the prediction error is less than the preset error threshold, if yes, the data completion model corresponding to the output corresponding hyperparameter group is a feasible model, otherwise it is an infeasible model, and S61 is re-executed.

Images