Industrial and commercial user load data cleaning method and device for energy storage safety

Abstract

The invention discloses an industrial and commercial user load data cleaning method and device for energy storage safety, and belongs to the technical field of electric power data cleaning. The method comprises the following steps: acquiring original load data, and preprocessing an original load sequence to obtain a preprocessed load sequence; performing time domain decomposition on the preprocessed load sequence to obtain a trend component, a periodic component and a remainder component; performing frequency domain decomposition on the remainder component to obtain intrinsic mode components of different frequencies; wherein the intrinsic mode component comprises a low-frequency component and a high-frequency component; performing abnormal date confirmation based on the high-frequency component, and correcting a remainder component of the abnormal date based on a preset PSO-BP prediction model; and superposing the trend component, the periodic component and the corrected remainder component to obtain corrected load data. According to the method, the accuracy and reliability of user power data cleaning of the industrial and commercial energy storage system can be remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of power data cleaning technology, and in particular relates to a method and apparatus for cleaning load data of industrial and commercial users for energy storage security. Background Technology

[0002] With the global energy transition and the construction of new power systems, commercial and industrial energy storage systems have entered a stage of large-scale application as a core means to improve grid flexibility, achieve demand-side response, and reduce demand-based electricity costs. High-quality electricity load data is regarded as the "nutritional food" for the intelligent "brain" of energy storage systems. The capacity configuration, technology selection, and all optimal charging and discharging control decisions of energy storage systems are highly dependent on the accurate understanding of user-side load characteristics.

[0003] Existing methods for cleaning and restoring electricity user load data mainly rely on initial screening based on business rules or traditional statistical evaluation models. Some studies use business experience databases to logically filter the original collected sequences by setting thresholds for single anomaly indicators such as missing values, zero values, and instantaneous jump values. Other studies use moving averages, Z-score spatial distribution methods, or simple clustering algorithms (such as K-means and DBSCAN) to analyze the statistical characteristics of historical load distribution and statically remove data points that deviate from the main distribution. These methods have a certain ability to identify obvious bad numbers in scenarios such as single metering faults and terminal offline, providing support for basic electricity data management.

[0004] For the full lifecycle operation and maintenance of commercial and industrial energy storage systems deeply integrated with "source, grid, load, and storage," existing technologies suffer from the following three core defects, seriously threatening the safe and economical operation of energy storage systems: First, inaccurate data restoration leads to risks in energy storage configuration and selection. The core power and capacity configuration of energy storage highly depends on the accurate analysis of users' maximum demand and peak-valley characteristics. If existing cleaning methods cannot accurately restore "false peaks" or "leaked peaks" caused by meter failures or communication interruptions, it will directly lead to over-configuration (wasted investment) or under-configuration (ineffective peak shaving), and may even mislead the selection decisions for technologies such as lithium batteries, flywheels, or supercapacitors. Second, abnormal feature interference leads to "learning distortion" of energy storage control strategies. Modern energy storage control algorithms (such as those based on reinforcement learning or model predictive control) need to be trained on historical data. Existing technologies struggle to separate atypical patterns such as holiday shutdowns and equipment failures from the basic patterns. If the control strategy "absorbs" this contaminated data, it will cause the algorithm to undercharge or over-discharge on normal working days, seriously deviating from the economic expectations of peak-valley price arbitrage and demand saving. Third, residual high-frequency noise poses safety hazards to energy storage systems. For commercial and industrial energy storage systems involved in frequency regulation or second-level response, data cleaning indicators often focus on surface point values, failing to extract deep structural features during the sequence evolution process. Uncleaned power data noise can accumulate into huge errors in cell state of charge estimation, easily leading to battery overcharging and over-discharging, resulting in serious safety accidents such as thermal runaway.

[0005] Therefore, there is an urgent need to develop a data cleaning method that can deeply decompose load composition, accurately identify multidimensional anomalies, and achieve high-fidelity restoration, so as to provide "high-nutrient, high-purity" data support for the safe operation and accurate decision-making of industrial and commercial energy storage systems. Summary of the Invention

[0006] The purpose of this invention is to provide a method and apparatus for cleaning industrial and commercial user load data for energy storage security, in order to solve the problem of inaccurate data restoration that easily occurs when performing large-scale data processing on existing power load data cleaning.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] In a first aspect, the present invention provides a method for cleaning industrial and commercial user load data for energy storage security, comprising the following steps:

[0009] Obtain the raw load data within the target power grid system and preprocess the raw load data to obtain a preprocessed load sequence;

[0010] The preprocessed load sequence is decomposed in the time domain to obtain the trend component, periodic component and residual component.

[0011] The remaining components are decomposed in the frequency domain to obtain intrinsic mode components of different frequencies; wherein the intrinsic mode components include: low-frequency components and high-frequency components.

[0012] Abnormal dates are confirmed based on the high-frequency components, and the residual components of the abnormal dates are corrected based on a preset PSO-BP prediction model; wherein the input of the preset PSO-BP prediction model is the residual components, date type and meteorological factors corresponding to the abnormal date, and the output is the corrected residual components.

[0013] The corrected load data is obtained by superimposing the trend component, the periodic component, and the corrected residual component.

[0014] The method for temporal decomposition of the preprocessed load sequence includes:

[0015] The preprocessed load sequence is iterated through nested inner and outer loops until the number of outer loop iterations reaches a preset value, and the corresponding trend component, periodic component and residual component are output.

[0016] The inner loop includes the following steps:

[0017] The preprocessed load sequence is de-trended to obtain a de-trended sequence; the de-trended sequence is smoothed by LOESS regression to obtain the periodic component of the current iteration round; the preprocessed load sequence is de-periodic to obtain a de-periodic sequence, and the de-periodic sequence is smoothed by LOESS regression to obtain the trend component of the current iteration round.

[0018] The outer loop includes the following steps:

[0019] Based on the periodic and trend components obtained from the inner loop, the remainder component of the current iteration round is calculated. Based on the remainder component, the robust weight of each time point is calculated, and the robust weight is fed back into the inner loop to participate in the LOESS regression smoothing process.

[0020] The method for frequency domain decomposition of the remainder components includes:

[0021] The remaining components are decomposed using the EMD method, with the following formula:

[0022]

[0023] in, Indicates the remainder component. This represents the i-th intrinsic mode component. Let n denote the residual function, and n represent the number of intrinsic mode components decomposed.

[0024] The method for confirming abnormal dates based on the high-frequency components includes:

[0025] The high-frequency components are grouped according to date type, and the high-frequency components of the group are processed in the feature space based on density clustering algorithm. The dates corresponding to data points that fall into the preset low-density region are marked as abnormal dates.

[0026] Before correcting the residual components of the abnormal dates based on the preset PSO-BP prediction model, the steps include: determining the correlation between meteorological factors and load, and the correlation between date type and load.

[0027] If the meteorological factor is related to the load, then the meteorological factor is input into the preset PSO-BP prediction model; otherwise, the meteorological factor is set to zero. If the date type is related to the load, then the date type is input into the preset PSO-BP prediction model; otherwise, the date type is set to zero.

[0028] The method for determining the correlation between meteorological factors and load is as follows: Based on the low-frequency component, calculate the Pearson correlation coefficient between the 7-day moving average of the load and the daily average temperature series corresponding to the low-frequency component. The calculation formula is:

[0029]

[0030] in, Represents the correlation coefficient; Represents a daily average temperature sequence; This represents the 7-day moving average sequence of load; The standard deviations of the daily average temperature series and the 7-day moving average load series are represented by , and cov(·) represents the covariance function. If the correlation coefficient is greater than a preset meteorological threshold, it indicates that the load is correlated with meteorological factors; otherwise, it is not correlated.

[0031] The method for determining the correlation between date type and load is as follows: using the abnormal date as the date to be detected, a correlation analysis between the abnormal date and load is performed based on a probability distribution function, wherein the probability distribution function is:

[0032]

[0033] in, This represents the probability that the daily load level falls within the normal range. This indicates the actual load on the day to be tested; This indicates the expected electricity consumption for the same type of date; The standard deviation of electricity consumption for the corresponding date type is represented by exp(·), which represents an exponential function. If the probability value is less than a preset probability threshold, it indicates that the load is related to the date type; otherwise, it is not related.

[0034] If the date type is related to the load, the corrected load data will be adjusted a second time based on the date type.

[0035] The preset PSO-BP prediction model is composed of a convolutional neural network and a backpropagation neural network. It extracts deep structural features from the input data through two sets of convolutional layers and pooling layers, and performs nonlinear mapping through the BP network to map the extracted deep structural features to the output target. The nonlinear mapping uses the minimum mean square error of the training set as the fitness function, and the initial weights and thresholds of the PSO-BP prediction model are iteratively optimized in the global parameter space using the particle swarm optimization algorithm.

[0036] Secondly, the present invention provides a data cleaning device for industrial and commercial user loads oriented towards energy storage security, used to implement the above-mentioned data cleaning method for industrial and commercial user loads oriented towards energy storage security, the device comprising:

[0037] The data preprocessing module is used to acquire the raw load data in the target power grid system, process the random missing electricity and significant outliers in the raw load data, and obtain the preprocessed load sequence.

[0038] The data decomposition module is used to perform time-domain decomposition on the preprocessed load sequence to obtain trend components, periodic components, and residual components; and to perform frequency-domain decomposition on the residual components to obtain intrinsic mode components of different frequencies; wherein the intrinsic mode components include: low-frequency components and high-frequency components.

[0039] The data correction module is used to construct a preset PSO-BP prediction model, confirm abnormal dates based on the high-frequency components, and correct the remaining components of the abnormal dates based on the preset PSO-BP prediction model.

[0040] The data output module is used to superimpose the trend component, periodic component, and corrected residual component to obtain corrected load data.

[0041] Beneficial effects: The method and apparatus for cleaning industrial and commercial user load data for energy storage safety of the present invention effectively overcomes the blindness and limitations of traditional cleaning methods that directly process raw data. It uses LOESS-based seasonal trend decomposition to extract trend components and periodic components with strong regularity from the time domain dimension as load benchmarks; it deeply mines the instantaneous disturbance characteristics of different frequency bands from the frequency domain dimension, avoiding the one-sidedness of evaluation by a single statistical law, and can accurately distinguish between normal power consumption fluctuations of users and abnormal changes induced by external factors, thus providing accuracy in load data cleaning and restoration.

[0042] This invention overcomes the bottleneck of existing technologies that rely solely on geometric indicators such as distance or density for identification, by integrating correlation analysis of meteorological factors and dynamic date type into the cleaning model. This enables a systematic characterization of the entire chain of complex systems such as industrial and commercial energy storage systems, fundamentally solving the problem of safety and accuracy assessment bias caused by neglecting external environmental interference. It ensures that the cleaned data conforms to users' long-term electricity consumption patterns while accurately reflecting specific production plan changes. This refined processing method provides highly reliable raw data support for demand-side analysis in the electricity spot market and refined safety management of the power grid. Attached Figure Description

[0043] Figure 1 This is a flowchart of the industrial and commercial user load data cleaning method for energy storage security according to the present invention;

[0044] Figure 2 This is a flowchart of the time-domain decomposition process of the industrial and commercial user load data cleaning method for energy storage security according to the present invention. Detailed Implementation

[0045] To illustrate in detail the technical solutions adopted by the present invention to achieve the intended technical objectives, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Furthermore, the technical means or technical features in the embodiments of the present invention can be replaced without creative effort. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0046] Example 1

[0047] refer to Figure 1 and Figure 2 As shown, a method for cleaning industrial and commercial user load data for energy storage security includes the following steps:

[0048] Step 1: Obtain the raw load data within the target power grid system and preprocess the raw load data to obtain the preprocessed load sequence.

[0049] In this embodiment, the raw load data of industrial and commercial users is first acquired. To address the issues of random missing data points and significant outliers caused by power sensor hardware failures, communication protocol transmission errors, or manual input bias, the raw load data is preprocessed. Based on the short-term stationarity characteristics of industrial and commercial loads, the moving average of the first three samples and the last four samples of the outlier data points is used to initially fill in the missing values. This preprocessing eliminates discontinuities in the time domain of the raw load data, constructing a continuous preprocessed load sequence that is both temporally continuous and statistically significant, providing a high-quality input source for subsequent multidimensional component stripping.

[0050] Step 2: Perform time-domain decomposition on the preprocessed electricity consumption sequence to obtain trend components, periodic components, and residual components.

[0051] The STL algorithm (Seasonal and Trend decomposition using Loess), based on LOESS (Locally Estimated Scatterplot Smoothing), is used to decompose preprocessed electrical sequences containing complex noise into mutually orthogonal trend components, periodic components, and residual components. STL is an iterative nonparametric regression process based on robust local weighted regression, including inner and outer loops. Its advantage lies in obtaining periodic terms that are minimally affected by outliers.

[0052] The preprocessed load sequence is iterated through nested inner and outer loops until the number of outer loop iterations reaches a preset value, and the corresponding trend component, periodic component and residual component are output.

[0053] The preprocessed load sequence is de-trended to obtain a de-trended sequence; the de-trended sequence is smoothed by LOESS regression to obtain the periodic component of the current iteration round; the preprocessed load sequence is de-periodic to obtain a de-periodic sequence, and the de-periodic sequence is smoothed by LOESS regression to obtain the trend component of the current iteration round.

[0054] Based on the periodic and trend components obtained from the inner loop, the remainder component of the current iteration round is calculated. Based on the remainder component, the robust weight of each time point is calculated, and the robust weight is fed back into the inner loop to participate in the LOESS regression smoothing process.

[0055] The inner loop uses steps such as detrending, smoothing of periodic subsequences, and low-pass filtering to accurately extract the periodic components that reflect the core production patterns of industrial and commercial users; the outer loop allocates robust weights based on the calculated residual values ​​and effectively reduces the interference of abnormal mutations on component extraction through a non-parametric regression process, thereby accurately separating the trend components and periodic components that reflect the long-term basic electricity consumption level of users.

[0056] The specific steps of the inner loop are as follows:

[0057] (1) Detrending: from the preprocessed load sequence Subtract the trend component generated in the previous iteration. Obtain the detrended sequence ;

[0058] (2) Periodic subsequence smoothing: Extract points at the same period position in the detrended sequence (e.g., every Monday) to form a periodic subsequence, perform LOESS regression smoothing on each periodic subsequence, and extend it one period before and after to obtain the smoothed periodic components. .

[0059] LOESS regression smoothing mechanism selects smoothing parameters Smoothing parameters It is a positive integer representing the width of the local smoothing window selected in the LOESS locally weighted linear regression process; when processing load data, this parameter determines the fineness of the STL decomposition in separating the trend and periodic components.

[0060] Taking periodic subsequence smoothing as an example, the process of LOESS regression smoothing is as follows: selecting from periodic subsequences The observation sample points closest to the target time point constitute a local subset, and weights are assigned to each observation sample point based on its distance from the target time point. To enhance robustness, the assigned weights are calculated based on distance weights and robustness weights. Multinomial regression fitting is performed on the local subset to obtain the regression smooth value of the target time point. The regression smooth value of each point in the periodic subsequence is calculated to obtain the regression model of the corresponding periodic subsequence. The model is then extended by one period forward and backward to obtain the periodic component of the current iteration round.

[0061] The distance weights are derived from a weighting function, which is defined as follows:

[0062]

[0063] in, This represents the distance weight assigned to each observed sample point, used to control the point's influence in local regression. This represents the normalized distance between the target time point and the observed sample points. The distance weight decreases as the distance increases, and the distance weight of points exceeding the critical distance is set to 0 in order to extract local features.

[0064] The optimization objective of LOESS regression smoothing is to minimize local fitting error by solving the regression coefficients using the weighted least squares method.

[0065]

[0066] This represents the time point in a periodic subsequence and its corresponding load value. This represents the weight at time point i, which is a composite value of the distance weight and the robustness weight. This represents the slope of the fitted regression curve equation. This represents the constant term (intercept) of the fitted regression curve equation.

[0067] (3) Low-pass filtering of periodic subsequences: for periodic components The process involves sequentially performing a moving average (with a window length typically set to 3 periods) and filtering to extract the low-frequency components. .

[0068] (4) Detrending of periodic components: Subtracting low-frequency components from the smoothed periodic sequence yields the periodic components of the current cycle. :

[0069]

[0070] (5) Deperiodicization: from the original sequence Subtracting the current periodic component from the middle yields the deperiodic sequence:

[0071]

[0072] (6) Trend series smoothing: Perform LOESS regression smoothing on the de-periodic series to obtain the updated trend components. .

[0073] The main purpose of the outer loop is to introduce robust weights to suppress the impact of outliers in the original data (such as meter reading errors or sudden equipment failures) on the decomposition results. The specific steps are as follows:

[0074] (1) Calculate the remainder term based on the trend component obtained from the inner loop. and periodic components Calculate the current remainder component :

[0075]

[0076] (2) Calculate the robust weights. Based on the magnitude of the residual components, use the smoothing weight function to calculate the robust weights at each time point t. :

[0077]

[0078] The robust weight at time point t is used to identify and reduce the interference of anomalous behavior during the decomposition process. This represents the remainder component calculated in the current iteration; denoted by , represents the median of the absolute values ​​of the remainder components, serving as a benchmark for measuring data fluctuation. B represents the bisquared weighting function, used to assess the degree of anomaly; the closer it is to 1, the more reliable the data at that point is.

[0079] (3) Feed the robust weights back to the inner loop to participate in the LOESS regression smoothing process until a preset fixed number of iterations is reached, and then output the final decomposition result.

[0080] Step 3: Perform frequency domain decomposition on the remaining components to obtain the intrinsic mode components and residual functions at different frequencies.

[0081] For the residual components obtained in step two, which have strong randomness, aperiodicity, and nonlinear characteristics, this embodiment uses the EMD (Empirical Mode Decomposition) method to decompose the residual components in order to extract the instantaneous feature scale of the signal:

[0082] Using the remaining components as the input signal, the extreme points (maximum and minimum values) of the input signal are extracted. The upper and lower envelopes are established by fitting the maxima and minimum values ​​to a cubic spline function, and the local envelope average of the input signal is removed. The formula is as follows:

[0083]

[0084] in, This represents the sequence to be screened obtained by subtracting the local envelope mean from the input signal. This represents the input signal, i.e., the remainder component. , This represents the arithmetic mean of the upper and lower envelopes.

[0085] Determine the sequence to be filtered Does it meet the IMF criteria? If so, then The first IMF (Intrinsic Mode Functions) (high-frequency components) will be used; otherwise, it will be used. Repeat the above steps as a new input signal until the extracted sequence to be screened meets the IMF condition.

[0086] The IMF conditions are: the number of extreme points is equal to or differs by at most 1 from the number of zero-crossing points; at any given time, the mean of the envelope of the local maximum and the envelope of the local minimum of the signal is 0.

[0087] After multiple "screening" iterations, the complex non-stationary signal is decomposed into several intrinsic mode components arranged in descending order of frequency, and a residual function (monotonic function or constant) that characterizes the long-term evolution trend.

[0088]

[0089] in, The i-th intrinsic mode component represents the fluctuation of the signal at different time scales; The residual function represents the overall average trend of the signal, and n represents the number of intrinsic mode components decomposed.

[0090] Inherent modal components include low-frequency components and high-frequency components.

[0091] This process maps the originally chaotic residual features to the time-frequency domain space, where high-frequency components can keenly capture instantaneous abnormal information caused by power grid fluctuations or unplanned start-up and shutdown of equipment.

[0092] Step 4: Determine anomalous dates based on high-frequency components and perform correlation analysis on date attributes to obtain anomalous dates and their date types. Perform meteorological correlation analysis based on low-frequency components to obtain meteorological factors.

[0093] The high-frequency components generated by the frequency domain decomposition in step three are used as the identification targets. First, the high-frequency components are grouped according to date type (such as specific weekdays, weekends, or public holidays). Then, the DBSCAN density clustering algorithm is applied in the feature space. Since normal load fluctuations have high local density under the same date type, outliers falling into low-density regions are labeled as abnormal date tags in the original sequence. By setting a reasonable neighborhood radius and minimum number of included samples, the algorithm can automatically identify and divide density-connected point clusters.

[0094] To address the heterogeneity of industrial and commercial load during holidays, a normal distribution model is used to establish a baseline distribution of electricity consumption during non-holiday periods. The abnormal date is used as the test date, and the probability that the load on the test date falls into the electricity consumption baseline distribution during non-holiday periods is calculated.

[0095]

[0096] in, This represents the probability that the daily load level falls within the normal range. This indicates the actual load on the day to be tested; This represents the expected (average) electricity consumption during non-holiday periods for the same type of date (such as weekends or weekdays). This represents the standard deviation of electricity consumption for the corresponding date type, and exp(·) represents the exponential function.

[0097] If the probability value is less than the significance level of the preset probability threshold (0.05), it indicates that the load is related to the date type. It is determined that the date to be detected is strongly coupled with the holiday attribute, and a special probability distribution adjustment mechanism needs to be started at the end of the cleaning process to correct the prediction bias.

[0098] The load of industrial and commercial users is significantly affected by temperature. Based on low-frequency components, meteorological correlation analysis was performed simultaneously to calculate the Pearson correlation coefficient between the 7-day moving average series of load and the daily average temperature series.

[0099]

[0100] in, The correlation coefficient is used to quantify the strength of the association between meteorological factors and load. Represents a daily average temperature sequence; : Represents the 7-day moving average sequence of load; represents the standard deviation of the daily average temperature sequence and the 7-day moving average load sequence, and cov(·) represents the covariance function.

[0101] If the correlation coefficient is greater than the preset meteorological threshold (0.6), the power load is determined to have significant temperature sensitivity characteristics, and meteorological factors need to be introduced as endogenous variables into the subsequent correction model.

[0102] Low-frequency components and residual functions characterize the trend components in the load that evolve slowly over time. In industrial and commercial power consumption scenarios, this evolution trend is mainly driven by ambient temperature. The 7-day moving average sequence of the load is a numerical representation of the low-frequency components in the time domain. Its fluctuation trend is highly physically consistent with the fluctuation trend of the low-frequency components, thus ensuring that the correlation coefficient can truly reflect the driving weight of meteorological factors on the long-term trend of the load.

[0103] Step 5: Correct the remaining components of the abnormal dates based on the preset PSO-BP neural network correction model.

[0104] For the abnormal date points identified in step four, this embodiment uses the PSO-BP prediction model to perform nonlinear numerical correction. The PSO-BP prediction model uses the 15 samples before and the 15 samples after the center of the time window of the abnormal date point as the sample sequence, and the corresponding residual components, date type, and meteorological factors as multidimensional feature inputs.

[0105] The PSO-BP prediction model is composed of a convolutional neural network and a backpropagation neural network. First, it uses two sets of convolutional layers and pooling layers to automatically integrate features and extract deep structured features from the input data. Then, it performs nonlinear mapping through the BP network to map the extracted deep structured features to the output target. To avoid the technical bottleneck of traditional BP networks being prone to getting trapped in local minima and having slow convergence speed, the particle swarm optimization algorithm is used to iteratively optimize the initial weights and thresholds of the neural network in the global parameter space. The model evolves by minimizing the mean square error of the training set as the fitness function, thus optimizing the initial weights and thresholds of the prediction model in the global parameter space.

[0106] In this embodiment, the model of the convolutional neural network is expressed as follows:

[0107]

[0108] in, This represents the output value of the k-th convolutional layer at position i; This represents the convolution kernel (weights) of the convolutional layer. This represents the bias term of the convolutional layer. This represents a sample sequence defined based on abnormal dates. The multidimensional feature input of the PSO-BP prediction model (including residual components, date type, and meteorological factors) is captured by a sliding convolution window to capture local fluctuations in the input features.

[0109] Based on the analysis results of step four, if the meteorological correlation coefficient is determined... And the probability of being affected by holidays This indicates that the residual fluctuations of the industrial and commercial user are decoupled from meteorological conditions and specific date attributes. In this case, to reduce the model input dimensionality, avoid overfitting the neural network to irrelevant noise features, and improve global optimization efficiency, meteorological factors and date types are set to zero. The input vector of the PSO-BP prediction model will consist only of the remainders of the 15 samples before and after the center of the outlier window, without necessarily including meteorological parameters and date type encoding parameters. Otherwise, if meteorological factors are related to the load, they will be incorporated into the input vector; if date types are related to the load, they will be incorporated into the input vector. This ensures that the modified model can accurately capture the load mutation characteristics driven by the environment, thereby guaranteeing the decision-making accuracy of the energy storage system under complex operating conditions.

[0110] The corrected residual component of the anomalous date is predicted by the trained PSO-BP prediction model.

[0111] Step Six: Data Reconstruction.

[0112] The corrected residual component obtained in step five is superimposed with the trend component and periodic component extracted in step two to obtain the corrected load data, thereby completing the physical and logical correction of the abnormal data.

[0113] Before outputting the cleaned data, to address the statistical distortion caused by the scarcity of holiday data samples, a probability distribution function is introduced to perform a secondary adjustment on the corrected load data. This invention introduces a normal distribution (Gaussian distribution) probability model, which establishes a baseline electricity consumption distribution based on the mean μ and standard deviation σ of electricity consumption on similar dates (such as weekdays or weekends) during historical non-holiday periods.

[0114] The secondary adjustment is implemented as follows: In the final stage before data output, for data determined to be "holiday-related", the probability distribution function is used as a constraint to perform probability weighting or distribution mapping on the load values ​​initially corrected by PSO-BP, thereby offsetting the model prediction bias caused by the scarcity of holiday samples and ensuring that the cleaning results truly reflect the specific electricity consumption characteristics of the holiday.

[0115] Example 2

[0116] A data cleaning device for industrial and commercial user loads aimed at energy storage security, comprising:

[0117] The data preprocessing module is used to acquire the raw load data in the target power grid system, process the random missing electricity and significant outliers in the raw load data, and obtain the preprocessed load sequence.

[0118] The data decomposition module is used to perform time-domain decomposition on the preprocessed load sequence to obtain trend components, periodic components, and residual components; and to perform frequency-domain decomposition on the residual components to obtain intrinsic mode components of different frequencies; wherein the intrinsic mode components include: low-frequency components and high-frequency components.

[0119] The data correction module is used to construct a preset PSO-BP prediction model, confirm abnormal dates based on the high-frequency components, and correct the remaining components of the abnormal dates based on the preset PSO-BP prediction model.

[0120] The data output module is used to superimpose the trend component, periodic component, and corrected residual component to obtain corrected load data.

[0121] In summary, the industrial and commercial user load data cleaning method and apparatus for energy storage safety of the present invention effectively overcomes the blindness and limitations of traditional cleaning methods that directly process raw data. It utilizes LOESS-based seasonal trend decomposition to extract trend and periodic components with strong regularity from the time domain as load benchmarks; and deeply mines the instantaneous disturbance characteristics of different frequency bands from the frequency domain, avoiding the one-sidedness of evaluation by a single statistical law. It can accurately distinguish between normal power consumption fluctuations of users and abnormal changes induced by external factors, providing accuracy in load data cleaning and restoration.

[0122] This invention overcomes the bottleneck of existing technologies that rely solely on geometric indicators such as distance or density for identification, by integrating correlation analysis of meteorological factors and dynamic date type into the cleaning model. This enables a systematic characterization of the entire chain of complex systems such as lithium battery energy storage or industrial electricity consumption, fundamentally solving the problem of safety and accuracy assessment bias caused by neglecting external environmental interference. It ensures that the cleaned data conforms to users' long-term electricity consumption patterns while accurately reflecting specific production plan changes. This refined processing method provides highly reliable raw data support for demand-side analysis in the electricity spot market and refined safety management of the power grid.

[0123] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for cleaning industrial and commercial user load data for energy storage security, characterized in that, Includes the following steps: Obtain the raw load data within the target power grid system and preprocess the raw load data to obtain a preprocessed load sequence; The preprocessed load sequence is decomposed in the time domain to obtain the trend component, periodic component and residual component. Frequency domain decomposition is performed on the remaining components to obtain intrinsic mode components at different frequencies; The intrinsic modal components include: low-frequency components and high-frequency components; Abnormal dates are confirmed based on the high-frequency components, and the residual components of the abnormal dates are corrected based on a preset PSO-BP prediction model; wherein the input of the preset PSO-BP prediction model is the residual components, date type and meteorological factors corresponding to the abnormal date, and the output is the corrected residual components. The corrected load data is obtained by superimposing the trend component, the periodic component, and the corrected residual component.

2. The method for cleaning industrial and commercial user load data for energy storage security according to claim 1, characterized in that, The method for temporal decomposition of the preprocessed load sequence includes: The preprocessed load sequence is iteratively decomposed into nested inner and outer loops until the number of outer loops reaches a preset value, and the corresponding trend component, periodic component and residual component are output. The inner loop includes the following steps: The preprocessed load sequence is de-trended to obtain a de-trended sequence; the de-trended sequence is smoothed by LOESS regression to obtain the periodic component of the current iteration round; the preprocessed load sequence is de-periodic to obtain a de-periodic sequence, and the de-periodic sequence is smoothed by LOESS regression to obtain the trend component of the current iteration round. The outer loop includes the following steps: Based on the periodic and trend components obtained from the inner loop, the remainder component of the current iteration round is calculated. Based on the remainder component, the robust weight of each time point is calculated, and the robust weight is fed back into the inner loop to participate in the LOESS regression smoothing process.

3. The method for cleaning industrial and commercial user load data for energy storage security according to claim 1, characterized in that, The method for frequency domain decomposition of the remainder components includes: The remaining components are decomposed using the EMD method, with the following formula: in, Indicates the remainder component. This represents the i-th intrinsic mode component. Let n denote the residual function, and n represent the number of intrinsic mode components decomposed.

4. The method for cleaning industrial and commercial user load data for energy storage security according to claim 1, characterized in that, The method for confirming abnormal dates based on the high-frequency components includes: The high-frequency components are grouped according to date type, and the high-frequency components of the group are processed in the feature space based on density clustering algorithm. The dates corresponding to data points that fall into the preset low-density region are marked as abnormal dates.

5. The method for cleaning industrial and commercial user load data for energy storage security according to claim 1, characterized in that, Before correcting the residual components of the abnormal dates based on the preset PSO-BP prediction model, the steps include: determining the correlation between meteorological factors and load, and the correlation between date type and load. If the meteorological factor is related to the load, then the meteorological factor is input into the preset PSO-BP prediction model; otherwise, the meteorological factor is set to zero. If the date type is related to the load, then the date type is input into the preset PSO-BP prediction model; otherwise, the date type is set to zero.

6. The method for cleaning industrial and commercial user load data for energy storage security according to claim 5, characterized in that, The method for determining the correlation between meteorological factors and load is as follows: Based on the low-frequency component, calculate the Pearson correlation coefficient between the 7-day moving average of the load and the daily average temperature series corresponding to the low-frequency component. The calculation formula is: in, Represents the correlation coefficient; Represents a daily average temperature sequence; This represents the 7-day moving average sequence of load; The standard deviations of the daily average temperature series and the 7-day moving average load series are represented by , and cov(·) represents the covariance function. If the correlation coefficient is greater than a preset meteorological threshold, it indicates that the load is correlated with meteorological factors; otherwise, it is not correlated. The method for determining the correlation between date type and load is as follows: using the abnormal date as the date to be detected, a correlation analysis between the abnormal date and load is performed based on a probability distribution function, wherein the probability distribution function is: in, This represents the probability that the daily load level falls within the normal range. This indicates the actual load on the day to be tested; This indicates the expected electricity consumption for the same type of date; The standard deviation of electricity consumption for the corresponding date type is represented by exp(·), which represents an exponential function. If the probability value is less than a preset probability threshold, it indicates that the load is related to the date type; otherwise, it is not related.

7. The method for cleaning industrial and commercial user load data for energy storage security according to claim 6, characterized in that, If the date type is related to the load, the corrected load data will be adjusted a second time based on the date type.

8. The method for cleaning industrial and commercial user load data for energy storage security according to claim 1, characterized in that, The preset PSO-BP prediction model is composed of a convolutional neural network and a backpropagation neural network. It extracts deep structural features from the input data through two sets of convolutional layers and pooling layers, and performs nonlinear mapping through the BP network to map the extracted deep structural features to the output target. The nonlinear mapping uses the minimum mean square error of the training set as the fitness function, and the initial weights and thresholds of the PSO-BP prediction model are iteratively optimized in the global parameter space using the particle swarm optimization algorithm.

9. A data cleaning device for industrial and commercial user loads oriented towards energy storage security, characterized in that, The apparatus for implementing the industrial and commercial user load data cleaning method for energy storage security as described in any one of claims 1 to 8, the apparatus comprising: The data preprocessing module is used to acquire the raw load data in the target power grid system, process the random missing electricity and significant outliers in the raw load data, and obtain the preprocessed load sequence. The data decomposition module is used to perform time-domain decomposition on the preprocessed load sequence to obtain trend components, periodic components, and residual components; and to perform frequency-domain decomposition on the residual components to obtain intrinsic mode components of different frequencies; wherein the intrinsic mode components include: low-frequency components and high-frequency components. The data correction module is used to construct a preset PSO-BP prediction model, confirm abnormal dates based on the high-frequency components, and correct the remaining components of the abnormal dates based on the preset PSO-BP prediction model. The data output module is used to superimpose the trend component, periodic component, and corrected residual component to obtain corrected load data.

Images