A time-sharing day frequency band feature engineering-based abnormal data identification method and system

Abstract

The application belongs to the field of power grid anomaly identification, and relates to an abnormal data identification method and system based on time-of-day and frequency-band feature engineering, comprising: processing initial multi-source data to obtain multi-dimensional time series data; decomposing and reconstructing the multi-dimensional time series data to obtain frequency-band multi-dimensional time series data of multiple frequency bands; for each frequency-band multi-dimensional time series data, establishing a frequency-band data regression model and screening a core feature set; based on the core feature set of an evaluation period, outputting a monitoring data prediction value of the evaluation period through the frequency-band data regression model, and determining abnormal monitoring data based on the monitoring data prediction value and target evaluation data; thereby achieving accurate identification of power grid monitoring data anomalies, and providing technical support for fine technical management and control of power grid operation, equipment fault troubleshooting and dispatching strategy optimization.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power grid anomaly identification, and specifically discloses an abnormal data identification method and system based on time-of-day and frequency band feature engineering. BACKGROUND

[0002] With the upgrading of China's power grid to a multi-element collaborative form of source, network, load and storage, and the continuous increase of new energy proportion, power grid enterprises at all levels have significantly enhanced the demand for fine monitoring and technical control of power grid operation status. Power grid core operation technical indicators (such as various types of power generation, key equipment output, load response data, etc.) as irregular time series are the core technical carriers reflecting the power supply and demand balance status, equipment operation health degree and dispatching execution effectiveness. The fluctuations of the above technical indicators are influenced by multiple technical and environmental factors: both technical factors including regional load demand dynamic changes, natural weather conditions, power grid topology structure characteristics, core equipment operation status, etc.; and operation and control factors involving power grid dispatching strategy execution deviation, new energy output prediction accuracy deviation, etc. Among them, the index fluctuation caused by objective natural condition changes, normal equipment operation and maintenance, and reasonable load fluctuation belongs to the normal technical range allowed by the power grid operation; while the index fluctuation caused by unreasonable factors (such as abnormal low power generation caused by equipment fault concealment, output data distortion caused by sensor failure, and load response anomaly caused by dispatching strategy technical execution deviation) will directly interfere with the real-time power balance calculation of the power grid, increase the safety risks of frequency deviation and voltage out-of-limit, and may also lead to new energy consumption obstruction and key equipment overload, and even threaten regional power supply continuity and destroy the overall operation stability of the power grid in severe cases.

[0003] Current control means for power grid core technical indicator anomalies mainly rely on single indicator fixed threshold constraints. For example, set the absolute value threshold of power generation deviation, the fluctuation amplitude threshold of equipment output, and the delay threshold of load response, etc. However, the changes of power grid core technical indicators have significant dynamics, complexity and multi-correlation. For example, new energy power generation is influenced by multiple technical factors such as short-time weather mutation, prediction accuracy deviation and equipment start-stop state, and equipment output data varies with real-time load demand and line transmission capacity of the power grid, so it is difficult for fixed threshold to adapt to the dynamic adjustment of indicators with technical conditions. Inappropriate threshold setting may not only lead to missed or false judgments of abnormal data, but also ignore the coordinated anomaly among multiple indicators, reducing the accuracy and response efficiency of power grid technical control.

[0004] The existing research on the abnormality of the core technical indicators of the power grid still has obvious technical limitations: first, the research scope is focused on the short-term dramatic fluctuations of a single technical indicator, and the analysis is focused on the short-term numerical deviation amplitude of the indicator, and the technical characteristics of the medium and long-term abnormality are not fully explored. Long-term abnormalities often imply deep problems such as equipment aging and scheduling strategy technical bias, which are prone to cause chain risks due to identification lag; second, the feature analysis of the abnormality of the indicator lacks technical subdivision of the time scale. The core technical indicators of the power grid all show significant periodical fluctuation characteristics: short-term fluctuations, medium-term periodic deviations, and long-term structural deviations. Among them, short-term fluctuations can be quickly identified through real-time monitoring, while medium and long-term abnormalities often have identification lag or missed judgment problems due to slow changes and hidden technical causes; third, the technical correlation analysis of the influencing factors and the indicators lacks pertinence. Different technologies and environmental factors have different effects on different indicators at different time scales.

[0005] Therefore, the present application provides an abnormal data identification method and system based on time-of-day and frequency band feature engineering, which aims to quantify the abnormal fluctuation amplitude outside the normal fluctuation range caused by objective technical conditions for power grid monitoring data (such as various types of power generation, key equipment output, load response data, and electricity prices, etc.) at different time scales; and through the time-of-day and frequency band technical design, the feature correlation between the power grid monitoring data and the internal and external technical influencing factors is constructed to accurately identify whether the power grid monitoring data is abnormal, providing technical support for the fine technical control of power grid operation, equipment fault diagnosis, and scheduling strategy optimization. SUMMARY

[0006] The present application aims to provide an abnormal data identification method and system based on time-of-day and frequency band feature engineering, which solves the problem of accurately identifying whether the power grid monitoring data is abnormal. The specific scheme is as follows:

[0007] An abnormal data identification method based on time-of-day and frequency band feature engineering, comprising: processing initial multi-source data to obtain multi-dimensional time series data; decomposing and reconstructing the multi-dimensional time series data to obtain frequency band multi-dimensional time series data of multiple frequency bands; for each frequency band multi-dimensional time series data, establishing a frequency band data regression model and a core feature set; based on the core feature set of the to-be-evaluated time period, outputting the monitoring data prediction value of the to-be-evaluated time period through the frequency band data regression model, and determining the abnormal monitoring data based on the monitoring data prediction value and the target evaluation data.

[0008] Further, the processing of the initial multi-source data to obtain multi-dimensional time series data comprises: performing integrity check on the obtained initial multi-source data to obtain an integrity check result; the integrity check is implemented by checking the meta information of each type of data; performing data outlier cleaning on the initial multi-source data with the integrity check result being complete to eliminate abnormal data and obtain non-abnormal multi-source data; the data outlier cleaning comprises: performing outlier detection on multi-dimensional data by using Mahalanobis distance and performing outlier detection on single-dimensional data by using Grubbs criterion cycle; performing data missing filling on the non-abnormal multi-source data to obtain complete multi-source data; the data missing filling comprises: performing data filling on spatial data by using Kriging interpolation method, performing data filling on time data by using linear interpolation method, or performing data filling on geographic data by using inverse distance weighting method; dividing load days based on load data in the complete multi-source data to obtain multi-type load days and load day monitoring indicators; dividing other data in the complete multi-source data according to daily granularity to obtain multi-source daily granularity data, and aligning the multi-source daily granularity data with the multi-type load days and the load day monitoring indicators of the corresponding load days to obtain multi-dimensional time series data of time-divided days.

[0009] Further, the processing of the initial multi-source data to obtain multi-dimensional time series data comprises: performing integrity check on the obtained initial multi-source data to obtain an integrity check result; the integrity check is implemented by checking the meta information of each type of data; performing data outlier cleaning on the initial multi-source data with the integrity check result being complete to eliminate abnormal data and obtain non-abnormal multi-source data; the data outlier cleaning comprises: performing outlier detection on multi-dimensional data by using Mahalanobis distance and performing outlier detection on single-dimensional data by using Grubbs criterion cycle; performing data missing filling on the non-abnormal multi-source data to obtain complete multi-source data; the data missing filling comprises: performing data filling on spatial data by using Kriging interpolation method, performing data filling on time data by using linear interpolation method, or performing data filling on geographic data by using inverse distance weighting method; dividing load days based on load data in the complete multi-source data to obtain multi-type load days and load day monitoring indicators; dividing other data in the complete multi-source data according to daily granularity to obtain multi-source daily granularity data, and aligning the multi-source daily granularity data with the multi-type load days and the load day monitoring indicators of the corresponding load days to obtain multi-dimensional time series data of time-divided days.

[0010] Further, the processing of the initial multi-source data to obtain multi-dimensional time series data comprises: performing integrity check on the obtained initial multi-source data to obtain an integrity check result; the integrity check is implemented by checking the meta information of each type of data; performing data outlier cleaning on the initial multi-source data with the integrity check result being complete to eliminate abnormal data and obtain non-abnormal multi-source data; the data outlier cleaning comprises: performing outlier detection on multi-dimensional data by using Mahalanobis distance and performing outlier detection on single-dimensional data by using Grubbs criterion cycle; performing data missing filling on the non-abnormal multi-source data to obtain complete multi-source data; the data missing filling comprises: performing data filling on spatial data by using Kriging interpolation method, performing data filling on time data by using linear interpolation method, or performing data filling on geographic data by using inverse distance weighting method; dividing load days based on load data in the complete multi-source data to obtain multi-type load days and load day monitoring indicators; dividing other data in the complete multi-source data according to daily granularity to obtain multi-source daily granularity data, and aligning the multi-source daily granularity data with the multi-type load days and the load day monitoring indicators of the corresponding load days to obtain multi-dimensional time series data of time-divided days.

[0011] Further, the dividing the other data in the complete multi-source data by day granularity to obtain multi-source day granularity data comprises: reducing sampling of fine-granularity data by a maximum triangle three-bar dimension reduction algorithm to obtain day granularity data; the fine-granularity data refers to data with a sampling frequency less than one day; and interpolating coarse-granularity data by an inverse distance weighted difference algorithm to obtain day granularity data; the coarse-granularity data refers to data with a sampling frequency greater than one day.

[0012] Further, the establishing a frequency band data regression model and screening a core feature set comprises: extracting features from the frequency band multi-dimensional time series data to obtain an internal feature sequence and a plurality of external feature sequences; the internal feature sequence is used to represent the associated change relationship of the monitoring data in the time dimension; and the external feature sequence is used to represent the change of other data; calculating correlation coefficients of the monitoring data sequence with the internal feature sequence and the plurality of external feature sequences respectively, sorting the plurality of feature sequences in descending order of absolute values of the correlation coefficients to obtain a feature importance sequence; constructing a frequency band data regression model, and screening the plurality of feature sequences based on performance of the frequency band data regression model under a plurality of importance threshold values to obtain an initial core feature set; and performing a redundancy check on the initial core feature set based on correlation coefficients between the plurality of feature sequences to eliminate redundant feature sequences to obtain a core feature set.

[0013] Further, the screening the plurality of feature sequences to obtain an initial core feature set comprises: determining a current core feature set based on a current importance threshold value; dividing the current core feature set into a plurality of training sets and a validation set; training an initial frequency band data regression model through the plurality of training sets to obtain a trained frequency band data regression model; verifying the trained frequency band data regression model through the validation set to obtain model performance; determining a new importance threshold value, and repeating the determination of the performance of the trained frequency band data regression model by taking the new importance threshold value as the current importance threshold value; taking a model corresponding to the best performance as a final frequency band data regression model, taking a corresponding current core feature set as an initial core feature set, and taking the current importance threshold value as a final importance threshold value; and the best performance refers to the minimum mean square error of the model.

[0014] Further, when the abnormal monitoring data is abnormal electricity price data, the method further comprises processing the abnormal electricity price data to determine the behavior causing the abnormality.

[0015] Further, the processing of the abnormal electricity price data to determine the behavior causing the abnormality comprises: determining a category to which the abnormal electricity price data belongs; the category comprises a peak day high frequency abnormality, a peak day low frequency abnormality, a peak day trend abnormality, a valley day high frequency abnormality, a valley day low frequency abnormality, a valley day trend abnormality, a flat day high frequency abnormality, a flat day low frequency abnormality, and a flat day trend abnormality; determining one or more category indicators based on the category to which the abnormal electricity price data belongs; and matching the category indicators to corresponding indicator ranges to determine the behavior causing the abnormality.

[0016] The application further provides a time-of-day and frequency band feature engineering abnormal data identification system using the time-of-day and frequency band feature engineering abnormal data identification method.

[0017] The application has the following advantages and beneficial effects:

[0018] The application avoids regarding fluctuations in monitoring data caused by normal changes in the power grid system as abnormal data by constructing a mapping relationship between monitoring data and internal and external influencing factors through time-of-day and frequency band division.

[0019] The application constructs a feature engineering of peak day, valley day, and flat day influence monitoring data from short-term high frequency items, periodic low frequency items, and long-term trend items based on data decomposition and integration technology.

[0020] The application constructs an evaluation index system of abnormal behavior by analyzing possible data abnormal behaviors of peak day, valley day, and flat day in short-term, periodic, and long-term, and provides a quantifiable evaluation standard for the traceability and attribution of abnormal data. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1An exemplary flow chart of an abnormal data identification method of time division day division frequency band feature engineering provided by the present application is shown in the figure.

[0022] Figure 2 An exemplary module diagram of an abnormal data identification system of time division day division frequency band feature engineering provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0023] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0024] An abnormal data identification method of time division day division frequency band feature engineering provided by the present application is shown in the figure, and the technical solution thereof includes the following steps: Figure 1

[0025] Step 1: The data collection and preprocessing module is executed to process the initial multi-source data to obtain multi-dimensional time series data.

[0026] The initial multi-source data refers to the original data of the power grid system collected, which can include node monitoring historical data, regional meteorological historical data, node load historical data, unit data, fuel data, carbon data, renewable energy output prediction, power grid node topology data and regional hydrological data, etc. The monitoring data can include power generation data and electricity price data. The regional meteorological historical data includes temperature, wind speed and irradiation, etc. The power grid node topology structure includes line capacity and node location, etc.

[0027] The initial multi-source data is preprocessed to obtain multi-dimensional time series data. The preprocessing process includes data integrity verification, data outlier cleaning, data missing filling and load day division, etc.

[0028] The obtained initial multi-source data is subjected to integrity verification to obtain an integrity verification result; the integrity verification is realized by checking the meta information of each type of data. For example, the data integrity verification checks whether the meta information such as time stamp, data source identifier, data quality flag, etc. of each type of data is complete, whether the data source is correct at 96-point time of monitoring data (such as power generation and on-grid power data and declaration data and clearing data), and whether the logical relationship between data is correctly matched.

[0029] ​perform data outlier cleaning on the initial multi-source data with the integrity check result being complete, and remove abnormal data to obtain non-abnormal multi-source data; the data outlier cleaning includes performing outlier detection on multi-dimensional data by using Mahalanobis Distance and performing outlier detection on single-dimensional data by using Grubbs Criterion in a loop. The outlier cleaning is used to identify discrete outliers possibly existing in the original data, so as to propose abnormal data and abnormal behaviors in historical data. The multi-dimensional data can include meteorological data, monitoring data of each node and load data. The single-dimensional data can include fuel data and carbon data. After removing the outliers, the data is processed according to the data missing value.

[0030] The Mahalanobis Distance screening process is as follows: assuming that multi-dimensional sequences such as meteorological data, node monitoring data and node load data can be regarded as a data set , the dimensions of different data sets can be different, but the number of days is consistent and is , the Mahalanobis Distance of each point in each data set is calculated respectively:

[0031] ;

[0032] wherein is an m-dimensional data set in s days, is a total mean vector of the data set, is a covariance matrix of the data set and the inverse matrix of the covariance matrix.

[0033] Based on the input of each data set (S-day m-dimensional meteorological data vector), the output of the Mahalanobis Distance set of each data set can be obtained, which is represented as wherein is the Mahalanobis Distance of the data of the s day in the data set, if the square of the Mahalanobis Distance of the data of the day satisfies: wherein is the threshold value of the chi-square distribution when the significance level is , it is indicated that the value of the day is an outlier.

[0034] The Grubbs Criterion screening process is as follows: assuming that single-dimensional sequences such as fuel data and carbon data are , the statistic of Grubbs test is calculated:

[0035] ;

[0036] wherein represents the arithmetic mean of the single-dimensional sequence, after the statistic of Grubbs test is calculated, the statistic is compared with the Grubbs critical value For comparison, the critical value can be obtained by looking up the table according to the sample size S and the significance level, if The point farthest from the mean is considered to be an outlier. After repeated verification, all outlier outputs can be obtained (only one outlier is output each time).

[0037] The missing data of the non-outlier multi-source data is filled to obtain complete multi-source data. The missing data filling includes filling the spatial data by using the Kriging interpolation method, filling the time data by using the linear interpolation method, or filling the geographic data by using the inverse distance weighting method. The spatial data can include meteorological data. The time data can include monitoring data and load data.

[0038] The Kriging interpolation method steps are: inputting each climate sequence and its spatial position coordinates after anomaly screening into the Kriging interpolation method, and outputting the predicted value of the missing and anomaly deleted points and the estimated variance representing the prediction error estimate. The linear interpolation method steps are: identifying the missing points of the node monitoring data, node load, fuel data sequence after anomaly screening and the values of the known points near the missing points, and filling the missing values according to the relative days between the known points and the missing points according to the linear interpolation method.

[0039] Load days are divided based on the load data in the complete multi-source data to obtain multi-type load days and load day monitoring indicators. The load day division refers to dividing the monitoring data into electricity peak days, electricity valley days and electricity flat days according to the load demand data. The time of the present invention refers to the 96-point time of the electricity market, and the day refers to the peak / valley / flat day.

[0040] The steps of peak-valley-flat day division are: first, identifying the maximum value and the average value of the 96-time load per day, and sorting all maximum loads and average loads. The load peak-valley-flat time is determined according to the experience setting of the load ratio threshold, for example, set to 10%, that is, the load day corresponding to the first 10% of the maximum load or the average load is regarded as the load peak day, and the load day corresponding to the last 10% of the maximum load or the average load is regarded as the load valley day. Finally, the union of the peak days determined according to the maximum load and the peak days determined according to the average load is taken as the final load peak day, and the load valley day is determined in the same way, and the remaining days are flat days.

[0041] The monitoring data of each day is in the form of summation or weighted average based on different types. Taking the power generation as an example, the time division monitoring data is:

[0042]

[0043] Wherein, represents the power generation sum of the peak / valley / flat day. represents the s-day The power generation at time t; t represents a time variable; T represents the total number of times, for example, 96; s represents a date variable, , time represents the total number of days monitored.

[0044] Taking the electricity price as an example, the time-weighted average monitoring data of each day is:

[0045] ;

[0046] Among them, represents the weighted price of peak / valley / flat day, t represents 96 time points; represents the load demand at time t on day s; the load demand of each day is the sum of the load at each time of the day; represents the electricity price at time t on day s.

[0047] Other data in the complete multi-source data are divided by day granularity to obtain multi-source day granularity data, and the multi-source day granularity data are aligned with the multi-class load day and the load day monitoring index of the corresponding load day to obtain time-divided day multi-dimensional time series data, including: fine-grained data is reduced by a maximum triangle three-bucket dimension reduction algorithm to obtain day granularity data; fine-grained data refers to data with a sampling frequency less than one day; coarse-grained data is interpolated by an inverse distance weighted difference algorithm to obtain day granularity data; coarse-grained data refers to data with a sampling frequency greater than one day.

[0048] For example, fine-grained meteorological data is down-sampled to the same granularity as the weighted average monitoring data and load data. The present application adopts the LTTB (Largest Triangle Three Buckets) dimension reduction algorithm to adjust the fine-grained meteorological data (for example, the sampling frequency is 1 hour) to daily sampling, retaining the details of the meteorological time series data while ensuring the consistency of the data in the time scale, meeting the subsequent data analysis and modeling needs.

[0049] The steps of the LTTB dimension reduction algorithm include:

[0050] The various types of fine-grained sampling data of each meteorological station are processed by the LTTB algorithm, first dividing each sequence into "buckets" of approximately equal size in order, with each bucket containing a certain number of consecutive data points. The number of buckets and the number of data points in the bucket are determined by a set parameter. In order to down-sample the meteorological data to each day, this parameter is set to allow each bucket to contain all fine-grained data for each day. The first and last buckets only contain the first and last data points of the original data.

[0051] Next, the LTTB algorithm traverses all the storage buckets and selects a point from each bucket. The first storage bucket contains only one point, so this point will be selected. Subsequently, the algorithm will process three buckets at a time, selecting points from left to right in time order:

[0052] The next bucket of the selected representative point uses the already determined representative point, and the next bucket of the selected representative point uses the average value of all points as a temporary representative point, at which time the area of the triangle formed by each point in the middle bucket (the current bucket) and the front and rear representative points is calculated, the point forming the largest triangular area is selected as the final representative point of the current bucket.

[0053] The LTTB extracts the most representative data in each bucket as the output sequence by the maximum triangle method, that is, the representative value of the daily meteorological data is finally determined, while the characteristics of the original data are maximally retained.

[0054] Considering that the meteorological conditions are different at different geographical positions, at the geographical space level, the present application adopts the inverse distance weighted method (IDW) to interpolate the discrete meteorological station data near the node, inputs the data of the known meteorological stations near the node, combines the power grid topological structure data, obtains the unknown meteorological data of the geographical position of the node, and constructs the corresponding relationship between the geographical position of the power grid node and the meteorological grid network.

[0055] The basic steps of the inverse distance weighted interpolation method include:

[0056] After the meteorological data is down-sampled, the daily meteorological representative value of each meteorological station is obtained as the known point position and observation value in the inverse distance weighted interpolation. For each node, first, identify the meteorological stations near the node, and the identification logic is the five meteorological stations closest to the node. Input the meteorological data and position information of the adjacent nodes and the position information of the node into the inverse distance weighted interpolation method:

[0057] ;

[0058] Wherein, is the estimated meteorological value of the geographical position of the node n, is the known meteorological value of the adjacent meteorological station, and R is the number of adjacent stations. is the Euclidean distance from the rth meteorological station to the node.

[0059] In this way, the meteorological value of the node is estimated by weighted calculation according to the principle that the closer the meteorological station to the node, the greater the influence on the node.

[0060] The feature sequence of the fuel data and the carbon data is a daily granularity data, and the granularity after the weighted average of the monitoring data is the same, and it is assumed that the influence degree on each node is the same.

[0061] Step two, the data decomposition and reconstruction module is executed, the multi-dimensional time series data is decomposed and reconstructed, and a plurality of frequency band multi-dimensional time series data of frequency bands is obtained, including:

[0062] Each type of load day is subjected to multi-component empirical mode decomposition, to obtain the connotation modal component and trend item of multiple types of load days; the connotation modal component includes multiple frequencies, including:

[0063] Based on the monitoring data and the associated data, a multi-dimensional time sequence is constructed to obtain the multi-dimensional time sequence characteristics. For peak days, valley days and flat days, multi-dimensional time sequences of monitoring data and their supply and demand influencing factors (load, weather, fuel, carbon emissions) are constructed. Taking the power generation data as an example, the multi-dimensional time sequence can be , wherein, is the power generation sequence, is the influencing factor sequence, and time represents the peak / valley / flat day. Taking the electricity price data as an example, the multi-dimensional time sequence can be , wherein is the electricity price sequence, is the influencing factor sequence, and time represents the peak / valley / flat day (since the traceability analysis of electricity price is carried out for peak, valley and flat days respectively, the analysis steps for each day are the same, and in order to simplify, the subscript of time is omitted in the following). The multi-component empirical mode decomposition (MEMD) algorithm is used to decompose the multi-dimensional time sequence characteristics to obtain connotation modal components of different frequencies and trend items. Lempel-Ziv algorithm is used for reconstruction to obtain high-frequency items and low-frequency items, including: through the Lempel-Ziv algorithm, the complexity of the connotation modal component of each type of load day is calculated; the connotation modal components of each type of load day are sequentially added according to the complexity, and the sum of the first number of connotation modal components and the sum of the second number of connotation modal components of each type of load day are taken as the high-frequency item and the low-frequency item respectively; the first number is the number of the minimum connotation modal component whose complexity sum is greater than the reconstruction threshold, starting from the connotation modal component with the largest complexity of each type; the second number refers to the number of connotation modal components other than the first number.

[0064] The Lempel-Ziv algorithm is used to reconstruct the multiple connotation modal components of each type of load day to obtain the high-frequency item and the low-frequency item of each type of load day. Taking the electricity price data as an example, the multi-dimensional time sequence of each type of load day is input into the MEMD method, and the decomposition module and the trend item of each day are output, wherein is the number of decomposition modules, are the decomposition values of electricity price and influencing factors on the th decomposition module, respectively, are the trend items of electricity price and influencing factors, respectively, 、 and The shape is the same, that is, the same number of rows and columns.

[0065] The Lempel-Ziv algorithm reconstruction step is to calculate the lz value of each When the cumulative value of each IMF exceeds the threshold value, the corresponding IMF is the high-frequency term, and the threshold value is generally 0.8, specifically:

[0066] ;

[0067] If the above formula is satisfied, the sum of to is high frequency, and the sum of the remaining IMF is low frequency, and the trend term represents the trend of the sequence. The frequency band in the present application refers to high frequency, low frequency and trend term.

[0068] The high-frequency term, low-frequency term and trend term of each type of load day are taken as the frequency band multi-dimensional time series data.

[0069] Step three, time division day frequency band construction feature engineering execution, for each of the frequency band multi-dimensional time series data, a frequency band data regression model is established and a core feature set is screened. For the high-frequency term of short-term fluctuation, the low-frequency term of medium-term fluctuation and the trend term of long-term trend, the feature engineering of each time day is constructed to determine the regression relationship of each frequency band monitoring data and its supply and demand influencing factors. Feature engineering is used to determine the degree of influence of internal and external influencing factors on each decomposition module of monitoring data. The performance form is the sorting of the correlation coefficient between the features and the monitoring data in each decomposition module. The decomposition module includes a peak day decomposition module, a valley day decomposition module and a flat day decomposition module; each type of decomposition module includes a high-frequency term, a low-frequency term and a trend term.

[0070] Feature extraction is performed on the frequency band multi-dimensional time series data to obtain an internal feature sequence and a plurality of external feature sequences; the internal feature sequence is used to represent the correlation change relationship of the monitoring data in the time dimension; and the external feature sequence is used to represent the change of other data. For example, time series features, load features, weather features, fuel features and carbon emission features are extracted respectively. Among them, the time series features are internal influence features of the monitoring data, and the remaining features are external influence features. For example, the monitoring data is a time series, and the internal influence feature thereof refers to the lag value of the decomposition module of the monitoring data itself; specifically, the high-frequency term of the monitoring data on day t may be affected by the high-frequency term of the monitoring data on day t-1. The external influence feature refers to factors other than the lag value of the monitoring data itself, including temperature, wind speed and other weather conditions, fuel data and carbon data, etc.

[0071] The internal feature sequence refers to the historical lag value of the node monitoring data sequence in each decomposition module item. The lag value threshold of the monitoring data decomposition time sequence is determined based on the ACF value and the PACF value. The historical data of each lag time interval is extracted with the lag value threshold as the upper limit. For example, for the historical lag value of the high-frequency monitoring data, the value of the monitoring data high-frequency item at day t may be affected by day t-1 and day t-2; and the low-frequency item may be affected by day t-1 to day t-10. For another example, if the lag value threshold is determined to be 5 according to the PACF and ACF values, the correlation between the monitoring data at day t and the monitoring data sequence from day t-1 to day t-5 is constructed, that is, the monitoring data at day t is predicted by the monitoring data sequence from day t-1 to day t-5.

[0072] The correlation coefficients of the monitoring data sequence with the internal feature sequence and the plurality of external feature sequences are calculated, and the plurality of feature sequences are sorted in descending order of the absolute values of the correlation coefficients to obtain a feature importance sequence. The monitoring data sequence refers to a sequence composed of monitoring data values. For example, all the extracted features are calculated based on the Pearson correlation coefficient to obtain the importance of each feature in affecting the monitoring data. Then, the features are sorted in descending order of importance to form a feature importance sequence.

[0073] The expression of the Pearson correlation coefficient is as follows:

[0074]

[0075] Wherein, PCC represents the Pearson correlation coefficient value; and are the monitoring data and a certain feature sequence, respectively, and are the average values of the two sequences.

[0076] A frequency band data regression model is constructed, and based on the performance of the frequency band data regression model under the importance threshold of multiple scales, a plurality of feature sequences are screened to obtain an initial core feature set, including: determining a current core feature set based on a current importance threshold; dividing the current core feature set into a plurality of training sets and a validation set; training an initial frequency band data regression model through the plurality of training sets to obtain a trained frequency band data regression model; verifying the trained frequency band data regression model through the validation set to obtain a model performance; determining a new importance threshold, and taking the new importance threshold as the current importance threshold, repeating the determination of the performance of the trained frequency band data regression model; taking the model with the best performance as the final frequency band data regression model, taking the corresponding current core feature set as the initial core feature set, and taking the current importance threshold as the final importance threshold; the best performance refers to the minimum mean square error of the model. ​

[0077] For example, the K-fold cross-validation method is used to cross-check each frequency band module to determine the optimal threshold value for feature screening. According to the threshold value, the influence of each feature on the monitoring data in each time period and each frequency band is determined. The optimal threshold value refers to the threshold value of the correlation between the feature and the monitoring data. For example, if the threshold value is set to 0.5 according to the cross-checking, the prediction model performs best, which means that only the features with a correlation greater than 0.5 are selected, and the features with a correlation less than 0.5 are deleted. The correlation coefficient between the monitoring data and the features in each time period and each frequency band is determined. Each time period refers to the high frequency, low frequency, and trend term in the peak, valley, and flat period.

[0078] The specific steps of the K-fold cross-validation method are as follows:

[0079] The training set of each decomposition module is divided into multiple folds (e.g., 5 folds or 10 folds, etc.). Each time, one fold is used as the validation set, and the remaining folds are used as the training set, ensuring that each fold has the opportunity to be used as the validation set. Next, set the threshold range. Determine the threshold range to be tried. For example, start from 0.1 and increment by 0.1 to 0.9 to form multiple different threshold points for subsequent evaluation of the impact of different thresholds on model performance. For each determined threshold, use a simple machine learning model (such as LSTM, ELM, XGBoost) to train on the training set and predict on the validation set. The performance of the model at this threshold is evaluated by calculating the prediction error (such as mean square error), and the model performance index corresponding to each threshold is recorded. According to the relationship between the threshold and the model performance analysis results, select the threshold that makes the model perform best (such as the smallest mean square error) on the validation set as the best threshold. According to the determined best threshold, screen the sorted feature importance sequence, select those features with importance higher than the threshold as core features, and construct a feature matrix for different frequency bands and time periods.

[0080] Based on the correlation coefficients between multiple feature sequences, perform redundancy checking on the initial core feature set, eliminate redundant feature sequences, and obtain the core feature set. For example, perform redundancy checking on the features based on the Pearson correlation coefficient. Calculate the correlation coefficient between the features, and for highly correlated feature pairs, retain one representative feature and eliminate the other. For example, in addition to the lag value of the monitoring data itself, if the correlation between temperature and humidity is strong, select the one with the highest correlation coefficient with the monitoring data and delete the other. Determine the regression model of the monitoring data in each time period and each frequency band using the Pearson correlation coefficient as the parameter.

[0081] Step four, the abnormal data identification module is executed, based on the core feature set of the to-be-evaluated period, the monitoring data prediction value of the to-be-evaluated period is output through the frequency band data regression model, and based on the monitoring data prediction value and the target evaluation data, the abnormal monitoring data is determined. The regression model of step three describes the change of the monitoring data affected by the normal supply and demand fluctuation of the market, and the evaluation logic of the abnormal monitoring data is that the monitoring data fluctuation caused by the normal supply and demand change is the abnormal monitoring data.

[0082] For example, the monitoring data prediction value of each frequency band and each period is calculated respectively, and after integration, it is compared with the target monitoring data. Each frequency band and each period refers to high, medium and low frequency bands, and peak, valley and flat periods. An abnormal monitoring data evaluation index is designed, and if the abnormal index exceeds the threshold value, it means that the monitoring data is abnormal. The threshold value refers to the threshold value of the index, which can be determined according to a large amount of historical data, expert experience and actual market conditions. For example, the abnormal index value of the electricity price is calculated to be 1, and the threshold value is 0.5, so the electricity price data is abnormal. The monitoring data abnormality evaluation index is:

[0083] ;

[0084] Among them, is the monitoring data abnormality index of each period, and are the integrated monitoring data prediction value and the target evaluation monitoring data (the actual monitoring data monitoring value), and if the monitoring data abnormality index exceeds the threshold value, it is abnormal data.

[0085] When the abnormal monitoring data is abnormal electricity price data, step five is further included, which processes the abnormal electricity price data to determine the behavior causing the abnormality. It includes: determining the category to which the abnormal electricity price data belongs; the categories include peak day high frequency abnormality, peak day low frequency abnormality, peak day trend abnormality, valley day high frequency abnormality, valley day low frequency abnormality, valley day trend abnormality, flat day high frequency abnormality, flat day low frequency abnormality and flat day trend abnormality; based on the category to which the abnormal electricity price data belongs, one or more category indexes are determined; the category index is matched to the corresponding index range to determine the behavior causing the abnormality.

[0086] For example, an index system of abnormal electricity price influencing factors is constructed for each frequency band and each period, and the basic information of the unit and the unit declaration information data are used to identify the abnormal behavior causing the abnormal electricity price. For example, these indexes will determine a threshold value according to experience or data, and if the actual calculation is outside the threshold value, it means that there is such abnormal behavior. For example: if the peak time high frequency item appears abnormal electricity price, the evaluation index of the peak time high frequency can be calculated accordingly to determine which behavior causes the abnormality, so as to realize abnormality tracing and behavior positioning.

[0087] Peak day high frequency evaluation index. The sequence characteristics corresponding to the peak time high frequency are large load fluctuations, and the sequence presents short-term dramatic fluctuations. The corresponding behavior is the short-term sudden abnormal behavior at the peak time, such as reporting high price at the peak time, sudden shortage of available capacity outside the plan, etc. The short-term abnormal behavior of electricity peak period causing abnormal electricity price includes economic retention, physical retention, and congestion arbitrage. The specific indicators are:

[0088] Economic retention index:

[0089] Peak day declared price deviation rate = (peak day price - short-term marginal cost) / short-term marginal cost;

[0090] Physical retention index:

[0091] Unit capacity sudden drop rate = (rated capacity - peak day actual available capacity) / rated capacity;

[0092] Congestion arbitrage:

[0093] Artificial congestion yield rate = (peak day congestion yield - normal congestion yield average) / normal congestion yield average;

[0094] Among them, the determination of normal congestion yield average is based on the virtual clearing of units in peak day according to the declared cost.

[0095] Peak day low frequency evaluation index. The characteristics corresponding to the peak time low frequency are large load fluctuations, and the sequence presents periodic fluctuations. The corresponding behavior is the periodic abnormal behavior that may occur at the peak time. For example, seasonal capacity retention is achieved through maintenance. The periodic abnormal behavior of electricity peak day causing abnormal electricity price includes seasonal capacity retention, and electricity generating units intentionally delaying unit commissioning or maintenance before the high temperature or cold wave season to create seasonal shortage. The specific indicators are:

[0096] Peak day maintenance deviation degree = (actual maintenance start time - historical maintenance start time average) / standard deviation;

[0097] The standard deviation is calculated based on the deviation day sequence of historical maintenance start time and historical maintenance start time average.

[0098] Peak day maintenance extension rate = (actual maintenance duration - declared maintenance duration) / same type unit average maintenance duration;

[0099] Peak day trend evaluation index. The trend abnormal behavior of electricity peak day causing abnormal electricity price includes artificially reducing the flexibility of resources and actively maintaining the transmission bottleneck. The specific indicators are:

[0100] Flexibility gap growth rate = Δ (peak day load demand - adjustable resource capacity) / year;

[0101] ;

[0102] wherein, is the annual congestion revenue growth rate, and the congestion revenue trend is the three-year moving average of the annual congestion revenue growth rate.

[0103] The valley day high-frequency evaluation index. The short-term abnormal behavior of causing abnormal electricity prices during the low electricity consumption period occurs in groups that contain both energy storage and renewable energy. Renewable over-forecasted power further lowers electricity prices during the low consumption period, and energy storage can arbitrage through charging due to the pressure of consumption. The specific index is:

[0104] Valley day charging arbitrage index = corr (renewable energy output prediction positive error, battery charging bid volume);

[0105] wherein, corr represents the correlation coefficient.

[0106] The valley day low-frequency evaluation index. The periodic abnormal behavior of causing abnormal electricity prices on low electricity consumption days includes the scheduling control of hydropower to affect electricity prices. The specific index is:

[0107] Hydrological scheduling deviation index = (actual water release - optimized scheduling amount) / basin average;

[0108] The valley day trend evaluation index. The trend abnormal behavior of causing abnormal electricity prices on low electricity consumption days includes the suppression of negative electricity prices, the delay of energy storage economy, and the maintenance of traditional power source revenue. The specific index is:

[0109] Negative electricity price suppression index = 1 - (number of markets allowing negative electricity prices / total number of markets);

[0110] wherein, the number of markets allowing negative electricity prices refers to the number of markets allowing market clearing prices to be negative.

[0111] The flat day high-frequency evaluation index. The moderate load demand of electricity consumption on flat days causes short-term abnormal behavior of abnormal electricity prices, including unit operation climbing ability to cause short-term supply and demand imbalance. The specific index is:

[0112] Flat day climbing manipulation index = |approved climbing rate - actual climbing rate| / technical limit;

[0113] The flat day low-frequency evaluation index. The periodic abnormal behavior of causing abnormal electricity prices on flat days includes intentionally underreporting the climbing ability during the spring and autumn load transition season, delaying the response speed of unit start and stop, and extending the operation time of high-priced units. The evaluation index of flat day low frequency is mainly the transition season price difference elasticity:

[0114] Transition season price difference elasticity = |(maximum value of transition season flat day electricity price - minimum value of transition season flat day electricity price) / load change rate;

[0115] wherein, the transition season mainly refers to the spring and autumn seasons.

[0116] Normal day trend evaluation index. The trend abnormal behavior of abnormal electricity price on normal day includes the hysteresis of climbing standard. The specific index is:

[0117] Standard hysteresis coefficient = (current standard - international advanced standard) / advanced standard;

[0118] Through the index system established in step five, the behavior tracing and responsibility positioning of abnormal electricity price on peak day, valley day and normal day in short term, medium term and long term are carried out respectively. Finally, a complete market force tracing path is formed, the behavior attribution and mechanism identification are clear, and the empirical evidence is provided for post-supervision and policy optimization.

[0119] The application also provides an abnormal data identification system of time-sharing day and frequency band feature engineering, as shown in Figure 2 The abnormal data identification system of time-sharing day and frequency band feature engineering comprises a data collection and preprocessing module, a data decomposition and reconstruction module, a time-sharing day and frequency band feature engineering construction module and an abnormal data identification module. The data collection and preprocessing module is used for processing initial multi-source data to obtain multi-dimensional time series data. The data decomposition and reconstruction module is used for decomposing and reconstructing the multi-dimensional time series data to obtain frequency band multi-dimensional time series data of multiple frequency bands. The time-sharing day and frequency band feature engineering construction module is used for establishing a frequency band data regression model and screening a core feature set for each frequency band multi-dimensional time series data. The abnormal data identification module is used for outputting a monitoring data prediction value of a to-be-evaluated period through the frequency band data regression model based on the core feature set of the to-be-evaluated period, and determining abnormal monitoring data based on the monitoring data prediction value and target evaluation data.

[0120] The above is only a preferred embodiment of the application and is not used to limit the application. For those skilled in the art, the application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A time-sharing day frequency band feature engineering abnormal data identification method, characterized in that, The method comprises the following steps: processing initial multi-source data to obtain multi-dimensional time series data; decomposing and reconstructing the multi-dimensional time series data to obtain frequency band multi-dimensional time series data of multiple frequency bands; for each frequency band multi-dimensional time series data, establishing a frequency band data regression model and screening a core feature set, comprising: extracting features from the frequency band multi-dimensional time series data to obtain an internal feature sequence and multiple external feature sequences; the internal feature sequence is used to represent the correlation change relationship of the monitoring data in the time dimension; the external feature sequence is used to represent the change of other data; calculating the correlation coefficients of the monitoring data sequence with the internal feature sequence and the multiple external feature sequences respectively, sorting the multiple feature sequences in descending order of the absolute values of the correlation coefficients to obtain a feature importance sequence; constructing a frequency band data regression model, and based on the performance of the frequency band data regression model under a multi-scale importance threshold, screening the multiple feature sequences to obtain an initial core feature set, comprising: determining a current core feature set based on the current importance threshold; dividing the current core feature set into multiple training sets and a validation set; training the initial frequency band data regression model through the multiple training sets to obtain a trained frequency band data regression model; verifying the trained frequency band data regression model through the validation set to obtain the performance of the model; determining a new importance threshold, and taking the new importance threshold as the current importance threshold to repeat the determination of the performance of the trained frequency band data regression model; taking the model with the best performance as the final frequency band data regression model, taking the corresponding current core feature set as the initial core feature set, and taking the current importance threshold as the final importance threshold; the best performance refers to the minimum mean square error of the model; based on the correlation coefficients between the multiple feature sequences, performing a redundancy check on the initial core feature set to eliminate redundant feature sequences to obtain a core feature set; when the abnormal monitoring data is abnormal electricity price data, further comprising processing the abnormal electricity price data to determine the behavior causing the abnormality, comprising: determining the category to which the abnormal electricity price data belongs; the categories include peak day high frequency anomaly, peak day low frequency anomaly, peak day trend anomaly, valley day high frequency anomaly, valley day low frequency anomaly, valley day trend anomaly, flat day high frequency anomaly, flat day low frequency anomaly, and flat day trend anomaly; determining one or more category indicators based on the category to which the abnormal electricity price data belongs; matching the category indicators to the corresponding indicator ranges to determine the behavior causing the abnormality; based on the core feature set of the to-be-evaluated period, outputting the monitoring data prediction value of the to-be-evaluated period through the frequency band data regression model, and determining the abnormal monitoring data based on the monitoring data prediction value and the target evaluation data. 2.The time-division fractional band feature engineering-based abnormal data identification method according to claim 1, characterized in that, The processing of the initial multi-source data to obtain the multi-dimensional time series data comprises: performing integrity checking on the obtained initial multi-source data to obtain an integrity checking result; the integrity checking is realized by checking the meta information of each type of data. Perform data outlier cleaning on the initial multi-source data with the integrity check result being complete, eliminate abnormal data, and obtain non-abnormal multi-source data; data outlier cleaning includes abnormal value detection of multi-dimensional data using Mahalanobis distance and abnormal value detection of single-dimensional data using Grubbs criterion cycle; Perform data missing filling on the non-abnormal multi-source data, and obtain complete multi-source data; data missing filling includes data filling of spatial data using Kriging interpolation method, data filling of time data using linear interpolation method, or data filling of geographic data using inverse distance weighted method; Divide the load day based on the load data in the complete multi-source data, and obtain multi-class load day and load day monitoring index; Divide other data in the complete multi-source data by day granularity, obtain multi-source day granularity data, and align the multi-source day granularity data with the multi-class load day and the load day monitoring index of the corresponding load day, to obtain multi-dimensional time series data of split-time day. 3.The time-division fractional band feature engineering-based abnormal data identification method according to claim 2, characterized in that, Decompose and reconstruct the multi-dimensional time series data to obtain frequency band multi-dimensional time series data of multiple frequency bands, including: Perform multi-element empirical mode decomposition on each class of load day to obtain connotation modal component and trend item of the multi-class load day; the connotation modal component includes multiple frequencies; Perform Lempel-Ziv algorithm reconstruction on multiple connotation modal components of each class of load day to obtain high-frequency item and low-frequency item of each class of load day; Take the high-frequency item, low-frequency item and trend item of each class of load day as the frequency band multi-dimensional time series data. 4.The time-division fractional band feature engineering-based abnormal data identification method of claim 3, wherein, Perform multi-element empirical mode decomposition to obtain connotation modal component and trend item, including: Construct multi-dimensional time series based on monitoring data and associated data to obtain multi-dimensional time series features; Decompose the multi-dimensional time series features by multi-element empirical mode decomposition algorithm to obtain connotation modal components of different frequencies and trend items; Perform Lempel-Ziv algorithm reconstruction to obtain high-frequency item and low-frequency item, including: Calculate the complexity of each class of load day connotation modal component by Lempel-Ziv algorithm; Add each class of load day connotation modal component in order of complexity size one by one, and take the sum of the first order number of connotation modal components and the sum of the second order number of connotation modal components of each class of load day as the high-frequency item and the low-frequency item; the first order number is the minimum number of connotation modal components from the connotation modal component with the highest complexity of each class, so that the sum of the connotation modal component complexity is greater than the minimum connotation modal component number; the second order number refers to the number of connotation modal components other than the first order number. 5.The time-division fractional band feature engineering-based abnormal data identification method according to claim 2, characterized in that, The division of other data in the complete multi-source data by day granularity to obtain multi-source day granularity data, including: Downsample fine-grained data by maximum triangle three-bar dimensionality reduction algorithm to obtain day granularity data; fine-grained data refers to data with a sampling frequency less than one day; Interpolate coarse-grained data by inverse distance weighted difference algorithm to obtain day granularity data; coarse-grained data refers to data with a sampling frequency greater than one day.

6. A time-of-day and frequency band feature engineering-based abnormal data identification system using the time-of-day and frequency band feature engineering-based abnormal data identification method according to any one of claims 1 to 5, characterized by, It includes a data collection and preprocessing module, a data decomposition and reconstruction module, a split-time day and frequency band feature engineering construction module, and an abnormal data identification module; The data collection and preprocessing module is configured to process the initial multi-source data to obtain multi-dimensional time series data; The data decomposition and reconstruction module is configured to decompose and reconstruct the multi-dimensional time series data to obtain frequency band multi-dimensional time series data of a plurality of frequency bands; The time and day division and frequency band division feature engineering construction module is configured to, for each of the frequency band multi-dimensional time series data, construct a frequency band data regression model and screen a core feature set; The abnormal data identification module is configured to, based on the core feature set of an evaluation period to be evaluated, output a monitoring data prediction value of the evaluation period to be evaluated through the frequency band data regression model, and determine abnormal monitoring data based on the monitoring data prediction value and target evaluation data.

Images