Abnormality detection method and device of power system, medium and equipment

Abstract

The invention discloses an anomaly detection method and device for a power system, a medium and equipment. According to the method, the feature data of the network traffic of the current power system is acquired, and the data is subjected to dimension reduction processing by using the principal component analysis method, so that key features are extracted. Then, the key features are input into a pre-trained linear regression model containing regularization terms, and the model is formed by training based on historical data, preset hyper-parameters and a loss function and can output flow prediction values of a plurality of continuous time points; and after an actual flow value of the current power system is obtained within a preset time interval, a residual error between the predicted value and an actual value is calculated. And finally, according to a preset quartile distance method and the residual value, determining abnormal points of the network flow of the power system, including single-point abnormality, persistence abnormality and mode abnormality. The problem that the abnormality of the power system cannot be accurately and efficiently detected in the prior art is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of abnormality detection of power systems, and in particular to a method, device, medium and equipment for abnormality detection of power systems. Background Art

[0002] With the rapid development of smart grids and the Industrial Internet of Things, the complexity and data volume of power system networks are constantly increasing. Monitoring and anomaly detection of power system network traffic are of great significance for ensuring the stable operation of the power system and preventing network attacks and equipment failures. In existing technologies, anomaly detection of network traffic mainly relies on the following methods:

[0003] Statistical analysis-based methods: Detect anomalies by analyzing the statistical characteristics of network traffic (such as mean, variance, peak value, etc.). This method is simple and easy to implement, but its ability to detect complex network environments and hidden abnormal behaviors is limited.

[0004] Machine learning-based methods include neural networks and support vector machines (SVMs). These methods can learn complex patterns of network traffic, but they usually require a large amount of training data and are subject to the risk of overfitting.

[0005] Methods based on time series analysis, such as the Autoregressive Moving Average (ARMA) model and the Seasonal Autoregressive Moving Average (SARIMA) model, are suitable for traffic data with obvious temporal patterns, but are less capable of detecting nonlinear characteristics and sudden anomalies. However, existing evaluation methods have the following shortcomings:

[0006] High model complexity: Many machine learning models (such as deep neural networks) require a lot of computing resources for training and inference, making it difficult to respond quickly in real-time monitoring scenarios.

[0007] Overfitting problem: Complex models are prone to overfitting the training data, resulting in insufficient generalization ability to new data in practical applications.

[0008] Insufficient anomaly detection accuracy: Existing methods have high false positive and false negative rates when detecting hidden anomalies (such as low-rate DDoS attacks, malicious program heartbeat signals, etc.).

[0009] Lack of explainability: Although some deep learning-based models have good performance, it is difficult to explain their decision-making process, which is not conducive to the understanding and application of power system operation and maintenance personnel.

[0010] These deficiencies result in the inability of existing technologies to accurately and efficiently detect anomalies in the power system. Summary of the Invention

[0011] The present invention provides a method, device, medium and equipment for detecting anomalies in an electric power system, so as to solve the problem in the prior art that anomalies occurring in the electric power system cannot be accurately and efficiently detected.

[0012] In a first aspect, the present application provides a method for detecting anomalies in a power system, comprising:

[0013] Obtain characteristic data of current power system network traffic;

[0014] According to a preset principal component analysis method, the feature data is reduced in dimension to extract key features;

[0015] Inputting the key features into a preset linear regression model containing a regularization term so that the linear regression model outputs flow prediction values at multiple consecutive time points; wherein the linear regression model is obtained by training an initial linear regression model on characteristic data of historical power system network flow according to preset hyperparameters and a preset loss function;

[0016] According to a preset time interval, the actual flow value of the current power system is obtained, and the residual between the flow prediction value and the actual flow value is calculated;

[0017] According to the preset interquartile range method and the residual, the abnormal point of the current power system network flow is determined.

[0018] This application obtains the characteristic data of the current power system network flow and uses the principal component analysis (PCA) method to reduce the dimension of the characteristic data and extract key features. This process not only reduces the feature dimension and reduces the computational complexity, but also retains the information that is most influential for flow prediction, thereby improving the efficiency and accuracy of the model. Subsequently, the key features after dimensionality reduction are input into a pre-trained linear regression model with a regularization term. The model is trained based on historical data and preset hyperparameters and loss functions, and has good generalization ability and anti-overfitting properties. The model outputs flow prediction values for multiple consecutive time points, providing a benchmark for real-time monitoring and prediction. Furthermore, by calculating the residual between the predicted value and the actual flow value and combining the interquartile range (IQR) method to determine anomalies, real-time anomaly detection of power system network flow is achieved. This method can effectively identify single-point anomalies, persistent anomalies, and pattern anomalies, improving the accuracy and response speed of anomaly detection. Overall, this application significantly improves the accuracy of power system network flow prediction and the reliability of anomaly detection by optimizing feature processing and model construction. This application effectively solves the problem that the existing technology cannot accurately and efficiently detect anomalies in the power system.

[0019] As a preferred embodiment of the first aspect, the feature data is subjected to dimensionality reduction according to a preset principal component analysis method to extract key features, specifically:

[0020] Calculating a covariance matrix of the feature data;

[0021] According to a preset calculation formula, the eigenvalues and eigenvectors of the covariance matrix are obtained;

[0022] Sorting by the size of the eigenvalues, selecting the first M eigenvectors whose cumulative variance contribution rate reaches a preset threshold as principal components; wherein M is a positive integer;

[0023] The feature data is projected onto the principal component to obtain the key features after dimensionality reduction.

[0024] In this preferred embodiment, the present application calculates the covariance matrix of the feature data and solves its eigenvalues and eigenvectors, and can sort the eigenvectors according to the size of the eigenvalues, and then select the top M eigenvectors whose cumulative variance contribution rate reaches a preset threshold as principal components. This process not only effectively reduces the dimension of the feature data and reduces the computational complexity, but also retains the most influential feature information for network traffic prediction and anomaly detection. The feature data is projected onto these principal components to obtain the key features after dimensionality reduction, which further improves the training efficiency and prediction accuracy of the model, while enhancing the model's processing ability and generalization performance for high-dimensional data, providing a solid technical foundation for efficient monitoring and anomaly detection of power system network traffic.

[0025] As a preferred embodiment of the first aspect, the abnormal point of the current power system network flow is determined according to the preset interquartile range method and the residual, specifically:

[0026] Calculate the first quantile and the second quantile of the sequence of the residuals;

[0027] The interquartile range is calculated based on the first quartile and the second quartile;

[0028] Determining an abnormal threshold range based on the interquartile range;

[0029] Determining an abnormal point of current power system network flow based on the residual and the abnormal threshold range;

[0030] The abnormal points include single point abnormalities, continuous abnormalities and pattern abnormalities.

[0031] In this preferred embodiment, the present application calculates the first quantile (Q1) and the third quantile (Q3) of the residual sequence, and further solves the interquartile range (IQR=Q3-Q1). The present invention can determine the abnormal threshold range based on IQR, that is, the lower limit is Q1-1.5×IQR and the upper limit is Q3+1.5×IQR. This method uses the comparison of residuals and abnormal threshold ranges to accurately identify abnormal points in the power system network flow. Specifically, a single point anomaly refers to the residual exceeding the threshold range at a certain time point, a persistent anomaly refers to the residual exceeding the threshold range at multiple consecutive time points, and a pattern anomaly refers to the occurrence of unnatural periodic fluctuations in the residual sequence. This anomaly detection method based on the interquartile range can not only effectively reduce the false alarm rate and the missed alarm rate, but also quickly respond to different types of anomalies, significantly improving the accuracy and reliability of power system network flow monitoring, and providing a strong guarantee for the stable operation of the power system.

[0032] As a preferred embodiment of the first aspect, the linear regression model is obtained by training an initial linear regression model on historical characteristic data of power system network traffic according to preset hyperparameters and a preset loss function, specifically:

[0033] Obtain historical characteristic data of power system network traffic;

[0034] Inputting the historical feature data into an initial linear regression model so that the initial linear regression model trains the historical feature data according to a preset regularization strength, learning rate, and loss function; wherein the loss function includes a mean square error term and an L2 regularization term;

[0035] When the value of the loss function no longer decreases within a preset time period or reaches a preset number of iterations, the training is stopped to obtain the linear regression model.

[0036] The initial linear regression model is trained on the historical feature data according to a preset regularization strength, learning rate and loss function, specifically:

[0037] The optimization objective function of the initial linear regression model during training is:

[0038]

[0039] Where, α≥0 is the preset regularization strength; Represents the fitting error, which is used to measure how well the model fits the training data; is the sum of the squares of the parameters, which is used to control the size of the parameters.

[0040] In this preferred embodiment, the present application obtains historical characteristic data of the power system network flow and inputs it into the initial linear regression model. The present invention uses a preset regularization strength, learning rate and loss function to train the model. The loss function contains a mean square error term and an L2 regularization term, wherein the mean square error term is used to measure the deviation between the model prediction value and the actual value, and the L2 regularization term prevents overfitting by limiting the size of the model parameters. When the value of the loss function no longer decreases within a preset time period or reaches a preset number of iterations, the training process stops, and an optimized linear regression model is finally obtained. This process not only improves the model's ability to fit historical data, but also enhances the model's generalization ability through regularization means, enabling it to more accurately predict future flow and detect anomalies.

[0041] As a preferred embodiment of the first aspect, the abnormal point of the current power system network flow is determined based on the residual and the abnormal threshold range; wherein the abnormal point includes a single point abnormality, a continuous abnormality, and a pattern abnormality, specifically:

[0042] If the residual at the first time point exceeds the abnormality threshold range, determining that the first time point is a single point abnormality;

[0043] If the residuals of multiple consecutive time points exceed the abnormality threshold range, it is determined that the multiple consecutive time points are persistent abnormalities;

[0044] If the sequence of the residuals exhibits non-natural periodic fluctuations, it is determined to be a pattern anomaly.

[0045] In this preferred embodiment, the present application can accurately identify different types of abnormal points in the power system network traffic by setting an abnormal threshold range based on the residual. Specifically, if the residual at a certain time point exceeds the abnormal threshold range, it is determined to be a single point abnormality, which helps to promptly discover occasional abnormal events; if the residuals at multiple consecutive time points all exceed the abnormal threshold range, it is determined to be a persistent abnormality, which usually indicates a potential network attack or equipment failure, and immediate measures need to be taken; in addition, if the residual sequence shows unnatural periodic fluctuations, it is determined to be a pattern abnormality, which may reveal the periodic activities of malicious programs. This hierarchical anomaly detection method not only improves the accuracy and reliability of anomaly detection, but also can quickly locate the root cause of the problem according to the type of anomaly, thereby significantly improving the monitoring efficiency and security of the power system network, and providing a strong guarantee for the stable operation of the power system.

[0046] As a preferred embodiment of the first aspect, the characteristic data of the current power system network traffic include packet rate, byte rate, peak traffic, traffic standard deviation, number of protocol packets, number of abnormal protocols, timestamp, number of failed connections, packet loss rate, retransmission rate, average packet size and maximum packet size.

[0047] In this preferred embodiment, the present application can comprehensively capture the multi-dimensional information of the power system network traffic by selecting a variety of feature data including packet rate, byte rate, peak traffic, traffic standard deviation, the number of protocol packets, the number of abnormal protocols, timestamp, number of failed connections, packet loss rate, retransmission rate, average packet size and maximum packet size. These features not only cover the basic statistical characteristics of the traffic (such as rate, standard deviation), but also include information at the protocol level (such as the number of protocol packets, the number of abnormal protocols) and time-related features (such as timestamps). This multi-dimensional feature selection enables the model to more accurately reflect the dynamic changes of network traffic, thereby improving the detection accuracy of network anomalies. For example, packet rate and byte rate can reflect the real-time load of traffic; the number of abnormal protocols and the number of failed connections can help identify potential network attacks or equipment failures; and timestamps and periodic features (such as average packet size) can help discover the periodic activities of malicious programs.

[0048] In a second aspect, the present application provides an abnormality detection device for a power system. The abnormality detection device for a power system includes an acquisition module, a dimensionality reduction module, a prediction module, a calculation module, and a detection module;

[0049] The acquisition module is used to obtain characteristic data of the current power system network traffic;

[0050] The dimensionality reduction module is used to reduce the dimensionality of the feature data according to a preset principal component analysis method to extract key features;

[0051] The prediction module is used to input the key features into a preset linear regression model containing a regularization term, so that the linear regression model outputs flow prediction values at multiple consecutive time points; wherein the linear regression model is obtained by training an initial linear regression model on characteristic data of historical power system network flow according to preset hyperparameters and a preset loss function;

[0052] The calculation module is used to obtain the actual flow value of the current power system according to a preset time interval and calculate the residual between the flow prediction value and the actual flow value;

[0053] The detection module is used to determine the abnormal points of the current power system network flow according to the preset interquartile range method and the residual.

[0054] This device uses five modules to divide the work and coordinate work to more accurately detect anomalies in the power system. This application obtains the characteristic data of the current power system network flow and uses the principal component analysis (PCA) method to reduce the dimension of the characteristic data and extract key features. This process not only reduces the feature dimension and reduces the computational complexity, but also retains the information that has the greatest impact on flow prediction, thereby improving the efficiency and accuracy of the model. Subsequently, the key features after dimensionality reduction are input into a pre-trained linear regression model containing a regularization term. The model is trained based on historical data and preset hyperparameters and loss functions, and has good generalization ability and anti-overfitting characteristics. The model outputs flow prediction values for multiple consecutive time points, providing a benchmark for real-time monitoring and prediction. Furthermore, by calculating the residual between the predicted value and the actual flow value and combining the interquartile range (IQR) method to determine the anomaly point, real-time anomaly detection of the power system network flow is achieved. This method can effectively identify single-point anomalies, persistent anomalies, and pattern anomalies, improving the accuracy and response speed of anomaly detection. Overall, this application significantly improves the accuracy of power system network flow prediction and the reliability of anomaly detection by optimizing feature processing and model construction. This effectively solves the problem that existing technologies cannot accurately and efficiently detect anomalies in power systems.

[0055] In a third aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium including a stored computer program. When the computer program is executed, the device containing the computer-readable storage medium is controlled to execute the power system anomaly detection method described above. The beneficial effects thereof are the same as those of the power system anomaly detection method provided in the first aspect of the present application.

[0056] In a fourth aspect, the present application provides a terminal device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements any one of the power system anomaly detection methods described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 : A flow chart of an embodiment of a method for detecting abnormality in a power system provided by the present application;

[0058] Figure 2 : A flow chart of an embodiment of the anomaly detection data processing process provided by this application;

[0059] Figure 3 : A flowchart of an embodiment of each module for abnormality detection of a power system provided by this application;

[0060] Figure 4: A structural diagram of an embodiment of an abnormality detection device for an electric power system provided in this application. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0062] Example 1

[0063] Please refer to Figure 1 , which is a method for detecting abnormalities in a power system provided by an embodiment of the present invention.

[0064] In this embodiment, the process of the power system abnormality detection method in this application is described in detail through steps S01-S05.

[0065] This application uses the Ridge linear regression model to predict the network security flow data of the power system. The security of the power system network is particularly important in the smart grid environment. Predicting network security flow can help the operation and maintenance team detect potential attacks or abnormal behaviors in advance, thereby effectively preventing risks. The Ridge linear regression model is suitable for processing complex, high-dimensional and multicollinear network flow data due to its stability and anti-overfitting ability. The overall process of this system method is as follows Figure 2 As shown:

[0066] The first step is to select the relevant features of power system network security flow prediction;

[0067] The second step is to collect relevant characteristic data of network security flows of various power systems and divide them into training sets and test sets, and then perform time series processing on the training set data;

[0068] The third step is to perform feature engineering on the training set data (e.g. standardization);

[0069] The fourth step is to use PCA (principal component analysis) method to select the main k features;

[0070] The fifth step is to initialize a set of model hyperparameter values, such as [0.01 0.1 1 10 100];

[0071] The sixth step is to train the ridge linear regression model.

[0072] After the seventh step of training the ridge linear regression model, input the test set to observe whether the data results meet expectations.

[0073] In the eighth step, if it does not meet the requirements, adjust the hyperparameter values and retrain the model until the test results meet the expectations.

[0074] In the ninth step, if it meets the expectations, output the prediction results of the model.

[0075] S01: Obtain the characteristic data of the current power system network traffic.

[0076] S02: According to the preset principal component analysis method, reduce the dimension of the characteristic data and extract the key features.

[0077] As a preferred embodiment of Embodiment 1, the method of reducing the dimension of the characteristic data and extracting the key features according to the preset principal component analysis method is specifically as follows:

[0078] Use the PCA (principal components analysis) method for feature reduction. After selecting multiple features, in order to eliminate some feature data with less impact on the results and reduce the computational complexity of the model algorithm, the PCA method is used here to select more important feature data.

[0079] For the standardized matrix S, first calculate the average value for each column in S Then subtract this average value from all points in the matrix Thus, matrix B is obtained.

[0080] Next, find the covariance matrix C of B, and the calculation is as follows:

[0081]

[0082] The obtained matrix C is a p×p matrix. Then find the eigenvalues and eigenvectors of matrix C. Since C is a p×p matrix, p matrix eigenvalues λ i and eigenvectors v i can be obtained. Sort according to the magnitudes of the eigenvalues, and then use the cumulative variance contribution rate method (Explained Variance) to calculate the contribution rate of each eigenvalue:

[0083]

[0084] Next, calculate the cumulative contribution rate:

[0085]

[0086] Select the value of M (M < p) such that the cumulative contribution rate reaches a predetermined threshold, usually this threshold can be 90% or 95%. That is, select the first k principal components so that the cumulative contribution rate of these principal components is as close as possible to 1 (100%) but not less than 90% or 95%.

[0087] Next, the eigenvectors corresponding to the first M eigenvalues are selected according to the selected M value and arranged from top to bottom into a matrix P. Finally, the P matrix is multiplied by the transpose of the B matrix to obtain the matrix X'=PB after the PCA method is completed. T :

[0088]

[0089] After completing PCA (Principal Component Analysis) on the standardized matrix, a new initial matrix X' is obtained.

[0090] In this preferred embodiment, the present application calculates the covariance matrix of the feature data and solves its eigenvalues and eigenvectors, and can sort the eigenvectors according to the size of the eigenvalues, and then select the top M eigenvectors whose cumulative variance contribution rate reaches a preset threshold as principal components. This process not only effectively reduces the dimension of the feature data and reduces the computational complexity, but also retains the most influential feature information for network traffic prediction and anomaly detection. The feature data is projected onto these principal components to obtain the key features after dimensionality reduction, which further improves the training efficiency and prediction accuracy of the model, while enhancing the model's processing ability and generalization performance for high-dimensional data, providing a solid technical foundation for efficient monitoring and anomaly detection of power system network traffic.

[0091] S03: Input the key features into a preset linear regression model containing a regularization term, so that the linear regression model outputs flow prediction values for multiple consecutive time points; wherein, the linear regression model is obtained by training an initial linear regression model on characteristic data of historical power system network flow based on preset hyperparameters and a preset loss function.

[0092] As a preferred embodiment of the first embodiment, the key features are input into a preset linear regression model containing a regularization term, so that the linear regression model outputs flow prediction values at multiple consecutive time points, wherein the linear regression model is obtained by training an initial linear regression model on characteristic data of historical power system network flow according to preset hyperparameters and a preset loss function, specifically:

[0093] First, obtain the characteristic data of historical power system network traffic;

[0094] The characteristic data includes packet rate, byte rate, peak traffic, traffic standard deviation, protocol distribution (i.e., the number of protocol packets), number of abnormal protocols (unexpected number of network protocol packets), timestamp (indicating time characteristics such as hour of the day, day of the week, month, etc.), number of failed connections, packet loss rate, retransmission rate, average packet size and maximum packet size.

[0095] After obtaining the characteristic data of the current power system network flow, the method further includes:

[0096] After feature selection, the data is time-serialized. For example, time serialization can be performed on time ranges such as one minute, ten minutes, one hour, and 24 hours. That is, data is collected within this time range, including the selected feature data and traffic result data.

[0097] When obtaining the result value of time series, you can obtain it as follows: For example, if you choose to collect data by day, you can select the data from the 1st to the 7th day as the input sample, and the traffic package data of the 8th day as the result output value; the data from the 2nd to the 8th day as the input sample, and the traffic package data of the 9th day as the result output value. According to the above method, the data collected and obtained are divided into a part of the data as the training set and a part as the test set. The initial training set data constructs the following initial matrix C and result matrix Y:

[0098]

[0099] Y=(y1 y2…y N ) T

[0100] Among them, N represents the number of samples collected, that is, the matrix columns represent the collected traffic feature data. p represents the number of features, and the c value of the matrix represents the collected feature observation value of a certain sample; Y is the traffic result corresponding to the result matrix identification sample, y i Indicates the traffic result value that can be predicted for the i-th sample. For example, you can select the number of traffic packets or a specific data type value you want to predict (such as the number of TCP packets, the number of abnormal data packets, etc.).

[0101] Data cleaning is a key step in data analysis and modeling to ensure data quality and integrity. The initial data collected may contain noise or missing values. Missing values can be filled with mean, median, mode or interpolation. The next step is to normalize the matrix C. Here, the cosine distance metric is used for normalization, that is, each matrix element c of the initial matrix c is normalized. ij calculate Thus we get the standardized matrix S:

[0102]

[0103] Then, the new initial matrix X' is obtained after PCA (principal component analysis) of the standardized matrix;

[0104] First, we need to initialize a set of values for the hyperparameter α that defines the regularization strength of the model, such as [0.01 0.1 110100]. Then we get the new initial matrix X:

[0105]

[0106] That is, add a column to the leftmost side of the matrix based on X', with all values being 1.

[0107] Next, we will use the latest initial matrix X and the result matrix Y to train the model. Ridge regression limits the size of model parameters by adding an L2 norm regularization term to the loss function. The optimization objective function of the model is as follows:

[0108]

[0109] Among them, α ≥ 0 is the defined regularization strength hyperparameter, and you can choose an initialization from the previously defined values before training the model; Represents the fitting error, which measures how well the model fits the training data; is the sum of the squares of the parameters, controlling the size of the parameters.

[0110] Since only β is an unknown parameter in the objective function L(β), the initial construction of the ridge regression model can be completed after calculating β. The calculation formula is as follows:

[0111] β=(X T X+αI) -1 X T Y

[0112] Where I represents the unit matrix. The β value can be obtained according to the above formula, and the ridge regression model can be constructed by substituting it into the objective function.

[0113] In order to optimize the model, the training set data and the cross validation (CV) method can be used to evaluate the effect of different regularization strengths α and select the optimal parameters:

[0114] (1) Divide the training set into k subsets (the k value should be less than the number of samples in the training set), select one of the subsets as the validation set each time, and the remaining k-1 subsets as the training set.

[0115] (2) For each α value, the performance on the test set is calculated after the training model is completed. The mean-square error (MSE) can be calculated as follows:

[0116]

[0117] Where n represents the number of test set samples, Represents the model output value, y iRepresents the true value in the test set. Select the α value that minimizes the mean square error and substitute it into the objective function L(β) to retrain the model parameter β. Substituting it into y = βX, we can obtain the flow value predicted by the model in this case.

[0118] In this preferred embodiment, the present application obtains historical characteristic data of the power system network flow and inputs it into the initial linear regression model. The present invention uses a preset regularization strength, learning rate and loss function to train the model. The loss function contains a mean square error term and an L2 regularization term, wherein the mean square error term is used to measure the deviation between the model prediction value and the actual value, and the L2 regularization term prevents overfitting by limiting the size of the model parameters. When the value of the loss function no longer decreases within a preset time period or reaches a preset number of iterations, the training process stops, and an optimized linear regression model is finally obtained. This process not only improves the model's ability to fit historical data, but also enhances the model's generalization ability through regularization means, enabling it to more accurately predict future flow and detect anomalies.

[0119] S04: According to a preset time interval, the actual flow value of the current power system is obtained, and the residual between the flow prediction value and the actual flow value is calculated.

[0120] S05: Determine abnormal points of current power system network flow according to a preset interquartile range method and the residual.

[0121] As a preferred embodiment of the first embodiment, the abnormal point of the current power system network flow is determined according to the preset interquartile range method and the residual, specifically:

[0122] 1. First, calculate the data residual value of the predicted flow value at multiple consecutive time points and the actual flow value;

[0123] 2. Then calculate the 25% quantile Q1 and the 75% quantile Q3 of the residual value sequence;

[0124] 3. Calculate the interquartile range (IQR): IQR = Q3 - Q1.

[0125] 4. Calculate the lower and upper bounds of the anomaly threshold range. Data points that are smaller than the lower bound or larger than the upper bound are considered outliers:

[0126] Lower bound = Q1 - 1.5 × IQR;

[0127] Upper bound = Q3 + 1.5 × IQR;

[0128] 5. Abnormal judgment rules:

[0129] (1) Single point anomaly: If the residual value corresponding to a certain time point exceeds the threshold range, it is marked as a suspicious point.

[0130] (2) Continuous anomaly: If the threshold is triggered at multiple consecutive time points (e.g., ≥3 times within 10 minutes), it is determined to be a DDoS attack or equipment failure.

[0131] (3) Pattern anomaly: The residual value sequence shows unnatural periodic fluctuations (such as sudden increases at fixed intervals), which may be the heartbeat signal of a malicious program.

[0132] In this preferred embodiment, the present application calculates the first quantile (Q1) and the third quantile (Q3) of the residual sequence, and further solves the interquartile range (IQR=Q3-Q1). The present application can determine the abnormal threshold range based on IQR, that is, the lower limit is Q1-1.5×IQR and the upper limit is Q3+1.5×IQR. This method uses the comparison of residuals and abnormal threshold ranges to accurately identify abnormal points in the power system network flow. Specifically, a single point anomaly refers to the residual exceeding the threshold range at a certain time point, a persistent anomaly refers to the residual exceeding the threshold range at multiple consecutive time points, and a pattern anomaly refers to the occurrence of unnatural periodic fluctuations in the residual sequence. This anomaly detection method based on the interquartile range can not only effectively reduce the false alarm rate and the missed alarm rate, but also quickly respond to different types of anomalies, significantly improving the accuracy and reliability of power system network flow monitoring, and providing a strong guarantee for the stable operation of the power system.

[0133] The power system anomaly detection method of the present application also includes various modules, such as Figure 3 As shown;

[0134] It mainly includes functional modules such as data collection, preprocessing, feature engineering, model training, evaluation and deployment;

[0135] Data acquisition and preprocessing module:

[0136] This module is responsible for collecting flow-related characteristic data in the power system network by system organizational unit, such as flow data, connection log data, etc. After collection, outliers are removed, data is cleaned, and data is time-series processed.

[0137] Feature Engineering Module:

[0138] This module is responsible for extracting key features to enhance the model's predictive performance. For example, it extracts traffic packet data, traffic velocity data, protocol data, and connection data. It then calculates statistical features such as the mean, maximum, minimum, standard deviation, and variance of the traffic. Because too many traffic features are extracted, PCA is used here to reduce these features, retaining those that are more relevant to network traffic.

[0139] Ridge regression model training module:

[0140] This module receives real-time feature data after feature engineering, divides the processed data into training and test sets, constructs the initial data to train the Ridge regression model, defines model hyperparameters, and calculates all unknown model parameters. After training, the model is output to the Model Evaluation and Optimization module. It also supports reading offline data from the database module to analyze historical data and predict new target variables.

[0141] Database module:

[0142] This module is responsible for storing data processed in real time by the feature module and the prediction data output by the model deployment and monitoring modules, ensuring data security and integrity.

[0143] Model evaluation and optimization module:

[0144] After the model is trained, this module is responsible for using the mean square error to evaluate the model performance and adjust the model's hyperparameters and optimize them to improve the prediction effect.

[0145] Model evaluation and monitoring module:

[0146] This module is responsible for deploying the trained model to the production environment, predicting network traffic in real time, and monitoring its operational status. It processes network traffic data in real time using stream processing frameworks (such as Apache Kafka and Spark Streaming), inputs real-time features into the Ridge regression model, and outputs traffic predictions. These predictions are then compared with actual traffic values to identify abnormal fluctuations. If the predicted value exceeds a preset threshold, an alarm is triggered. The model's prediction performance is regularly monitored, and any degradation in performance triggers a retraining process.

[0147] This application obtains the characteristic data of the current power system network flow and uses the principal component analysis (PCA) method to reduce the dimension of the characteristic data and extract key features. This process not only reduces the feature dimension and reduces the computational complexity, but also retains the information that is most influential for flow prediction, thereby improving the efficiency and accuracy of the model. Subsequently, the key features after dimensionality reduction are input into a pre-trained linear regression model with a regularization term. The model is trained based on historical data and preset hyperparameters and loss functions, and has good generalization ability and anti-overfitting properties. The model outputs flow prediction values for multiple consecutive time points, providing a benchmark for real-time monitoring and prediction. Furthermore, by calculating the residual between the predicted value and the actual flow value and combining the interquartile range (IQR) method to determine anomalies, real-time anomaly detection of power system network flow is achieved. This method can effectively identify single-point anomalies, persistent anomalies, and pattern anomalies, improving the accuracy and response speed of anomaly detection. Overall, this application significantly improves the accuracy of power system network flow prediction and the reliability of anomaly detection by optimizing feature processing and model construction. This application effectively solves the problem that the existing technology cannot accurately and efficiently detect anomalies in the power system.

[0148] Example 2

[0149] Please refer to Figure 4 , is an abnormality detection device for an electric power system provided in an embodiment of the present application.

[0150] In this embodiment, the abnormality detection device of the power system includes an acquisition module 10 , a dimensionality reduction module 20 , a prediction module 30 , a calculation module 40 and a detection module 50 .

[0151] This application uses the Ridge linear regression model to predict the network security flow data of the power system. The security of the power system network is particularly important in the smart grid environment. Predicting network security flow can help the operation and maintenance team detect potential attacks or abnormal behaviors in advance, thereby effectively preventing risks. The Ridge linear regression model is suitable for processing complex, high-dimensional and multicollinear network flow data due to its stability and anti-overfitting ability. The overall process of this system method is as follows Figure 2 As shown:

[0152] The first step is to select the relevant features of power system network security flow prediction;

[0153] The second step is to collect relevant characteristic data of network security flows of various power systems and divide them into training sets and test sets, and then perform time series processing on the training set data;

[0154] The third step is to perform feature engineering on the training set data (e.g. standardization);

[0155] The fourth step is to use PCA (principal component analysis) method to select the main k features;

[0156] The fifth step is to initialize a set of model hyperparameter values, such as [0.01 0.1 1 10 100];

[0157] The sixth step is to train the ridge linear regression model.

[0158] After the seventh step of training the ridge linear regression model, input the test set to observe whether the data results meet expectations.

[0159] If the eighth step does not meet the requirements, adjust the hyperparameter values and retrain the model until the test results meet expectations.

[0160] In the ninth step, if the prediction results meet expectations, the model prediction results are output.

[0161] The acquisition module 10 is used to acquire characteristic data of the current power system network traffic.

[0162] The dimension reduction module 20 is used to reduce the dimension of the feature data according to a preset principal component analysis method to extract key features.

[0163] As a preferred embodiment of the second embodiment, the feature data is reduced in dimension according to a preset principal component analysis method to extract key features, specifically:

[0164] The PCA (principal components analysis) method is used for feature reduction. After selecting multiple features, in order to eliminate some feature data that has little impact on the results and reduce the computational complexity of the model algorithm, the PCA method is used here to select more important feature data.

[0165] For a normalized matrix S, first calculate the mean value for each column in S Then subtract the average value from all points in the matrix Thus we get the matrix B.

[0166] The next step is to find the covariance matrix C of B, which is calculated as follows:

[0167]

[0168] The obtained matrix C is a p×p matrix. Next, we find the eigenvalues and eigenvectors of the matrix C. Since C is a p×p matrix, we can find p matrix eigenvalues λ i and the eigenvector v i Sort by eigenvalue, and then use the Explained Variance method to calculate the contribution rate of each eigenvalue:

[0169]

[0170] Next, calculate the cumulative contribution rate:

[0171]

[0172] Select the value of M (M < p) such that the cumulative contribution rate reaches a predetermined threshold, which is usually 90% or 95%. That is, select the first k principal components so that the cumulative contribution rate of these principal components is as close as possible to 1 (100%) but not less than 90% or 95%.

[0173] Next, select the eigenvectors corresponding to the first M eigenvalues according to the selected value of M, and then arrange them into a matrix P from top to bottom. Finally, multiply the matrix P by the transpose of the matrix B to obtain the matrix X' = PB after the PCA method is processed. T :

[0174]

[0175] After completing the PCA (Principal Component Analysis) of the standardized matrix, a new initial matrix X' is obtained.

[0176] In this preferred embodiment, the present application can sort the eigenvectors according to the magnitudes of the eigenvalues by calculating the covariance matrix of the feature data and solving its eigenvalues and eigenvectors, and then select the first M eigenvectors with the cumulative variance contribution rate reaching the preset threshold as the principal components. This process not only effectively reduces the dimension of the feature data, reduces the computational complexity, but also retains the feature information that has the most influence on network traffic prediction and anomaly detection. Projecting the feature data onto these principal components to obtain the key features after dimensionality reduction further improves the training efficiency and prediction accuracy of the model, and at the same time enhances the processing ability and generalization performance of the model for high-dimensional data, providing a solid technical foundation for the efficient monitoring and anomaly detection of power system network traffic.

[0177] The prediction module 30 is used to input the key features into a preset linear regression model with a regularization term, so that the linear regression model outputs the traffic prediction values at consecutive multiple time points; wherein, the linear regression model is trained on the feature data of the historical power system network traffic according to the preset hyperparameters and the preset loss function for the initial linear regression model.

[0178] As a preferred embodiment of the second embodiment, the key features are input into a preset linear regression model containing a regularization term, so that the linear regression model outputs flow prediction values at multiple consecutive time points, wherein the linear regression model is obtained by training an initial linear regression model on characteristic data of historical power system network flow according to preset hyperparameters and a preset loss function, specifically:

[0179] First, obtain the characteristic data of historical power system network traffic;

[0180] The characteristic data includes packet rate, byte rate, peak traffic, traffic standard deviation, protocol distribution (i.e., the number of protocol packets), number of abnormal protocols (unexpected number of network protocol packets), timestamp (indicating time characteristics such as hour of the day, day of the week, month, etc.), number of failed connections, packet loss rate, retransmission rate, average packet size and maximum packet size.

[0181] After obtaining the characteristic data of the current power system network flow, the method further includes:

[0182] After feature selection, the data is time-serialized. For example, time serialization can be performed on time ranges such as one minute, ten minutes, one hour, and 24 hours. That is, data is collected within this time range, including the selected feature data and traffic result data.

[0183] When obtaining the result value of time series, you can obtain it as follows: For example, if you choose to collect data by day, you can select the data from the 1st to the 7th day as the input sample, and the traffic package data of the 8th day as the result output value; the data from the 2nd to the 8th day as the input sample, and the traffic package data of the 9th day as the result output value. According to the above method, the data collected and obtained are divided into a part of the data as the training set and a part as the test set. The initial training set data constructs the following initial matrix C and result matrix Y:

[0184]

[0185] Among them, N represents the number of samples collected, that is, the matrix columns represent the collected traffic feature data. p represents the number of features, and the c value of the matrix represents the collected feature observation value of a certain sample; Y is the traffic result corresponding to the result matrix identification sample, y i Indicates the traffic result value that can be predicted for the i-th sample. For example, you can select the number of traffic packets or a specific data type value you want to predict (such as the number of TCP packets, the number of abnormal data packets, etc.).

[0186] Data cleaning is a key step in data analysis and modeling to ensure data quality and integrity. The initial data collected may contain noise or missing values. Missing values can be filled with mean, median, mode or interpolation. The next step is to normalize the matrix C. Here, the cosine distance metric is used for normalization, that is, each matrix element c of the initial matrix c is normalized. ij calculate Thus we get the standardized matrix S:

[0187]

[0188] Then, the new initial matrix X' is obtained after PCA (principal component analysis) of the standardized matrix;

[0189] First, we need to initialize a set of values for the hyperparameter α that defines the regularization strength of the model, such as [0.01 0.1 110100]. Then we get the new initial matrix X:

[0190]

[0191] That is, add a column to the leftmost side of the matrix based on X', with all values being 1.

[0192] Next, we will use the latest initial matrix X and the result matrix Y to train the model. Ridge regression limits the size of model parameters by adding an L2 norm regularization term to the loss function. The optimization objective function of the model is as follows:

[0193]

[0194] Among them, α ≥ 0 is the defined regularization strength hyperparameter, and you can choose an initialization from the previously defined values before training the model; Represents the fitting error, which measures how well the model fits the training data; is the sum of the squares of the parameters, controlling the size of the parameters.

[0195] Since only β is an unknown parameter in the objective function L(β), the initial construction of the ridge regression model can be completed after calculating β. The calculation formula is as follows:

[0196] β=(X T X+αI) -1 X T Y

[0197] Where I represents the unit matrix. The β value can be obtained according to the above formula, and the ridge regression model can be constructed by substituting it into the objective function.

[0198] In order to optimize the model, the training set data and the cross validation (CV) method can be used to evaluate the effect of different regularization strengths α and select the optimal parameters:

[0199] (3) Divide the training set into k subsets (the k value should be less than the number of samples in the training set), select one of the subsets as the validation set each time, and the remaining k-1 subsets as the training set.

[0200] (4) For each α value, the performance on the test set is calculated after the training model is completed. The mean-square error (MSE) can be calculated as follows:

[0201]

[0202] Where n represents the number of test set samples, Represents the model output value, y i Represents the true value in the test set. Select the α value that minimizes the mean square error and substitute it into the objective function L(β) to retrain the model parameter β. Substituting it into y = βX, we can obtain the flow value predicted by the model in this case.

[0203] In this preferred embodiment, the present application obtains historical characteristic data of the power system network flow and inputs it into the initial linear regression model. The present invention uses a preset regularization strength, learning rate and loss function to train the model. The loss function contains a mean square error term and an L2 regularization term, wherein the mean square error term is used to measure the deviation between the model prediction value and the actual value, and the L2 regularization term prevents overfitting by limiting the size of the model parameters. When the value of the loss function no longer decreases within a preset time period or reaches a preset number of iterations, the training process stops, and an optimized linear regression model is finally obtained. This process not only improves the model's ability to fit historical data, but also enhances the model's generalization ability through regularization means, enabling it to more accurately predict future flow and detect anomalies.

[0204] The calculation module 40 is used to obtain the actual flow value of the current power system according to a preset time interval, and calculate the residual between the flow prediction value and the actual flow value.

[0205] The detection module 50 is used to determine abnormal points of the current power system network flow based on a preset interquartile range method and the residual.

[0206] As a preferred embodiment of the second embodiment, the abnormal point of the current power system network flow is determined according to the preset interquartile range method and the residual, specifically:

[0207] 1. First, calculate the data residual value of the predicted flow value at multiple consecutive time points and the actual flow value;

[0208] 2. Then calculate the 25% quantile Q1 and the 75% quantile Q3 of the residual value sequence;

[0209] 3. Calculate the interquartile range (IQR): IQR = Q3 - Q1.

[0210] 4. Calculate the lower and upper bounds of the anomaly threshold range. Data points that are smaller than the lower bound or larger than the upper bound are considered outliers:

[0211] Lower bound = Q1 - 1.5 × IQR;

[0212] Upper bound = Q3 + 1.5 × IQR;

[0213] 5. Abnormal judgment rules:

[0214] (1) Single point anomaly: If the residual value corresponding to a certain time point exceeds the threshold range, it is marked as a suspicious point.

[0215] (2) Continuous anomaly: If the threshold is triggered at multiple consecutive time points (e.g., ≥3 times within 10 minutes), it is determined to be a DDoS attack or equipment failure.

[0216] (3) Pattern anomaly: The residual value sequence shows unnatural periodic fluctuations (such as sudden increases at fixed intervals), which may be the heartbeat signal of a malicious program.

[0217] In this preferred embodiment, the present application calculates the first quantile (Q1) and the third quantile (Q3) of the residual sequence, and further solves the interquartile range (IQR=Q3-Q1). The present application can determine the abnormal threshold range based on IQR, that is, the lower limit is Q1-1.5×IQR and the upper limit is Q3+1.5×IQR. This method uses the comparison of residuals and abnormal threshold ranges to accurately identify abnormal points in the power system network flow. Specifically, a single point anomaly refers to the residual exceeding the threshold range at a certain time point, a persistent anomaly refers to the residual exceeding the threshold range at multiple consecutive time points, and a pattern anomaly refers to the occurrence of unnatural periodic fluctuations in the residual sequence. This anomaly detection method based on the interquartile range can not only effectively reduce the false alarm rate and the missed alarm rate, but also quickly respond to different types of anomalies, significantly improving the accuracy and reliability of power system network flow monitoring, and providing a strong guarantee for the stable operation of the power system.

[0218] This device uses five modules to divide the work and coordinate work to more accurately detect anomalies in the power system. This application obtains the characteristic data of the current power system network flow and uses the principal component analysis (PCA) method to reduce the dimension of the characteristic data and extract key features. This process not only reduces the feature dimension and reduces the computational complexity, but also retains the information that has the greatest impact on flow prediction, thereby improving the efficiency and accuracy of the model. Subsequently, the key features after dimensionality reduction are input into a pre-trained linear regression model containing a regularization term. The model is trained based on historical data and preset hyperparameters and loss functions, and has good generalization ability and anti-overfitting characteristics. The model outputs flow prediction values for multiple consecutive time points, providing a benchmark for real-time monitoring and prediction. Furthermore, by calculating the residual between the predicted value and the actual flow value and combining the interquartile range (IQR) method to determine the anomaly point, real-time anomaly detection of the power system network flow is achieved. This method can effectively identify single-point anomalies, persistent anomalies, and pattern anomalies, improving the accuracy and response speed of anomaly detection. Overall, this application significantly improves the accuracy of power system network flow prediction and the reliability of anomaly detection by optimizing feature processing and model construction. This effectively solves the problem that existing technologies cannot accurately and efficiently detect anomalies in power systems.

[0219] Example 3:

[0220] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for detecting abnormalities in a power system;

[0221] Wherein, if the power system anomaly detection method is implemented in the form of a software functional unit and used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0222] Example 4

[0223] The present application provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements any one of the power system anomaly detection methods described in Example 1.

[0224] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for detecting abnormality in a power system, characterized in that: include: Obtain characteristic data of current power system network traffic; According to a preset principal component analysis method, the feature data is reduced in dimension to extract key features; Inputting the key features into a preset linear regression model containing a regularization term so that the linear regression model outputs flow prediction values at multiple consecutive time points; wherein the linear regression model is obtained by training an initial linear regression model on characteristic data of historical power system network flow according to preset hyperparameters and a preset loss function; According to a preset time interval, the actual flow value of the current power system is obtained, and the residual between the flow prediction value and the actual flow value is calculated; According to the preset interquartile range method and the residual, the abnormal point of the current power system network flow is determined.

2. The power system anomaly detection method according to claim 1, characterized in that: According to the preset principal component analysis method, the feature data is reduced in dimension to extract key features, specifically: Calculating a covariance matrix of the feature data; According to a preset calculation formula, the eigenvalues and eigenvectors of the covariance matrix are obtained; Sorting by the size of the eigenvalues, selecting the first M eigenvectors whose cumulative variance contribution rate reaches a preset threshold as principal components; wherein M is a positive integer; The feature data is projected onto the principal component to obtain the key features after dimensionality reduction.

3. The power system anomaly detection method according to claim 1, characterized in that: The abnormal point of the current power system network flow is determined according to the preset interquartile range method and the residual, specifically: Calculate the first quantile and the second quantile of the sequence of the residuals; The interquartile range is calculated based on the first quartile and the second quartile; Determining an abnormal threshold range based on the interquartile range; Determining an abnormal point of current power system network flow based on the residual and the abnormal threshold range; The abnormal points include single point abnormalities, continuous abnormalities and pattern abnormalities.

4. The power system anomaly detection method according to claim 1, characterized in that: The linear regression model is obtained by training the initial linear regression model on the historical characteristic data of the power system network flow according to the preset hyperparameters and the preset loss function, specifically: Obtain historical characteristic data of power system network traffic; Inputting the historical feature data into an initial linear regression model so that the initial linear regression model trains the historical feature data according to a preset regularization strength, learning rate, and loss function; wherein the loss function includes a mean square error term and an L2 regularization term; When the value of the loss function no longer decreases within a preset time period or reaches a preset number of iterations, the training is stopped to obtain the linear regression model.

5. The power system abnormality detection method according to claim 4, characterized in that: The initial linear regression model is trained on the historical feature data according to a preset regularization strength, learning rate and loss function, specifically: The optimization objective function of the initial linear regression model during training is: Where, α≥0 is the preset regularization strength; Represents the fitting error, which is used to measure how well the model fits the training data; is the sum of the squares of the parameters, which is used to control the size of the parameters.

6. The power system anomaly detection method according to claim 3, characterized in that: The abnormal point of the current power system network flow is determined based on the residual and the abnormal threshold range; wherein the abnormal point includes a single point abnormality, a continuous abnormality, and a pattern abnormality, specifically: If the residual at the first time point exceeds the abnormality threshold range, determining that the first time point is a single point abnormality; If the residuals of multiple consecutive time points exceed the abnormality threshold range, the multiple consecutive time points are determined to be persistent abnormalities; If the sequence of the residuals exhibits non-natural periodic fluctuations, it is determined to be a pattern anomaly.

7. The power system anomaly detection method according to claim 1, characterized in that: The characteristic data of the current power system network traffic include packet rate, byte rate, peak traffic, traffic standard deviation, number of protocol packets, number of abnormal protocols, timestamp, number of failed connections, packet loss rate, retransmission rate, average packet size and maximum packet size.

8. An abnormality detection device for a power system, characterized in that: It includes acquisition module, dimensionality reduction module, prediction module, calculation module and detection module; The acquisition module is used to obtain characteristic data of the current power system network traffic; The dimensionality reduction module is used to reduce the dimension of the feature data according to a preset principal component analysis method to extract key features; The prediction module is used to input the key features into a preset linear regression model containing a regularization term, so that the linear regression model outputs flow prediction values at multiple consecutive time points; wherein the linear regression model is obtained by training an initial linear regression model on characteristic data of historical power system network flow according to preset hyperparameters and a preset loss function; The calculation module is used to obtain the actual flow value of the current power system according to a preset time interval and calculate the residual between the flow prediction value and the actual flow value; The detection module is used to determine the abnormal points of the current power system network flow according to the preset interquartile range method and the residual.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the abnormality detection method for the power system according to any one of claims 1 to 7.

10. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the method for detecting abnormality in the power system according to any one of claims 1 to 7 is implemented.

Images