An online incremental high-impact sample detection method for smart grids

Abstract

This invention proposes an online incremental high-influence sample detection method for smart grids, relating to the field of online incremental learning technology in machine learning. The method constructs a local hat matrix for samples arriving at the edge computing device's buffer, then calculates the influence of the samples in the buffer based on this matrix and sorts them by size. The top-ranked samples are stored in a fixed-size buffer. When a new sample arrives at the device, it is merged with the previously selected high-influence samples, and the high-influence samples are re-detected, updating the rearranged buffer. This process is repeated until the online learning ends. At this point, the high-influence samples recorded in the rearranged buffer represent the subset of samples with the highest influence among all training samples up to the current stage. This invention solves the problem of constructing a hat matrix based on all samples in edge computing environments with limited storage space, reducing the time and space overhead of the detection algorithm.

Description

Technical Field

[0001] This invention relates to the field of online incremental learning technology in machine learning, and to an incremental detection method for high-impact samples during online learning. Background Technology

[0002] In machine learning, the quality of training samples is a prerequisite for ensuring model accuracy, and high-impact samples are key to guaranteeing sample quality. Sample influence is a core metric for measuring the impact of each sample on the model during training. By identifying high-impact samples, protecting normal high-impact sample points, and removing or correcting abnormally high-impact samples, the stability of sample quality can be ensured. In smart grids, many prediction tasks, such as electricity forecasting, require influence analysis of online monitoring sample data.

[0003] Current high-impact sample detection methods are based on the Cook's distance method from classical statistical theory. This method requires constructing a hat matrix based on the entire sample data, resulting in high computational overhead. While many methods have optimized its computational performance, the dependency on the entire sample remains unchanged. However, in storage-constrained edge computing environments, online learning methods are typically used to train models. Training samples are usually overwritten by new samples after a single training run, making it difficult to construct a hat matrix based on the entire sample. Therefore, traditional high-impact sample detection methods become unsuitable in such storage-constrained online learning environments. With the development of smart grids, prediction tasks will be devolved to resource-constrained edge devices, requiring high-performance sample detection methods to achieve quality control of online monitoring data. Summary of the Invention

[0004] The technical problem this invention aims to solve is to address the challenge of constructing a hat matrix based on all samples when the storage space of edge devices in smart grids is limited. This invention proposes an online incremental high-influence sample detection method. This method constructs a local hat matrix only for samples currently arriving in the edge computing device's buffer. Based on this matrix, the influence of the samples in the current buffer is calculated and sorted by size. The top-ranked samples are stored in a fixed-size buffer (called the rehearsal buffer). When a new sample arrives at the device, the new sample is merged with the previously selected high-influence samples, and the high-influence samples are detected again, updating the rehearsal buffer. This process is repeated until the online learning ends. At this point, the high-influence samples recorded in the rehearsal buffer represent the subset of samples with the highest influence among all training samples up to the current stage.

[0005] The technical solution adopted by this invention to solve the above-mentioned technical problems is: an online incremental high-impact sample detection method for smart grids, comprising the following steps:

[0006] 1) Initialize the rearrangement buffer RB;

[0007] 2) For the current sample set reaching the edge device Constructing a hat matrix

[0008]

[0009] in, This represents the feature matrix corresponding to the samples that influence smart grid power prediction contained in time slice t. This indicates that time slice t contains the label vectors corresponding to the samples;

[0010] 3) For the current sample set Each sample (x) i ,y i Perform leave-one-out evaluation and calculate the difference in model predictions caused by each sample:

[0011]

[0012] in, To use the sample points (x) i ,y i sample set The trained model for x i The predicted value, To use without including sample points (x) i ,y i sample set The trained model for x i The predicted value, x i y i express The feature vector and label value corresponding to the i-th sample point in the dataset;

[0013] 4) Calculate the current sample set Each sample (x) i ,y i The distance C) i :

[0014]

[0015] Among them, h ii Representation matrix The corresponding x i The diagonal elements, d is the dimension of the sample features, and MSE is the model's performance in the current set. The mean square error;

[0016] 5) Sample set Each sample point in the data is arranged according to distance C. i Sort the samples and select the top r samples to store in the rearrangement buffer RB.

[0017] 6) When new samples When reaching edge computing devices, Samples from RB are merged into And process the sample set according to steps 2 to 5. Re-examine high-impact samples and overwrite the RB with the newly detected high-impact samples;

[0018] 7) Continue step 6) until the sample data stream stops or the detection algorithm ends.

[0019] This invention fully considers the transitivity of sample influence. During the online learning process, it sequentially detects each batch of high-influence sample points arriving at the edge device and passes the batch of high-influence sample points to the next batch of new sample points by rearranging the buffer, thus realizing an incremental method for detecting high-influence sample points.

[0020] The beneficial effects of this invention are:

[0021] (1) It solves the problem that it is difficult to construct a hat matrix based on all samples in edge computing environments with limited storage space, making the detection method based on Cook distance applicable to online learning environments;

[0022] (2) By rearranging the buffer records of some high-impact historical samples, the storage space of the detection algorithm is reduced;

[0023] (3) By transforming the high-cost offline calculation based on all samples into incremental calculation in small batches, the time cost of the detection algorithm is reduced. Attached Figure Description

[0024] Figure 1 This is a diagram illustrating an online learning environment.

[0025] Figure 2 This is a schematic diagram of the online incremental high-impact sample point detection method.

[0026] Figure 3 This is a flowchart of an online incremental high-impact sample point detection method. Detailed Implementation

[0027] In this embodiment, to simplify the problem description and facilitate a better understanding of the implementation process of the invention method, the detection of high-influence samples during the training process of a basic linear regression model is used as an application scenario. However, the actual application is not limited to linear regression models; it is also applicable to the detection of high-influence samples during the training process of other models such as support vector machines and deep learning linear models. Furthermore, this embodiment uses a scenario of electricity prediction in a power system to specifically illustrate the practical application of the method. In this scenario, the characteristics of the samples are the hourly average ambient temperature, hourly average ambient pressure, hourly average ambient relative humidity, and hourly exhaust vacuum. The label of the sample is the hourly net electricity. After learning from a set of training samples, the learning algorithm predicts the corresponding net electricity based on the characteristics of new samples (test samples). The goal of the method in this invention is to incrementally detect training samples that have a significant impact on net electricity prediction online.

[0028] The basic linear regression model in this embodiment is defined as follows:

[0029] y = h(X) = x1θ1 + x2θ2 + ... + x k θ k =Xθ,

[0030]

[0031] in Let represent the feature matrix corresponding to the training sample stream, N (N~∞) represent the total number of data stream samples, k represent the feature dimension of the sample, y represent the label vector corresponding to the sample, and θ represent the parameter vector of the model.

[0032] The online learning environment described in this embodiment refers to a storage-constrained edge computing environment, as specifically defined in the appendix. Figure 1 As shown in the figure, the rectangular dashed box represents the b samples that arrive at the edge device at time slice t-1. The solid rectangular box represents the b samples that arrive at the edge device in time slice t. at this time Already cover.

[0033] Appendix Figure 2The principle of the online incremental high - impact sample point detection method is shown. The rectangular block diagram in the middle of the figure represents the training samples that reach the edge device one after another. The dashed - framed part represents the samples that have been trained, and the solid - framed part represents the samples being trained. The rectangular block diagram above the figure represents the state change of the rehearsal buffer RB (Rehearsal buffer). The dashed - framed part represents the covered state, and the solid - framed part represents the current state. The circled numbers represent the steps of the high - impact sample detection method. The rounded - corner square box, diamond box, and solid arrows below the figure together show the online learning process. That is, whenever a sample reaches the edge device, the model is trained with these samples and the model parameters are updated. After each parameter update, it is necessary to determine whether the model converges. When the model converges, the optimal model parameters are obtained, and at this time, the high - impact sample detection method ends.

[0034] Assume that the high - impact sample selection method starts to execute from the first time slice. First, initialize the rehearsal buffer RB and the model. At the same time, use to train the model to obtain the model parameter θ1. In the second to fifth steps, calculate the influence of each sample in and select the largest r samples and store them in the rehearsal buffer RB. In the sixth step, when the time slice reaches the edge device, merge and the samples in the rehearsal buffer RB to re - detect the high - impact samples and overwrite RB with the detection results. At this time, synchronously use to train the model and update the parameters to θ2. In the seventh step, loop through the operations in the sixth step until the model reaches the convergence state. Assume that at time slice t, the model converges to obtain the optimal parameter θ * , at this time, the high - impact samples stored in the rehearsal buffer RB are approximately equal to the high - impact sample set of the cumulative sample set up to time slice t.

[0035] As shown in the appendix Figure 2 , the detailed implementation steps of the technical solution of the present invention are as follows:

[0036] S1. Initialize the rehearsal buffer RB

[0037] The rehearsal buffer is a dedicated storage area for storing high - impact sample points. Compared with the buffer for storing samples of the current time slice, the size r of the rehearsal buffer usually satisfies r < b << N. Based on this, the rehearsal buffer can reduce the storage space of historical samples.

[0038] S2. Construct a hat matrix for the sample set currently arriving at the edge computing device Through the hat matrix, the leverage of each sample can be calculated. Here, the leverage refers to the value h of the diagonal element in the hat matrix corresponding to each sampleii :

[0039]

[0040] The derivation of formula (5) is as follows: Assume that for Perform singular value decomposition (SVD), that is Substituting into formula (5) Then we have:

[0041]

[0042] Among them, the column vectors of matrix V are The eigenvectors of the covariance matrix are assumed to be e1, e2, ..., e d , then V=[e1…e d Substituting into formula (6), we get:

[0043]

[0044]

[0045] In the above formulas (5)-(7), σ1,σ2,…,σ d For matrix The corresponding singular values, e1, e2, ..., e d The principal component vector corresponding to each singular value. Represents the eigenvector x i In the principal component vector e k Projections onto the surface. e1, e2, ..., e d This indicates the matrix The main linear transformations represented by singular values ​​σ1, σ2, ..., σ d The singular values ​​represent the importance of each principal component vector (the more singular values, the more important the principal component vector). Therefore, the projection... This represents the eigenvector x. i Contribution in the direction of the principal component vector Represents the eigenvector x i The sum of the contributions in all principal component vector directions is the leverage force.

[0046] Solving for θ using the least squares method in formula (4) yields θ = (X T X) -1 X T If y, then:

[0047]

[0048] As can be seen from formula (8), the leverage force represents the contribution of each sample to the entire regression model.

[0049] S3. For the current sample set Each sample (x) i ,y i Perform leave-one-out evaluation and calculate the difference in model predictions caused by each sample:

[0050]

[0051] Where θ represents the inclusion of sample (x) i ,y i training sample set The model parameters θ obtained from training the model (-i) This indicates that the sample (x) is not included. i ,y i training sample set Model parameters obtained from training the model. This means that due to sample (x) i ,y i The difference arises from the effect of ).

[0052] S4. Calculate the current sample set Each sample (x) i ,y i The influence of ) is the distance from C. i :

[0053]

[0054] As shown in formula (10), influence represents the difference caused by a sample to the model multiplied by the sample's leverage. In other words, the influence of a sample is a measure of the magnitude of the effect produced by the combined effect of difference and leverage. Based on this definition, a sample that contributes a lot to the model (high leverage sample) is not necessarily a high-influence sample. This is because a high contribution only indicates that the sample contributes significantly to the current regression model, but it does not determine whether the current model is still consistent with the previous model (when it was not trained with this sample). If the contribution of the sample causes inconsistency in the model, then the sample is considered to have high influence. If the contribution of the sample does not cause the model to change, then the sample is not a high-influence sample. In this case, it can be understood that the sample contributes significantly to maintaining the original model. Therefore, for a normal and legitimate model containing noise or outliers, a high-influence sample means that the sample is most likely a noise or outlier sample. For an ideal sample set that does not contain noise or outliers, each sample should produce a small difference in the model, and all samples should produce legitimate model parameters. In this case, a high-influence sample means a normal high-influence sample that contributes the most to the legitimate model.

[0055] In a machine learning (online learning) task, most samples must conform to the assumption of independent and identically distributed (i.i.d.) distributions to ensure that the model converges to a useful state. This is a prerequisite for machine learning to function properly. Abnormally high-impact samples are relatively few in number compared to normally high-impact samples. Therefore, high influence is a relative evaluation. The transitivity of high influence is defined as follows: a high-impact sample in the current time slice is one of the high-impact samples among all samples up to the current time slice.

[0056] S5. Transfer the sample set Each sample point in the data is arranged according to distance C. i Sort the samples and select the top r samples to store in the rearrangement buffer RB.

[0057] Here, the number of high-impact samples, r, can be determined based on specific application requirements, such as the proportion of outlier detection in the application; r can also be determined empirically, for example... This indicates that the sample is an exceptionally high-impact sample. Let r be the total number of samples contained in the time slice. Then the number of r can be determined by... The optimal value of r can be determined by various methods. Alternatively, r can be determined through experimental observation, which involves iterating through different values ​​of r, selecting r high-impact samples to remove, and comparing the changes in the model's prediction results to determine the optimal value of r.

[0058] S6. When new samples When reaching edge computing devices, Samples from RB are merged into And process the sample set according to steps 2 to 5. Re-examine high-impact samples and overwrite the RB with the newly detected high-impact samples;

[0059] The size of the RB can be adjusted accordingly based on the storage capacity of the edge nodes. The storage strategy for the RB can also be adjusted, for example, during each detection... When a new high-impact sample is obtained, it is compared with the samples in the Reference Block (RB). Samples that do not match are stored in the RB. This strategy does not simply overwrite the samples already stored in the RB, but continuously expands the storage of high-impact samples. This leads to an increase in the storage of the RB, but the detection of high-impact samples becomes more accurate.

[0060] S7. Continue step S6 until the sample data stream transmission stops or the detection algorithm ends.

[0061] Generally, data transmission in online learning environments does not stop; the time it takes for an algorithm to finish is usually measured by referring to the appendix. Figure 2The convergence time in the model is when online learning reaches the convergence state and obtains the optimal model parameters θ. * At that moment, the result of the algorithm is to obtain the model parameters θ. * The r samples with the greatest influence.

Claims

1. An online incremental high-impact sample detection method for smart grids, comprising the following steps: 1) Initialize the rearrangement buffer RB; 2) For the current sample set reaching the edge device Constructing a hat matrix in, This represents the feature matrix corresponding to the samples that influence smart grid power prediction contained in time slice t. This indicates that time slice t contains the label vectors corresponding to the samples; 3) For the current sample set Each sample (x) i ,y i Perform leave-one-out evaluation and calculate the difference in model predictions caused by each sample: in, To use the sample points (x) i ,y i sample set The trained model for x i The predicted value of To use without including sample points (x) i ,y i sample set The trained model for x i The predicted value, x i y i express The feature vector and label value corresponding to the i-th sample point in the dataset; 4) Calculate the current sample set Each sample (x) i ,y i The distance C) i : Among them, h ii Representation matrix The corresponding x i The diagonal elements, d is the dimension of the sample features, and MSE is the model's performance in the current set. The mean square error; 5) Sample set Each sample point in the data is arranged according to distance C. i Sort the samples and select the top r samples to store in the rearrangement buffer RB. 6) When new samples When reaching edge computing devices, Samples from RB are merged into And process the sample set according to steps 2 to 5. Re-examine high-impact samples and overwrite the RB with the newly detected high-impact samples; 7) Continue step 6) until the sample data stream stops or the detection algorithm ends.

Images