Electric energy meter operation data variable screening method, electronic device and storage medium

Abstract

The application discloses an electric energy meter operation data variable screening method, an electronic device and a storage medium, constructs basic errors BE at each moment and a variable set corresponding to the basic errors BE; calculates the maximum information coefficient between the basic errors at each moment in the basic error vector of the intelligent electric energy meter and each sample in the sample data set, deletes the sample with the maximum information coefficient less than a set threshold, and the remaining samples constitute a variable set; detects an abnormal value, uses an improved weighted Euclidean distance and SC, CH quantitative analysis KNN abnormal value detection results, and uses the detected weight to correct the basic error abnormal value; corrects the basic error, performs second-step variable screening on the corrected basic error vector and the variable set, and finally determines the basic error vector and the variable set. The application can not only effectively and scientifically detect abnormal data, ensure the integrity of the data, but also quickly and reasonably screen irrelevant variables and joint variables, and avoid the collinearity problem.

Description

Technical Field

[0001] This invention relates to the technology for processing variables of electricity meter operation data, and in particular to a method for filtering variables of electricity meter operation data, an electronic device, and a storage medium. Background Technology

[0002] During operation, the measurement accuracy and reliability of electricity meters are mainly affected by environmental factors such as temperature and humidity. In addition, coastal areas are severely affected by salt spray and wind speed; in high-altitude areas, they are affected by altitude and sunlight. Electricity meters are also affected to varying degrees when measuring different amounts of electricity. Under the combined influence of numerous factors, the measurement accuracy and operational reliability of electricity meters gradually decrease. However, the impact of some factors far outweighs that of others, and pressure in the same region remains a relatively stable variable. The variables exhibit strong correlations, and introducing multiple strongly correlated variables into the model often leads to collinearity. As the terminal nerve of the smart grid, studying the metering accuracy and reliability of electricity meters under multiple influencing factors is crucial. Electricity meter operating data also contains outliers, which severely affect the convergence of subsequent research models and the reliability of calculation results. Some scholars and experts have conducted research on variable screening and anomaly detection, as follows:

[0003] Patent application CN115346682A discloses a variable screening method and system based on breast cancer data, as well as a readable storage medium, belonging to the field of medical data processing technology. The variable screening method for breast cancer data includes: acquiring breast cancer data; preprocessing the breast cancer data, including data normalization and outlier handling; performing correlation analysis on the preprocessed breast cancer data using the maximum information coefficient method to obtain a first screening result; and performing variable screening on the first screening result to obtain a second screening result. The variable screening method includes one or a combination of the following: Lasso algorithm, random forest, SIS variable screening, and DC-SIS variable screening. This method can solve the problems of marginal uncorrelation and joint correlation between variables, reduce the error rate, and make the data more biologically meaningful.

[0004] Patent application CN109459409B discloses a KNN-based near-infrared anomalous spectrum identification method to address the problem that the presence of anomalous spectral data in near-infrared spectral analysis severely affects the accuracy and reliability of spectral analysis models. The method includes the following steps: selecting a similarity metric, selecting a hyperparameter k, calculating the inter-spectral distance metric, finding the sample with the shortest distance to k, calculating the sample anomalous metric, sorting samples by anomalous metric, and identifying and removing samples with high anomalous metric values. This method is used for the identification and removal of anomalous spectra in the construction of near-infrared spectral analysis models.

[0005] However, the above scheme does not address the impact of each variable on the detection of outliers in the monitored object—the electricity meter's operating data. Furthermore, the screening steps for unrelated and jointly related variables are too complex. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method, electronic device and storage medium for filtering variables of electricity meter operation data, which are necessary to detect abnormal data and ensure data integrity; and to quickly and reasonably filter unrelated variables and joint variables to avoid multicollinearity problems.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for filtering variables of electricity meter operation data, comprising the following steps:

[0008] S1. Construct the basic error BE at each time point and the set of variables θ corresponding to the basic error BE: θ = [T, R, P, V, W, YW, IP, t], where T, R, P, V, W, YW, IP, and t represent temperature, humidity, pressure, light intensity, wind speed, salt spray, electricity, and time, respectively.

[0009] S2. Calculate the maximum information coefficient between the basic error at each moment in the basic error vector of the smart meter and each sample in the sample data set, delete the samples whose maximum information coefficient is less than a set threshold, and the remaining samples constitute the variable set θ1.

[0010] S3. Correct the basic error BE using the following formula: in, This is the corrected basic error. w s D represents the corrected weight value for the s-th data point. g,ω (s) represents the weighted Euclidean distance from the s-th data point to the g-th data point. weight ω i It is the maximum information coefficient between the basic error and the i-th variable in the variable set θ1, x s,i Let s represent the i-th variable x in the variable set θ1. g,i Let y represent the i-th variable in the variable set θ1 for data point g. s Let y be the basic error of data point s. g The basic error of data point g; n is the total number of variables in the variable set θ1;

[0011] The corrected basic error vector and the variable set θ1 are used as the selection variables.

[0012] This invention considers the impact of various variables on outlier detection of electricity meter operating data (basic error), and uses weighted Euclidean distance to correct the basic error to avoid the influence of outliers. This improves the accuracy of variable screening for electricity meter operating data, and can quickly and reasonably screen out unrelated and joint variables, avoiding multicollinearity problems.

[0013] In step S2, the formula for calculating the maximum information coefficient is: Where MIC(x, y) is the maximum information coefficient; |X|, |Y| are the number of grid segments divided in the X and Y directions, respectively; B is set to the total number of grids, i.e., |X||Y| raised to the power of 0.6; x and y are both random variables, y corresponds to the basic error in the basic error vector, x corresponds to the sample in the sample data set θ, and I(X, Y) is the mutual information.

[0014] This invention utilizes the maximum information coefficient to perform preliminary screening of outliers in sample data, thereby accelerating the screening of operational data variables.

[0015] Furthermore, in step S3 of the present invention, the specific implementation process of determining whether the basic error of the data points is normal includes:

[0016] 1) Define the intervals for the nearest neighbor value k and the abnormality ratio m; initialize k and m; where k is a positive integer, 0 < m ≤ 1;

[0017] 2) Calculate the weighted Euclidean distance from the current data point to every other data point;

[0018] 3) Sort all distance values, select the k nearest neighbors with the smallest distance, and then select the maximum distance value from the k nearest neighbors;

[0019] 4) For each of the remaining data points, repeat steps 2) and 3); after traversing all data points, obtain M distance values; M is the number of data points;

[0020] 5) Among the M distance values, the points whose distance is greater than the threshold M×m are designated as outliers, and the remaining points are designated as normal points; assign the first label value to the outliers and the second label value to the normal points;

[0021] 6) Calculate SC and CH: Where, a is the average of all distance values ​​in the same category as a among the M distance values; b is the average of all distance values ​​in the same category as b among the M distance values, a and b are the categories corresponding to the first label value or the second label value; n' is the total number of samples in the basic error vector and variable set θ1, e is the number of categories, and tr(B e ) is the covariance matrix between categorical data, tr(W) e ) is the covariance matrix of the data within each category, and tr is the trace of the matrix; the category is either abnormal or normal.

[0022] 7) Increase the value of k by the set step size, increase the value of m by the set step size, and return to step 2) until the upper limit of the neighboring value range and the abnormal ratio range is reached. The neighboring value and abnormal ratio corresponding to the optimal value of SC and CH are taken as the final neighboring value and abnormal ratio.

[0023] 8) For each data point corresponding to the basic error vector, repeat steps 3) to 5, using the final nearest neighbor and outlier ratio as inputs.

[0024] The presence of outliers can negatively impact data quality and unnecessarily interfere with the scientific and effective analysis of data. To avoid misidentification of normal data due to stress differences, various stresses must be considered when detecting outliers. This invention employs the k-nearest neighbor (KNN) method, extending the distance between common points to a high-dimensional distance across multiple variables. It weights variables in each dimension, highlighting the high-dimensional distances calculated for key influencing variables in anomaly detection. Combined with the indices SC and CH, accurate k and m values ​​are determined. These determined k and m values ​​are then used to identify outlier data points in the basic error vector, facilitating the correction of the basic error. This invention can quickly detect outlier data points and rapidly and reasonably filter out irrelevant and joint variables.

[0025] To eliminate the problem of multicollinearity of the fundamental error BE, step S3 is followed by:

[0026] S4. Further filter variables from the corrected basic error vector and variable set θ1. The filtered basic error vector and variable set are the final selected variables.

[0027] Furthermore, in this invention, the specific process of obtaining the filtered variable set θ2 includes: performing Spearman correlation analysis on each basic error in the corrected basic error vector and each variable in the variable set θ1 to obtain g Spearman correlation coefficients; where g is the number of variables in the variable set θ1.

[0028] For all Spearman correlation coefficients greater than the threshold, retain the variable corresponding to the maximum Spearman correlation coefficient and delete the variables corresponding to the remaining Spearman correlation coefficients.

[0029] In order to ensure data integrity, the threshold can be set to 0.8.

[0030] As part of the same inventive concept, the present invention also provides an electronic device comprising:

[0031] One or more processors;

[0032] A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the steps of the method described above.

[0033] As part of the same inventive concept, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0034] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention can not only effectively and scientifically detect abnormal data and ensure the integrity of data, but also quickly and reasonably screen unrelated variables and joint variables to avoid multicollinearity problems. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating the processing of energy meter operation data variables according to an embodiment of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Basic error is a key reliability indicator for electricity meters. The national metrological verification procedure for electronic AC energy meters, JJG 596-2012, stipulates that the basic error (BE) of smart meters is expressed as a relative error, and its expression is:

[0038]

[0039] In the formula, P ref Indicates the standard value; P test BE represents the measured value; BE represents the basic error. In actual operation, the basic error of an electricity meter is affected by various factors, and its value reflects the measurement accuracy of the meter. Therefore, the first step is to screen out unrelated variables affecting the electricity meter measurement. Measurement variables used to study the operational reliability of the electricity meter include temperature T, humidity R, pressure P, illumination V, wind speed W, salt spray YW, power IP, and time t.

[0040] Because the acquired data have different dimensions and inconsistent physical meanings, it is necessary to perform normalization preprocessing on the data.

[0041] The data normalization method is calculated as follows:

[0042]

[0043] In the formula, x' is the normalized variable; x min x max These are the minimum and maximum values ​​of variable x, respectively.

[0044] The Maximum Information Coefficient (MIC) is commonly used to analyze linear and nonlinear relationships between variables. It possesses universality, fairness, and symmetry, and offers higher accuracy compared to mutual information. Furthermore, MIC is a variable analysis method based on mutual information. The MIC expression is:

[0045]

[0046] In the formula, MIC(x, y) is the maximum information coefficient; |X|, |Y| are the number of grid segments in the X and Y directions, respectively (see: [link to grid division]). https: / / blog.csdn.net / FontThrone / article / details / 85227239 B is set to the total number of grids, i.e., |X||Y| raised to the power of 0.6. x and y are both random variables, y corresponds to the basic error in the basic error vector, and x corresponds to a sample in the sample data set θ. I(X,Y) is the mutual information, approximately equal to I(x,y). The expression is:

[0047]

[0048] In the formula, p(x,y) is the joint distribution of x and y, and the marginal distributions are p(x) and p(y), respectively. Therefore, I(X,Y) is approximately the relative entropy of the joint distribution and the marginal distribution of the two variables.

[0049] The MIC value ranges from [0,1]. A larger MIC indicates a stronger correlation between the two variables. A MIC of 0 indicates that the two variables are independent. The MIC threshold for variable selection can be selected empirically. In this embodiment, the input quantities are the basic error (BE) at each time step (each time step corresponds to a data point) and the set of variables corresponding to the basic error, θ = [T,R,P,V,W,YW,IP,t]. Therefore, the sample data in this embodiment is the set of BE and θ, i.e., [BE T,R,P,V,W,YW,IP,t]. The first step of variable selection involves using the MIC values ​​between the variables in the BE and θ variable sets. When the MIC value between BE and a variable is less than 0.2, it is considered a weakly correlated variable and is removed. After the first step of variable selection, the remaining variable set is θ1.

[0050] The presence of outliers can slow down model operation, lead to poor convergence, or even cause the model to fail. Therefore, outlier detection and correction are necessary. To avoid misidentification of normal data due to stress differences, various stresses need to be considered when detecting outliers. k-Nearest Neighbors (KNN) is a distance-based algorithm that extends the distance between common points to high-dimensional distances of multiple variables, weighting the variables in each dimension to highlight the high-dimensional distances calculated for the main influencing variables in outlier detection.

[0051] The distance is calculated using the Euclidean spatial distance formula, as shown below:

[0052]

[0053] In the formula, D g (s) represents the Euclidean distance from data point s (in this example, the data is time series data, so data point s is the data at time s) to the g-th data point; x s,i This indicates that data point s is the i-th variable in θ1, x g,i This represents the i-th variable in θ1 for data point g, with its value range being the variable set θ1, and n being the number of variables in θ1; y s Let y be the BE of data point s. g Let be the BE of data point g. The weights of each stress value and the fundamental error are equal and all are 1. However, the influence of each variable on BE is different. To comprehensively consider the influence of various variables on BE, a nonlinear weighting factor is introduced.

[0054]

[0055] In the formula, the weight ω i It is the MIC value between BE and the i-th variable, the variable range is within the variable set θ1, and n is the total number of variables in the variable set θ1; D g,ω (s) represents the weighted Euclidean spatial distance, which integrates the influence of each variable on the observed values ​​of the data points.

[0056] The nearest neighbor (k) and outlier ratio (m) in KNN are key parameters affecting detection accuracy. Outlier detection is similar to clustering, and is unsupervised. The silhouette coefficient (SC) and Calinski-Harabaz (CH) are methods for evaluating clustering quality, often used for samples without true labels. A grid search is used to set the intervals for parameters k and m, the number of search iterations N, and iterates through parameters k and m, evaluating the results of each iteration.

[0057] The SC (Solidity and Separation) index combines density and separation, with a value ranging from -1 to 1. This indicates that a value close to 1 is reasonable for sample clustering. CH (Dispersion and Separation) measures cluster density by calculating the sum of squared distances from each point in a cluster to the cluster center, and measures dataset separation by calculating the sum of squared distances from each centroid to the dataset center. The CH index is derived from the ratio of separation to density. Generally, a higher CH value indicates better outlier detection performance.

[0058] In each traversal search with k and m set, the weighted Euclidean distance D of the corresponding data is calculated. g,ω (s) are used to classify the data points of the sample into abnormal and normal data, and corresponding labels are assigned to them. However, the effectiveness of the classification depends on the combined values ​​of SC and CH to determine the optimal values ​​of parameters k and m.

[0059] The SC expression is:

[0060]

[0061] In the formula, 'a' represents the average distance to other data in the same category as 'a'; 'b' represents the average distance to the nearest sample from a different category as 'b'. In this case, class 'a' and class 'b' can be represented as outlier categories or normal value categories after setting k and m once.

[0062] The CH expression is:

[0063]

[0064] In the formula, n' is the total number of samples in the basic error vector and its corresponding variable set θ1; e is the number of categories, which is used for outlier detection in this embodiment, so there are only 2 categories (normal and abnormal); tr(B e ) is the covariance matrix between categories; tr(W e ) is the covariance matrix of the data within each category; tr is the trace of the matrix.

[0065] By combining the optimal values ​​of SC and CH, the nearest neighbor (k) and outlier ratio (m) of KNN are determined. Assuming the total number of data points is M, the basic process of KNN in this embodiment is as follows:

[0066] 1. Using grid search, set the intervals for nearest neighbor (k) and anomaly ratio (m), and set increment values ​​for each;

[0067] 2. Calculate the distance from the current data point to each of the other data points using formula (6);

[0068] 3. Sort each distance, then select the k nearest neighbors with the smallest distance, and use the largest distance value among these k nearest neighbors as the scoring item. The larger the distance, the greater the possibility that the current data point is abnormal.

[0069] 4. For each of the remaining data points, repeat steps 2 and 3 to finally select M distance values;

[0070] 5. Based on the set anomaly ratio (m), the number of outliers in the data is M×m. Let the maximum weighted Euclidean distance between the M×m data point and its k nearest neighbors be the threshold. Distance values ​​greater than the threshold among the M distance values ​​are set as outliers and labeled 1, while those less than the threshold are set as normal values ​​and labeled 0.

[0071] 6. Use the data points and their corresponding labels as inputs for SC and CH, and output the corresponding score values. That is, after each setting of the nearest neighbor (k) and the anomaly ratio (m), there are corresponding SC and CH scores.

[0072] 7. After traversing k and m, the grid search ends. Combining the scores of SC and CH, the optimal values ​​are selected, and the corresponding k and m are finally determined to identify the outliers and their corresponding distances.

[0073] 8. Using the k and m determined in step 7 as input, for each data point in the basic error vector, perform the operations in steps 2 to 5 to determine all outliers in the basic error vector.

[0074] In this embodiment of the invention, when M×m is a decimal, it is rounded to the nearest integer by default. For example, if M×m = 2.5, it is rounded to 3; if M×m = 4.4, it is rounded to 4.

[0075] To ensure the integrity of the data, and based on the outlier detection results described above, a weighted correction is applied to the outlier detection results using the weights assigned by Euclidean distance during the outlier detection process. The weight distribution for each data point is as follows:

[0076]

[0077] In the formula, w s D represents the adjusted weight value of the s-th sample data; k,w (s) represents the outlier weights assigned based on Euclidean distance. If the detection result is normal, the weight is 1; otherwise, the weight is exp(-D). g,ω (s)). By taking the exponential form of the weight values, the positive value criterion of the weights is guaranteed, while highlighting the outlier weights assigned based on Euclidean distance.

[0078] The weighted and corrected BE data is y * s The calculation formula is:

[0079]

[0080] For the corrected BE data, further variable screening is required. This screening starts from variable set θ1, resulting in variable set θ2, which represents the final selected variables. The final selected variables must exclude the issue of multicollinearity in the BE. The method used is the Spearman correlation coefficient. The Spearman correlation coefficient is suitable for testing linear and simple nonlinear distributions. Compared to the Pearson correlation coefficient, the Spearman correlation coefficient has fewer limitations and is not affected by outliers. If the corresponding values ​​of two variables have the same or similar order within each group (or one variable is a strictly monotonic function of the other), then they are significantly correlated. It is mainly used for correlation analysis of multicollinear variables.

[0081] In this embodiment of the invention, the specific screening method is as follows:

[0082] Spearman correlation analysis is performed on each basic error in the corrected basic error vector and each variable in the variable set θ1 to obtain g Spearman correlation coefficients; where g is the number of variables in the variable set θ1.

[0083] For all Spearman correlation coefficients greater than the threshold, retain the variable corresponding to the maximum Spearman correlation coefficient and delete the variables corresponding to the remaining Spearman correlation coefficients.

[0084] The expression for Spearman's correlation coefficient is:

[0085]

[0086] In the formula, n * d is the number of samples in the basic error vector or any of its corresponding variable vectors; i Let ρ be the difference in rank between the i-th corrected basic error vector and a data point in the variable set θ1. The absolute values ​​of their correlation strengths are as follows: 0.8-1.0 extremely strong correlation; 0.6-0.8 strong correlation; 0.4-0.6 moderate correlation; 0.2-0.4 weak correlation; 0.0-0.2 extremely weak correlation or no correlation. When ρ > 0.8, the variables are extremely strongly correlated, and one or more variables need to be removed. When the ρ of the remaining variables is < 0.8, no variable removal is necessary.

[0087] For example, the corrected basic error (BE) variable and the vectors of each variable in the variable set are represented in Table 1.

[0088] BE T R P V W t IP YW 0.0623 41 16 994.5 0.8 8 259 2.5 0.8 -0.1868 25.8 26.3 1008.4 4.6 66 287 2.5 4.6 0.2393 28.6 15.3 1007.4 1.3 59 308 2.5 1.3 -0.0155 19.8 14.8 1018.5 1.7 44 329 2.5 1.7 -0.0183 15.3 41.7 1006.4 0.5 0 336 2.5 0.5 -0.0425 5.5 44.6 1011.5 0.8 0 343 2.5 0.8 -0.0675 -0.3 27.7 1032.6 1.4 14 371 2.5 1.4 -0.073 -2.1 24.8 1026.8 1 11 378 2.5 1 0.1325 -1.1 27.9 1020.5 0.3 11 385 2.5 0.3

[0089] When calculating the Spearman correlation coefficient, the element values ​​in each vector (each column in Table 1) are first sorted in descending order, and the initial sorting values ​​are recorded. Taking the calculation of the Spearman correlation coefficient between BE and T (temperature) as an example, the initial sorting value of the maximum value (0.2393) in the basic error vector is 3 (the third element of BE), and the initial sorting value of the maximum value (41) in the temperature vector T is 1. Then d1 = 3 - 1 = 2, and so on, all d values ​​can be calculated. i The number of samples (n) * =9) Substituting into the above calculation formula, the Spearman correlation coefficient between the basic error vector and the temperature vector can be obtained.

[0090] During the screening process, if three out of the eight variables have Spearman correlation coefficients greater than 0.8 (for example, temperature, humidity, and pressure all have Spearman correlation coefficients greater than 0.8), and assuming that BE has the largest Spearman correlation coefficient with temperature, then the temperature vector is retained, while the humidity and pressure vectors are deleted.

[0091] Finally, the corrected BE data and the filtered variable set are output to make the data distribution more scientific and reliable, which facilitates subsequent research and analysis on the operational reliability of electricity meters.

[0092] Example 2

[0093] Embodiment 2 of the present invention provides an electronic device corresponding to Embodiment 1 above. The electronic device can be a processing device for a client, such as a mobile phone, a laptop, a tablet computer, a desktop computer, etc., to execute the method of the above embodiments.

[0094] The electronic device of this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method of Embodiment 1 described above.

[0095] In some implementations, the memory may be high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.

[0096] In other implementations, the processor can be any type of general-purpose processor, such as a central processing unit (CPU) or a digital signal processor (DSP), and there is no limitation here.

[0097] Example 3

[0098] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to Embodiment 1 above, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, they implement the steps of the method of Embodiment 1 above.

[0099] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0100] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0101] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0102] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0103] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0104] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for filtering variables in electricity meter operation data, characterized in that, Includes the following steps: S1. Construct the basic error BE at each time point and the set of variables θ corresponding to the basic error BE: θ=[T,R,P,V,W,YW,IP,t], where T,R,P,V,W,YW,IP,t represent temperature, humidity, pressure, light intensity, wind speed, salt spray, electricity, and time, respectively. S2. Calculate the maximum information coefficient between the basic error at each moment in the basic error vector of the smart meter and each sample in the sample data set. Delete samples whose maximum information coefficient is less than a set threshold. The remaining samples constitute the variable set θ1. S3. Correct the basic error BE using the following formula: ;in, This is the corrected basic error. w s This represents the corrected weight value for the s-th data point. Let g be the weighted Euclidean distance from the s-th data point to the g-th data point. Weight It is the maximum information coefficient between the basic error and the i-th variable in the variable set θ1, x s,i Let s represent the i-th variable x in the variable set θ1. g,i Let y represent the i-th variable in the variable set θ1 for data point g. s Let y be the basic error of data point s. g The basic error of data point g; n is the total number of variables in the variable set θ1; The corrected basic error vector and the variable set θ1 are used as the selection variables; S4. Further filter variables from the corrected basic error vector and variable set θ1. The filtered basic error vector and variable set are the final selected variables. In step S4, the specific process of obtaining the filtered variable set θ2 includes: Spearman correlation analysis is performed on each basic error in the corrected basic error vector and each variable in the variable set θ1 to obtain g Spearman correlation coefficients; where g is the number of variables in the variable set θ1. For all Spearman correlation coefficients greater than the threshold, retain the variable corresponding to the maximum Spearman correlation coefficient and delete the variables corresponding to the remaining Spearman correlation coefficients.

2. The method for filtering variables of electricity meter operation data according to claim 1, characterized in that, In step S2, the formula for calculating the maximum information coefficient is: Where MIC(x, y) is the maximum information coefficient; Let X and Y be the number of grid segments in the X and Y directions, respectively; B is set to the total number of grids, i.e., |X||Y| raised to the power of 0.6; x and y are random variables, y corresponds to the basic error in the basic error vector, x corresponds to the sample in the sample data set θ, and I(X,Y) is the mutual information.

3. The method for filtering variables of electricity meter operation data according to claim 1, characterized in that, The specific implementation process for determining whether the basic error of a data point is normal includes: 1) Define the ranges for the nearest neighbor value K and the outlier ratio R; initialize k and m; where k is a positive integer, 0 < m ≤ 1; 2) Calculate the weighted Euclidean distance from the current data point to every other data point; 3) Sort all distance values, select the n nearest neighbors with the smallest distance, and then select the largest distance value from the n nearest neighbors; 4) For each of the remaining data points, repeat steps 2) and 3); after traversing all data points, obtain M distance values; M is the number of data points; 5) Among the M distance values, points whose distance is greater than the threshold are designated as outliers, and the remaining points are designated as normal points; assign the first label value to the outliers and the second label value to the normal points; the threshold is set as the maximum weighted Euclidean distance between the M×m-th data point and its k nearest neighbors. 6) Calculate SC and CH: , , where a is the average of all distance values ​​of the same category as a among the M distance values; b is the average of all distance values ​​of the same category as b among the M distance values, and a and b are the categories corresponding to the first label value or the categories corresponding to the second label value; Let θ be the basic error vector and the total number of samples in the variable set θ1, where e is the number of classes, and tr(B) is the total number of samples in the variable set θ1. e ) is the covariance matrix between categorical data, tr(W) e ) is the covariance matrix of the data within each category, and tr is the trace of the matrix; the category is either abnormal or normal. 7) Increase the value of k by the set step size, increase the value of m by the set step size, and return to step 2) until the upper limit of the neighboring value range and the abnormal ratio range is reached. The neighboring value and abnormal ratio corresponding to the optimal values ​​of SC and CH are taken as the final neighboring value and abnormal ratio. 8) For each data point corresponding to the basic error vector, repeat steps 3) to 5, using the final nearest neighbor and outlier ratio as inputs.

4. The method for filtering variables of electricity meter operation data according to claim 1, characterized in that, The threshold is set to 0.

8.

5. An electronic device, characterized in that, include: One or more processors; A memory having stored one or more programs thereon, which, when executed by the one or more processors, cause the one or more processors to perform the steps of the method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 4.

Images