A Method for Solving Metering Error Based on Reinforcement Learning

Abstract

The present invention discloses a method for solving the error of an electric meter based on reinforcement learning, which includes Step 1: constructing matrix data of an energy conservation equation and preprocessing the power consumption data of the electric energy meter; Step 2: based on the law of energy conservation, initializing and solving the error rate value of the electric energy meter; Step 3: using a reinforcement learning model and based on the reinforcement learning qlearning algorithm, solving and iterating the state, action, and optimization objective in the data; Step 4: setting an iteration stop condition and comparing and outputting the final result of the model. The present invention first obtains the power consumption data of the electric energy meter, and then based on the law of energy conservation, constructs a power matrix and an error rate value for the data; uses the reinforcement learning qlearning error rate iterative solution algorithm to calculate the error of the electric meter, and through this method, the error of the smart electric meter can be accurately estimated.

Description

Technical Field

[0001] The present invention relates to the technical field of smart meters, and particularly to a method for solving meter errors based on reinforcement learning. Background Art

[0002] With the development of China's power grid, there are currently a large number of smart meter metering devices in the power grid, and their metering reliability affects thousands of families. At present, the management of metering device equipment is mainly based on periodic inspections and on-site operation and maintenance. This method has a large workload, high labor costs, and low efficiency.

[0003] Based on the above background, by using data such as archive data and user electricity consumption in the power consumption acquisition system, data preprocessing is carried out from multiple dimensions such as data quality and data changes, and a method for solving meter errors based on reinforcement learning is proposed. Thus, the calculation and output of meter metering error results are remotely realized, promoting the transformation of the operation and maintenance management of metering equipment, ensuring the overall stable operation of metering equipment, and providing strong support for the transformation of metering equipment management from "on-site operation and maintenance + periodic rotation" to "online operation and maintenance + precise rotation". Summary of the Invention

[0004] In view of at least one defect of the prior art, the purpose of the present invention is to provide a method for solving meter errors based on reinforcement learning. First, obtain the electricity consumption data of the electric energy meter, and then based on the law of conservation of energy, construct the electricity quantity matrix and error rate numerical value for the data; use the reinforcement learning qlearning error rate iterative solution algorithm to calculate the meter error, and through this method, the error of the smart meter can be accurately estimated.

[0005] To achieve the above purpose, the present invention adopts the following technical solution: A method for solving meter errors based on reinforcement learning, including the following steps:

[0006] Step 1: Construct the matrix data of the energy conservation equation, and preprocess the electricity consumption data of the electric energy meter;

[0007] Step 2: Based on the law of conservation of energy, initialize and solve the error rate value of the electric energy meter;

[0008] Step 3: Adopt a reinforcement learning model, and based on the reinforcement learning qlearning algorithm, solve and iterate the state, action, and optimization target in the data;

[0009] Step 4: Set the iteration stop condition, and compare and output the final result of the model.

[0010] In Step 1, construct the energy conservation equation;

[0011] Let the meter error be e, and substitute the single-day electricity consumption data to obtain the following equation:

[0012] Daily power consumption a1 i 供入计量设备a1 *(1 - e a1 ) + Daily power consumption a2 i 供入计量设备a2 *(1 - e a2 ) + … = Daily power consumption b1 i 供出计量设备b1 *(1 - e b1 ) + Daily power consumption b2 i 供出计量设备b2 *(1 - e b2 ) + … + Fixed loss Q(1)

[0013] Among them, i represents substituting the data of the i-th day, a1 and a2 are the first and second power meters for incoming power respectively, e a1 , e a2 are the error rates of the first and second power meters for incoming power respectively, b1 and b2 are the first and second power meters for outgoing power respectively, e b1 , e b2 are the error rates of the first and second power meters for outgoing power respectively, and Q is the fixed loss.

[0014] In step one, data preprocessing is performed, that is, data grouping;

[0015] Group the daily power consumption data in the line in the order of proximity of dates in the past year. The grouping rules are as follows:

[0016]

[0017] Among them, the date is represented by T.

[0018] In step two, the initial value solution of the error rate of the power meter is performed, including:

[0019] After grouping, take a set of data and substitute it into the energy conservation equation to obtain multiple daily power consumption equations containing error coefficients as follows:

[0020]

[0021] In the above equation set (3), only the error rate e of the power meter and the fixed loss Q are unknown variables, and a1, a2 …, b1, b2 … are all known variables. Substitute and solve the above equation set to obtain the error rate e of the power meter of the metering device and the fixed loss Q.

[0022] Equation set 3 is a system of multivariate linear equations. As long as the number of equations is greater than the number of unknowns, it can be solved.

[0023] In step three, the error rate e of the power meter of the metering device is iteratively optimized, that is, the reinforcement learning method is used to optimize the model;

[0024] Step A: Define evaluation indicators;

[0025] Solve the obtained error rate values and fixed loss Q from watt-hour meters a1, a2…b1, b2, and the multiple error rates corresponding to watt-hour meter a1;

[0026] e a1D1 e a1D2 D1 and D2 respectively represent the error rates obtained from the first group of daily electricity consumption and the second group of daily electricity consumption;

[0027] Calculate the coefficient of variation based on this error rate value of the watt-hour meter. The calculation formula is as follows:

[0028]

[0029] Among them, SD is the standard deviation, MEAN is the average value, and CV is the coefficient of variation;

[0030] Since the data in formula (2) is divided into 4 groups according to date T, the error rates of the watt-hour meters calculated by formula (3) will also have four groups and the watt-hour meter IDs in each group are the same; Calculate the standard deviation and average value for the four groups of error rate data of each watt-hour meter respectively, and then calculate the coefficient of variation of the watt-hour meter error rate based on formula (4); When the coefficient of variation is between 0 and 1, 1 and 2, 2 and 3, and above 3, the corresponding scores are 3, 1, 0, and -3 respectively; According to this rule, calculate the total score of multiple watt-hour meters to obtain the scoring value S0 of this solution process;

[0031] Step B: Define optimization operations;

[0032] (b1) Select the error rate of the watt-hour meter with the smallest coefficient of variation and the coefficient of variation not equal to 0. Since each watt-hour meter will obtain 4 groups of error rates in formula (3), the selected watt-hour meter in this operation step also has four groups of error rates. Calculate its mean value. The formula is as follows:

[0033]

[0034] Take the obtained mean value as a constant and substitute it into equation group (3) to re-solve the error rate of the watt-hour meter of the metering device:

[0035] (b2) Select the fixed loss Q, take the mean value as a constant, and substitute it into equation group (3) to re-solve the error rate of the watt-hour meter of the metering device;

[0036] (b3) Select the error rate of the watt-hour meter with the smallest coefficient of variation and the coefficient of variation not equal to zero. Assume the error rate of the watt-hour meter is 0 and substitute it into equation group (3) to re-solve the error rate of the watt-hour meter of the metering device;

[0037] Step C: Define the state;

[0038] Define a list with k*m columns; m is the maximum number of electricity meters in the data + 1; k ranges from 1 to 5, and k represents five categories of coefficient of variation, mean error rate, variance of error rate, standard deviation of error rate, and whether the error rate is a constant respectively. The list is used as the state value before the optimization operation of the reinforcement learning model selection, that is, the model input index;

[0039] Step D: Iterative optimization;

[0040] After obtaining the error rate for the first time, the model initializes the Bellman equation based on the current state, and thus dynamically selects the optimization operation in step B according to the Bellman equation to re-solve the error rate, re-calculate the score value S of this solution process, and accumulate the S values of each time as the score value in the total solution process.

[0041] In step four; when the S values no longer change after 5 times, stop the solution, set the number of iterations to 500 times; output the error rate analysis result.

[0042] Remarkable effect: The present invention provides a method for solving the error of an electric meter based on reinforcement learning. First, obtain the power consumption data of the electric meter, and then construct the power matrix and error rate value based on the law of conservation of energy; use the reinforcement learning qlearning error rate iterative solution algorithm to calculate the error of the electric meter, and the error of the smart electric meter can be accurately estimated by this method. Brief description of the drawings

[0043] Figure 1 It is the flow chart of the model solution algorithm in the present invention;

[0044] Figure 2 It is the schematic diagram of the qlearning algorithm

[0045] Figure 3 It is the verification diagram of the algorithm solution effect in the present invention. Detailed implementation manners

[0046] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0047] As Figures 1 - 3 shown, the present invention discloses a method for solving the error of an electric meter based on reinforcement learning, and the technical problems to be solved are as follows:

[0048] First, preprocess and screen the data from the aspects of the quality of the collected data and the trend of the change data of the power consumption data;

[0049] Secondly, construct the power matrix and error rate value for the data based on the law of conservation of energy;

[0050] Thirdly, based on the reinforcement learning qlearning algorithm, solve and iterate the state, action, and optimization objective in the data;

[0051] Finally, set the iteration stop condition and output the solution result of the model error rate.

[0052] The purpose of the present invention is to overcome the defects existing in the prior art and provide a new method for solving the error of an electric meter.

[0053] The main principle of the technical solution of the present invention is as follows:

[0054] A new method for solving the error of an electric meter based on reinforcement learning includes the following steps:

[0055] 1) Construct the matrix data of the energy conservation equation and perform data preprocessing;

[0056] 2) Initialize and solve the numerical value of the electric energy error rate;

[0057] 3) Adopt a reinforcement learning model to optimize the model result;

[0058] 4) Compare and output the final result of the model.

[0059] In step 1), construct the energy conservation equation.

[0060] Let the error of the electric energy meter be e, and substitute the daily electric energy data to obtain the following equation.

[0061] Daily electric energy a1 i 供入计量设备a1 *(1 - e a1 ) + daily electric energy a2 i 供入计量设备a2 *(1 - e a2 ) + … = daily electric energy b1 i 供出计量设备b1 *(1 - e b1 ) + daily electric energy b2 i 供出计量设备b2 *(1 - e b2 ) + … + fixed loss Q(1)

[0062] a1 i *(1 - e a1 ) + a2 i *(1 - e a2 ) + …… = b1 i *(1 - e b1 ) + b2 i *(1 - e b2 ) + …… + Q(1)

[0063] Among them, i represents substituting the data of the i-th day, a1 and a2 are the first and second power supplies into the electric energy meter respectively, e a1 , e a2They are the error rates of the first and second incoming electricity meters respectively, b1 and b2 are the first and second outgoing electricity meters respectively, and e b1 and e b2 are the error rates of the first and second outgoing electricity meters respectively, and Q is the fixed loss.

[0064] Then in step 1), perform data preprocessing - data grouping

[0065] Group the daily electricity consumption data in the line in the order of close dates in the past year. The grouping rules are exemplified as follows.

[0066]

[0067]

[0068] Among them, T represents the date.

[0069] In step 2), perform initialization analysis of the error rate of the metering device

[0070] After grouping, substitute a set of data into the energy conservation equation to obtain multiple daily electricity equations containing error coefficients as follows:

[0071]

[0072] In the above equations, only the meter error rate e and the fixed loss Q are unknown variables, and a1, a2..., b1, b2... are all known variables. Substitute and solve the above equations to obtain the meter error rate e and the fixed loss Q of the metering device.

[0073] In step 3), perform iterative optimization of the error rate of the metering device - adopt the reinforcement learning method to optimize the model

[0074] A Define the evaluation index:

[0075] The error rate values and fixed loss Q obtained from the meters a1, a2... b1, b2, and the multiple error rates corresponding to the meter a1.

[0076] e a1D1 and e a1D2

D1 and D2 represent the error rates obtained from the first group of daily electricity consumption and the second group of daily electricity consumption respectively.

[0077] Calculate the coefficient of variation with this meter error rate value. The calculation formula is as follows:

[0078]

[0079] Among them, SD is the standard deviation, MEAN is the average value, and CV is the coefficient of variation.

[0080] Since the data in formula (2) is divided into 4 groups according to the date, the meter error rates calculated by formula (3) will also have four groups and the meter IDs in each group are the same. The standard deviation and mean of the four groups of error rate data for each meter can be calculated respectively, and then the coefficient of variation of the meter error rate is calculated based on formula (4). When the coefficient of variation is between 0 and 1, 1 and 2, 2 and 3, and above 3, the corresponding scores are 3, 1, 0, and -3 respectively. According to this rule, the sum of the scores of multiple meters is calculated to obtain the scoring value S0 of this solution process.

[0081] B Define the optimization operation:

[0082] (b1) Select the meter error rate with the smallest coefficient of variation and the coefficient of variation not equal to 0. Since each meter will obtain 4 groups of error rates in formula three, the selected meter in this operation step also has four groups of error rates. Calculate its mean value, and the formula is as follows:

[0083]

[0084] Take the obtained mean value as a constant and substitute it into equation set (3) to re-solve the error rate of the metering device:

[0085] (b2) Select the fixed loss Q, calculate the mean value as a constant, and substitute it into equation set (3) to re-solve the error rate of the metering device;

[0086] (b3) Select the meter error rate with the smallest coefficient of variation and the coefficient of variation not equal to zero. Assume the error rate is 0 and substitute it into equation set (3) to re-solve the error rate of the metering device;

[0087] C Define the state:

[0088] Define a list with k*m columns [m is the maximum number of meters in the data + 1], k is 5 (representing five categories of coefficient of variation, mean error rate, error rate variance, error rate standard deviation, and whether the error rate is a constant respectively). The list is used as the state value before the optimization operation of the reinforcement learning model, that is, the input index of the model.

[0089] The example is as follows:

[0090] Table 1 Example table of the pre-state of error rate optimization

[0091] Category Value Coefficient of Variation of Table A1 1.2 Mean Error Rate of Table A1 2％ Variance of Error Rate of Table A1 25 Standard Deviation of Error Rate of Table A1 5 Whether the Error Rate of Table A1 is a Constant 0 … … Standard Deviation of Error Rate of Fixed Loss Q 5 Whether Fixed Loss Q is a Constant 0

[0092] The "..." in the table represents other meters, such as a2, a3... an, b1, b2, b3... bn.

[0093] D Iterative optimization:

[0094] After obtaining the first error rate, the model initializes the Bellman equation based on the current state, and thus dynamically selects the optimization operation in step B according to the Bellman equation to re-solve the error rate, recalculate the scoring value S of this solution process, and accumulate the S values of each time as the score in the total solution process (when the S value no longer changes after multiple times, set to 5 times, stop the solution, and tentatively set the number of iterations to 500 times).

[0095] Output the error rate analysis result in step 4).

[0096] Table 2: Corresponding relationship between the analysis results of the error rate of metering devices under data grouping and the date:

[0097]

[0098] In Table 2, A is the incoming electricity meter and B is the outgoing electricity meter;

[0099] Figure 1 This is the flow chart of the model solution algorithm in the present invention.

[0100] Figure 2 This is the schematic diagram of the qlearning algorithm. (Reinforcement learning algorithm)

[0101] Figure 3 This is the verification diagram of the algorithm solution effect in the present invention. The solution effect is obtained by constructing the errors of 12 electricity meters through simulation data.

[0102] Compared with the prior art, the electricity meter error solution algorithm proposed by the present invention has good static characteristics, dynamic characteristics and strong robustness to external interference. Through this method, the error of the smart electricity meter can be accurately estimated, the efficiency of electricity meter verification can be improved, the labor cost can be saved, the verification of electricity meters in all scenarios can be covered, and thus the service life of each batch of electricity meters can be extended.

[0103] Finally, it should be noted that: the above-listed are only specific implementation examples of the present invention. Of course, those skilled in the art can make changes and modifications to the present invention. If these modifications and variations fall within the scope of the claims of the present invention and its equivalent technologies, they should be considered as the protection scope of the present invention.

Claims

1. A method for solving the error of an electric meter based on reinforcement learning, characterized in that, It includes the following steps: Step 1: Construct the matrix data of the energy conservation equation and preprocess the power consumption data of the electricity meter. Step 2: Based on the law of energy conservation, initialize and solve the error rate value of the electricity meter. Step 3: Adopt a reinforcement learning model and, based on the reinforcement learning qlearning algorithm, solve and iterate the states, actions, and optimization objectives in the data. Step 4: Set the iteration stop condition and compare and output the final result of the model. In Step 1, construct the energy conservation equation. Let the error of the electricity meter be e and substitute the daily power consumption data to obtain the following equation: Daily power consumption a1 i 供入计量设备a1 *(1 - e a1 ) + Daily power consumption a2 i 供入计量设备a2 *(1 - e a2 ) + … = Daily power consumption b1 i 供出计量设备b1 *(1 - e b1 ) + Daily power consumption b2 i 供出计量设备b2 *(1 - e b2 ) + … + Fixed loss Q(1) Among them, i represents substituting the data of the i-th day, a1 and a2 are the first and second electricity meters for incoming power supply respectively, and e a1 , e a2 are the error rates of the first and second electricity meters for incoming power supply respectively, b1 and b2 are the first and second electricity meters for outgoing power supply respectively, and e b1 , e b2 are the error rates of the first and second electricity meters for outgoing power supply respectively, and Q is the fixed loss; In Step 1, perform data preprocessing, that is, data grouping. Group the daily power consumption data in the line in the order of similar dates. The grouping rules are as follows: In Step 2, the initialization and solution of the error rate value of the electricity meter include: After grouping, take a set of data and substitute it into the energy conservation equation to obtain multiple daily power consumption equations containing error coefficients as follows: In the above equations (3), only the error rate e of the electricity meter and the fixed loss Q are unknown variables, and a1, a2..., b1, b2... are all known variables. Substitute and solve the above equations to obtain the error rate e of the electricity meter of the metering device and the fixed loss Q. In Step 3, perform iterative optimization of the error rate e of the electricity meter of the metering device, that is, adopt a reinforcement learning method to optimize the model. Step A: Define the evaluation index. From the error rate values and fixed loss Q obtained by solving the electricity meters a1, a2... b1, b2, according to the daily power consumption data grouping rules, the multiple error rates corresponding to the electricity meter a1. e a1D1 ,e a1D2 are respectively the error rates of the first group of daily electricity amounts and the second group of daily electricity amounts of the first power supply electricity meter, D1 and D2 respectively represent the time periods of the first group and the time periods of the second group; Calculate the coefficient of variation with this error rate value of the electricity meter. The calculation formula is as follows: Among them, SD is the standard deviation, MEAN is the average value, and CV is the coefficient of variation. Since the data in formula (2) is divided into 4 groups according to the date T, the error rates of the electricity meters calculated by formula (3) will also have four groups and the electricity meter IDs in each group are the same; calculate the standard deviation and average value of the four groups of error rate data of each electricity meter respectively, and then calculate the coefficient of variation of the error rate of the electricity meter based on formula (4); when the coefficient of variation is between 0 and 1, 1 and 2, 2 and 3, and above 3, they correspond to scores of 3, 1, 0, and -3 respectively; according to this rule, calculate the total score of multiple electricity meters to obtain the scoring value S0 of this solution process. Step B: Define the optimization operation. (b1) Select the error rate of the electricity meter with the smallest coefficient of variation and the coefficient of variation not equal to 0. Since each electricity meter will obtain 4 groups of error rates in formula (3), the selected electricity meter in this operation step also has four groups of error rates. Calculate its mean value. The formula is as follows: Take the obtained mean value as a constant and substitute it into equations (3) to re-solve the error rate of the electricity meter of the metering device: (b2) Select the fixed loss Q, take the mean value as a constant, and substitute it into equations (3) to re-solve the error rate of the electricity meter of the metering device; (b3) Select the error rate of the electricity meter with the smallest coefficient of variation and the coefficient of variation not equal to zero. Assume that the error rate of the electricity meter is 0 and substitute it into equations (3) to re-solve the error rate of the electricity meter of the metering device; Step C: Define the state. Define a list with k * m columns; m is the maximum number of electricity meters in the data + 1; k ranges from 1 to 5, and k represents the coefficient of variation, mean error rate, variance of error rate, standard deviation of error rate, and error rate respectively, which are used as the state values before the reinforcement learning model selects optimization operations, that is, the model input indicators. Step D: Iterative optimization; After obtaining the error rate for the first time, the model initializes the Bellman equation based on the current state, and thus dynamically selects the optimization operation in step B according to the Bellman equation to re-solve the error rate, recalculate the score value S for this solution process, and accumulate the S values for each time to obtain the total score value in the overall solution process.

2. The method for solving the error of an electric meter based on reinforcement learning according to claim 1, wherein: In step four: When the S values no longer change after 5 times, stop the solution, set the number of iterations to 500 times; output the error rate analysis result.

Images