Intelligent electric meter dynamic error online calibration method and system based on reinforcement learning
By employing a reinforcement learning-based online dynamic error calibration method for smart meters and utilizing a long short-term memory network to construct a policy function model, dynamic calibration of smart meters is achieved. This solves the problem of difficulty in real-time correction of metering accuracy and improves detection efficiency and data reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTHEAST UNIV
- Filing Date
- 2026-02-03
- Publication Date
- 2026-05-29
AI Technical Summary
Existing electricity meters cannot achieve real-time correction of metering accuracy without disassembly and inspection, resulting in low resource utilization efficiency and delayed detection of abnormal meters.
A reinforcement learning-based online dynamic error calibration method for smart meters is adopted. By receiving the state matrix of the smart meter, a policy function model is constructed using a long short-term memory network for dynamic calibration. The parameters are optimized by using the reward value of the measured value and the preset real value to achieve dynamic calibration of the smart meter.
It enables rapid, batch testing of meter errors without power outages or meter disassembly, improving testing efficiency, ensuring real-time traceability and tamper-proofness of calibration data, providing credible evidence for both power suppliers and users, and ensuring the reliability of source data.
Smart Images
Figure CN122109972A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of smart meter dynamic online calibration technology, specifically a smart meter dynamic error online calibration method and system based on reinforcement learning. Background Technology
[0002] With the continuous growth of the global economy and the increasing demand for energy, electricity has become a crucial basic energy source for the modern economic system and social operation. Driven by the continuous development of information and communication technologies, data transmission technologies, decision support systems, and automatic control technologies, the construction of smart grids has become an important direction for the evolution of power systems and a strategic focus of the State Grid Corporation of China. Advanced Metering Infrastructure (AMI), as an important component of the smart grid, is used to realize the automatic collection and management of user electricity data. Smart meters are important devices in the advanced metering system of the smart grid, possessing functions such as energy metering, information storage and processing, network communication, automatic control, and information interaction. Statistical data shows that the global installed capacity of smart meters has exceeded one billion units, and the coverage rate of smart meters in my country continues to increase, with some regions achieving full deployment. While the current periodic mandatory dismantling and inspection system for electricity meters has ensured metering accuracy to a certain extent, problems such as low resource utilization efficiency and delayed detection of meter malfunctions still exist. Therefore, it is necessary to achieve error correction of the operating status of electricity meters, especially metering accuracy, without the need for dismantling and inspection. Summary of the Invention
[0003] To address the shortcomings mentioned in the background section, the present invention aims to provide a method and system for online calibration of dynamic errors in smart meters based on reinforcement learning.
[0004] Firstly, the objective of this invention can be achieved through the following technical solution: an online dynamic error calibration method for smart meters based on reinforcement learning, the method comprising the following steps: Receive the smart meter status matrix, wherein the smart meter status matrix is constructed based on the active power value collected by the smart meter at each collection time; The smart meter state matrix is input into a pre-established strategy function model, and the action types of smart meter operation are output. The smart meter is dynamically calibrated based on the action types of smart meter operation. The action types of smart meter operation include three types: decreasing, increasing, and keeping the preset adjustment value unchanged. The strategy function model is constructed based on a long short-term memory network. The measured values of the smart meter after dynamic calibration are obtained. The parameters of the pre-established strategy function model are updated and optimized by calculating the reward value of the measured value and the preset true value after the smart meter performs the operation, so as to obtain the optimized strategy function model. The dynamic calibration decision of the smart meter is optimized based on the optimized strategy function model.
[0005] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the process of constructing the smart meter state matrix is as follows: Define the states of the power distribution network at each moment as a finite set. , a certain The state at time point is defined as follows: ,but: (1) It is known that each meter is at Active power values collected at any given time And number all user meters except the master meter, then Each electricity meter The state matrix at each moment is used as the smart meter's state matrix, as follows: (2).
[0006] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the action type of the smart meter operation is a set of actions performed by the smart meter. As shown in the following formula: (3) (4) In the formula, express Real-time monitoring of all smart meter activity. yes Time of the first A smart meter is defined to perform only three actions: decrease, increase, and remain unchanged.
[0007] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the preset adjustment value is determined based on the error level of the smart meter.
[0008] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: when updating and optimizing the parameters of the pre-established strategy function model by calculating the reward value of the measured value and the preset true value after the smart meter performs the operation, the reward function for calculating the reward value is as follows: Set reward function Let the actual value of the smart meter be... When the smart meter's action approaches this true value, it will be as follows: (5) In the formula To execute several times , This represents the measured value after the smart meter has been operated.
[0009] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the pre-established strategy function model is as follows: In the policy function Next, the system runs Hours, collected Group data, based on the previously defined reward function This leads to the overall expectation. The formula is as follows: (6) In the formula For the parameters in a Long Short-Term Memory (LSTM) network, since the number of samplings is finite, the above formula is approximated as: (7) In the formula This represents the summation from the current state to the end of the reward calculation. A previously defined discount factor needs to be added during the calculation to control the reward result, as shown in the following formula: (8).
[0010] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: the loss function for updating and optimizing the parameters of the pre-established policy function model is as follows: Define a loss function using the negative of equation (7), as follows: (9) After the calculation is complete, the parameters are updated using gradient descent. The formula is as follows: (10) In the formula Let the loss function be (9). This is the learning rate.
[0011] Secondly, in order to achieve the above objectives, this invention discloses an online dynamic error calibration system for smart meters based on reinforcement learning, comprising: The data receiving module is used to receive the smart meter status matrix, wherein the smart meter status matrix is constructed based on the active power value collected by the smart meter at each collection time. The operation calibration module is used to input the smart meter state matrix into a pre-established strategy function model and output the action type of the smart meter operation. Based on the action type of the smart meter operation, the module performs dynamic calibration on the smart meter. The action types of the smart meter operation include three types: decreasing, increasing, and keeping the preset adjustment value unchanged. The strategy function model is constructed based on a long short-term memory network. The dynamic optimization module is used to obtain the measured values of the smart meter after performing dynamic calibration. By calculating the reward value of the measured value after the smart meter performs the operation and the preset true value, the module updates and optimizes the parameters of the pre-established strategy function model to obtain the optimized strategy function model. The optimized strategy function model is then used to optimize the dynamic calibration decision of the smart meter.
[0012] In another aspect of the present invention, in order to achieve the above-mentioned objective, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor. When the processor loads and executes the computer program, it employs the online calibration method for dynamic error of smart meters based on reinforcement learning as described above.
[0013] In another aspect of the present invention, in order to achieve the above-mentioned objective, a computer-readable storage medium is disclosed, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is loaded and executed by a processor, the online calibration method for dynamic error of smart meters based on reinforcement learning as described above is employed.
[0014] The beneficial effects of this invention are: This invention enables online calibration without power outages or meter removal, allowing for rapid and batch testing of a large number of meters, resulting in improved efficiency. The calibration data is traceable in real time and tamper-proof, providing a highly credible basis for settlement between electricity suppliers and users. Online calibration ensures the reliability of source data. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the online calibration scenario for dynamic errors in the distribution network of the transformer substation according to the present invention; Figure 3 This is a schematic diagram of the long and short time memory network of the present invention; Figure 4This is a schematic diagram of the training process of the present invention; Figure 5 This is a schematic diagram of the online calibration process of the present invention; Figure 6 This is a schematic diagram of the system structure of the present invention. Detailed Implementation
[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0017] Example 1: like Figure 1 As shown, the online dynamic error calibration method for smart meters based on reinforcement learning involves transmitting information from the smart meter to a concentrator, which then collects the information and transmits it to the master station. The master station processes the information and, through the concentrator, issues commands to the smart meters in the distribution network. The smart meters then perform dynamic calibration using their built-in reference sources according to the master station's commands. The method includes the following steps: S101: Receive the smart meter status matrix, wherein the smart meter status matrix is constructed based on the active power value collected by the smart meter at each collection time. The training subjects include intelligent agents Agent state space S, agent dynamics A, agent reward R, state transition function T, reward discount factor , by tuple composition.
[0018] Define the main station as an intelligent agent. Define the states of the power distribution network at each moment as a finite set. , a certain The state at time point is defined as follows: ,but: (1) If the present invention is known that each meter is in Active power values collected at any given time And number all user meters except the master meter. Each electricity meter The state matrix at time t is: (2) S102: Input the smart meter state matrix into the pre-established strategy function model, output the action type of smart meter operation, and perform dynamic calibration of the smart meter based on the action type of smart meter operation; wherein, the action type of smart meter operation includes three action types: decreasing, increasing and keeping unchanged by a preset adjustment value; the strategy function model is constructed based on a long short-term memory network; Define a set of actions for operating a smart meter. As shown in the following formula: (3) (4) In the formula, express Real-time monitoring of all smart meter activity. yes Time of the first The smart meter is the operation performed. Here, this invention defines that the smart meter can only perform three actions: decrease, increase, and remain unchanged, and the values for increase and decrease are set as constants in this invention. .For example, The value can be determined based on the error level of the smart meter. Generally, the error requirement for user meters is within 2%. The measured value of a meter is now... ,but However, it can also be determined based on other factors.
[0019] Define a reward function Here, the present invention assumes that the true value of the smart meter is When the smart meter's action approaches this true value, it will be as follows: (5) In the formula To execute several times , The reward is based on the reading of the table. ,in You can choose according to the requirements. When the smart meter selects not to operate, the reward value is... When the smart meter's action deviates significantly from the actual value, the reward value is... However, this invention requires a limitation. This is used to indicate that this is a bad action. Since the electricity meter readings are accumulated, the error value of each meter is known during the model training phase (the model cannot observe it), and the true value can be directly calculated and compared with the collected value.
[0020] Define a discount factor ,and You can choose any value here if you want to monitor the current state. The value of should be as small as possible if we want to focus on the long-term state. The value should be as large as possible.
[0021] Define a policy function The output is the current state of the system, and the output is the probability distribution of the next action.
[0022] The time-series data generated by smart meters contains complex electricity consumption patterns, and Long Short-Term Memory (LSTM) networks are an ideal model for processing this type of data. Their core advantage lies in their unique gating mechanism, which effectively models long-term dependencies and accurately captures multiple periodic and nonlinear abrupt changes in electricity consumption behavior, including daily, weekly, and yearly patterns. LSTM has demonstrated outstanding performance in key smart grid tasks such as load forecasting, non-intrusive load decomposition, and abnormal electricity consumption detection, becoming a core algorithm engine for extracting deep insights from massive amounts of meter data and driving the digitalization of the power grid.
[0023] This invention uses a Long Short-Term Memory (LSTM) network as the policy function. It includes 7 configurable parameters (input_size, hidden_size, num_layer, bias, batch_first, dropout, bidirectional). The input_size is the input... t The smart meter readings at any given time, along with hidden_size, num_layer, bias, batch_first, dropout, and bidirectional, are all model parameter values. hidden_size determines the model's memory capacity; a larger value indicates a more complex model and better learning performance, but also increases learning time. num_layer determines the number of LSTM layers to use. Bias determines whether to add a bias term 'b' to the gating calculation, providing the model with additional learning degrees of freedom. batch_first controls the order of input and output tensor dimensions; dropout settings prevent overfitting and improve generalization ability; bidirectional determines whether to use a bidirectional LSTM structure.
[0024] Data collection Run the LSTM network in the front-end system Hours, each unit time interval In one collection, a total of [data] can be collected. Group data, in which .
[0025] S103: Obtain the measured value of the smart meter after performing dynamic calibration, and update and optimize the parameters of the pre-established strategy function model by calculating the reward value of the measured value and the preset true value after the smart meter performs the operation, so as to obtain the optimized strategy function model. Based on the optimized strategy function model, optimize the dynamic calibration decision of the smart meter.
[0026] Calculate Returns In the policy function Next, the system runs Hours, collected Group data, based on the previously defined reward function The overall expectation can be calculated. The formula is as follows: (6) In the formula For the parameters in the above network, since the number of samplings is finite, the above formula is approximately: (7) In the formula This represents the summation from the current state to the end of the reward calculation. A previously defined discount factor needs to be added during the calculation to control the reward result, as shown in the following formula: (8) Update parameters Define a loss function using the negative of equation (7), as follows: (9) After the calculation is complete, the parameters are updated using gradient descent. The formula is as follows: (10) In the formula Let the loss function be (9). This is the learning rate.
[0027] Specifically, the present invention will be further illustrated below through embodiments: For a distribution area with 100 users, comprising 101 meters (1 main meter with a speed of 0.5 seconds and 100 sub-meters with a speed of level 1), the meter error can be modified without affecting normal user operation. To facilitate calculations, the electricity meters for the 100 users are numbered from 001 to 100. Therefore, the error for each sub-meter is... The newly installed table assumes an error in this invention. All values are 0. Starting from midnight, the concentrator collects data and transmits it to the main station every 15 minutes. (1) Model training: Based on the above scenario, define the above parameters for each time step. Corresponding to a state value (This is called the observation value at the current time), for example, at 1 o'clock, the user's sub-table observation value is... Next, this invention defines state action constants. Set to one percent of the current observations, at time 2, Table 2 needs to be reduced, meaning the observations are changed to... In the reward function Set to respectively The discount factor is set to Policy function This is an LSTM model. During the first training iteration, at time 0, the random policy function... and error In the policy function After running for 24 hours, a round of data is collected, and the data is processed using the policy functions in equations (9) and (10). Next, the time was reset to 0:00, at which point only random error occurred. Retain the policy function from the previous round. Repeat the above steps to update the policy function again. This process is repeated until the policy function is reached. Until convergence.
[0028] (2) On-site verification: Now, the present invention adjusts the error of the following 5 meters at 0:00, as shown in Table 1 below. Table 1 After 24 hours of model correction, this invention recalibrated the five meters (003, 015, 048, 078, and 094) on-site, and the errors of the five meters were found to be as shown in Table 2: Table 2 The fact that the error values of all five meters have decreased indicates that the main station has corrected the errors of these five meters, and the corrected meters all meet the meter grade requirements.
[0029] Example 2: To achieve the above objective, such as Figure 6 As shown, based on Embodiment 1, this invention discloses an online dynamic error calibration system for smart meters based on reinforcement learning, comprising: The data receiving module 11 is used to receive the smart meter status matrix, wherein the smart meter status matrix is constructed based on the active power value collected by the smart meter at each collection time. The operation calibration module 12 is used to input the smart meter state matrix into a pre-established strategy function model, output the action type of the smart meter operation, and perform dynamic calibration of the smart meter based on the action type of the smart meter operation; wherein, the action type of the smart meter operation includes three action types: decreasing, increasing and keeping unchanged by a preset adjustment value; the strategy function model is constructed based on a long short-term memory network; The dynamic optimization module 13 is used to obtain the measured value of the smart meter after performing dynamic calibration. By calculating the reward value of the measured value and the preset true value after the smart meter performs the operation, the pre-established strategy function model is updated and optimized to obtain the optimized strategy function model. The dynamic calibration decision of the smart meter is optimized based on the optimized strategy function model.
[0030] Based on the same inventive concept, this invention also provides a computer device, comprising: one or more processors, and a memory for storing one or more computer programs; the programs include program instructions, and the processor executes the program instructions stored in the memory. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, used to implement one or more instructions, specifically for loading and executing one or more instructions stored in a computer storage medium to implement the above-described method.
[0031] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium storing a computer program, which, when executed by a processor, performs the above-described method. This storage medium can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0032] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0033] The foregoing has shown and described the basic principles, main features, and advantages of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this disclosure. Various changes and modifications can be made to this disclosure without departing from its spirit and scope, and all such changes and modifications fall within the scope of this disclosure as claimed.
Claims
1. A method for online calibration of dynamic errors in smart meters based on reinforcement learning, characterized in that, The method includes the following steps: Receive the smart meter status matrix, wherein the smart meter status matrix is constructed based on the active power value collected by the smart meter at each collection time; The smart meter state matrix is input into a pre-established strategy function model, and the action types of smart meter operation are output. The smart meter is dynamically calibrated based on the action types of smart meter operation. The action types of smart meter operation include three types: decreasing, increasing, and keeping the preset adjustment value unchanged. The strategy function model is constructed based on a long short-term memory network. The measured values of the smart meter after dynamic calibration are obtained. The parameters of the pre-established strategy function model are updated and optimized by calculating the reward value of the measured value and the preset true value after the smart meter performs the operation, so as to obtain the optimized strategy function model. The dynamic calibration decision of the smart meter is optimized based on the optimized strategy function model.
2. The online calibration method for dynamic errors of smart meters based on reinforcement learning according to claim 1, characterized in that, The process of constructing the smart meter state matrix is as follows: Define the states of the power distribution network at each moment as a finite set. , a certain The state at time point is defined as follows: ,but: （1） It is known that each meter is at Active power values collected at any given time And number all user meters except the master meter, then Each electricity meter The state matrix at each moment is used as the smart meter's state matrix, as follows: （2）。 3. The online calibration method for dynamic errors of smart meters based on reinforcement learning according to claim 1, characterized in that, The action type of the smart meter operation is a set of actions performed by the smart meter. As shown in the following formula: （3） （4） In the formula, express Real-time monitoring of all smart meter activity. yes Time of the first A smart meter is defined to perform only three actions: decrease, increase, and remain unchanged.
4. The online calibration method for dynamic errors of smart meters based on reinforcement learning according to claim 1, characterized in that, The preset adjustment value is determined based on the error level of the smart meter.
5. The online calibration method for dynamic errors of smart meters based on reinforcement learning according to claim 1, characterized in that, When updating and optimizing the parameters of the pre-established strategy function model by calculating the reward value of the measured value after the smart meter performs the operation and the preset true value, the reward function for calculating the reward value is as follows: Set reward function Let the actual value of the smart meter be... When the smart meter's action approaches this true value, it will be as follows: （5） In the formula To execute several times , This is the measured value after the smart meter performs the operation.
6. The online calibration method for dynamic errors of smart meters based on reinforcement learning according to claim 5, characterized in that, The pre-established policy function model is as follows: In the policy function Next, the system runs Hours, collected Group data, based on the previously defined reward function This leads to the overall expectation. The formula is as follows: （6） In the formula For the parameters in a Long Short-Term Memory (LSTM) network, since the number of samplings is finite, the above formula is approximated as: （7） In the formula This represents the summation from the current state to the end of the reward calculation. A previously defined discount factor needs to be added during the calculation to control the reward result, as shown in the following formula: （8）。 7. The online calibration method for dynamic errors of smart meters based on reinforcement learning according to claim 6, characterized in that, The loss function used to update and optimize the parameters of the pre-established policy function model is as follows: Define a loss function using the negative of equation (7), as follows: （9） After the calculation is complete, the parameters are updated using gradient descent. The formula is as follows: （10） In the formula Let the loss function be (9). This is the learning rate.
8. A smart meter dynamic error online calibration system based on reinforcement learning, employing the smart meter dynamic error online calibration method based on reinforcement learning as described in any one of claims 1 to 7, characterized in that, include: The data receiving module is used to receive the smart meter status matrix, wherein the smart meter status matrix is constructed based on the active power value collected by the smart meter at each collection time. The operation calibration module is used to input the smart meter state matrix into a pre-established strategy function model and output the action type of the smart meter operation. Based on the action type of the smart meter operation, the module performs dynamic calibration on the smart meter. The action types of the smart meter operation include three types: decreasing, increasing, and keeping the preset adjustment value unchanged. The strategy function model is constructed based on a long short-term memory network. The dynamic optimization module is used to obtain the measured values of the smart meter after performing dynamic calibration. By calculating the reward value of the measured value after the smart meter performs the operation and the preset true value, the module updates and optimizes the parameters of the pre-established strategy function model to obtain the optimized strategy function model. The optimized strategy function model is then used to optimize the dynamic calibration decision of the smart meter.
9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on a processor. When the processor loads and executes the computer program, it employs the online calibration method for dynamic error of smart meters based on reinforcement learning, as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs the online calibration method for dynamic error of smart meters based on reinforcement learning, as described in any one of claims 1 to 7.