Relay tester self-calibration method based on artificial intelligence

Abstract

The invention provides a relay tester self-calibration method based on artificial intelligence, which belongs to the field of automatic test calibration and comprises the steps of acquiring and preprocessing basic data, establishing a self-calibration model based on inverse reinforcement learning, training the self-calibration model and performing model driving self-calibration. And a pretreatment stage. In the inverse reinforcement learning model, utilizing maximum entropy inverse reinforcement learning to perform reverse learning to obtain a reward function, and performing value iteration to obtain an optimal strategy. In the training stage, parameters of the reward function are iteratively optimized through simulation data, and a convergent reward function and strategy are obtained. And using the trained model as a controller to carry out self-calibration on the equipment. According to the method, the advantages of artificial intelligence are utilized, the calibration precision and efficiency of the relay tester can be improved, the operation and maintenance cost is reduced, the service life of equipment is prolonged, and the intelligent level of the equipment is improved.

Description

Technical Field

[0001] The invention belongs to the field of automatic test and calibration, and more specifically relates to a self-calibration method for a relay tester based on artificial intelligence. Background Art

[0002] Relay is an important power system protection device, and its working status directly affects the safety and stability of the power system. In order to ensure the normal operation of the relay, it needs to be calibrated regularly. The traditional relay calibration method is mainly manual operation, which requires a lot of manpower and is prone to errors. Therefore, for large-scale, high-precision relay test instruments, the traditional calibration method can no longer meet the needs.

[0003] In recent years, with the development of artificial intelligence technology, artificial intelligence has achieved breakthrough applications in many fields, such as natural language processing, robotics, image recognition, etc. In particular, the emergence of technologies such as deep learning and reinforcement learning has enabled artificial intelligence to better solve some complex problems.

[0004] However, in the field of protection equipment calibration of power systems, how to use artificial intelligence for automatic calibration and improve the accuracy and efficiency of calibration is still an important and challenging topic. One possible technical solution is to use reinforcement learning algorithms for calibration. However, this method requires a large number of training samples, and it is often difficult to obtain enough samples in practical applications. In addition, due to the complexity of the power system, there may be many unknown and unpredictable factors, which makes the direct application of reinforcement learning methods face great challenges.

[0005] Inverse reinforcement learning (IRL) is an important extension of reinforcement learning. Unlike traditional reinforcement learning, IRL does not directly solve the optimal strategy, but learns from the expert's strategy to solve the corresponding reward function, and then calculates the optimal strategy based on this reward function. Therefore, compared with reinforcement learning, IRL does not require a large number of training samples, can better adapt to complex environments, and the problems it solves are more in line with actual needs. Summary of the invention

[0006] The present invention makes use of this advanced artificial intelligence technology and applies inverse reinforcement learning to the self-calibration process of the relay tester, effectively solving the problem of self-calibration of the relay tester based on artificial intelligence, and improving the efficiency and accuracy of users' calibration of power equipment around the world.

[0007] In order to achieve the above object, the present invention is implemented by adopting the following technical scheme: the method comprises:

[0008] Basic data collection and preprocessing: In the initial stage of equipment calibration, the calibration data, operating parameters and output results of the equipment are collected, and then these data are preprocessed;

[0009] Establish a self-calibration model based on inverse reinforcement learning; use inverse reinforcement learning to learn the optimal calibration of the device calibration task;

[0010] Train a self-calibrated model;

[0011] Model-driven self-calibration: Use the self-calibration model as a controller to self-calibrate the device; when new observation data is input, the model predicts the optimal control strategy and self-calibrates the device.

[0012] In one embodiment, the basic data acquisition collects basic data of the relay tester, including a calibration data set D of the device, an operating parameter set P of the device, and an output result set O of the device; and is obtained through the built-in sensor of the device or an external device.

[0013] In one embodiment, the pretreatment comprises:

[0014] (1) Outlier Detection

[0015] Use Z-Score to detect and remove outliers in the data set:

[0016] (2) Data cleaning, using interpolation methods for processing;

[0017] (3) Data standardization, using the maximum and minimum normalization method,

[0018] Through the above steps, the processed calibration data set D', the operation parameter set P' and the output result set O' are obtained, which is ready for the next step of model building;

[0019] In one embodiment, the self-calibration model includes:

[0020] (1) Define state space and action space

[0021] Determine the state space S, i.e., the calibration data and operating parameters of the device, and the action space A, i.e., the actions that change the device parameters; the state s belongs to the state space S, and the action a belongs to the action space A;

[0022] (2) Determine the optimization goal

[0023] The goal is to maximize the expected cumulative reward under strategy π, that is, to find the optimal strategy π*, the formula is as follows:

[0024]

[0025] E is expectation, is the expected cumulative reward under strategy π, where R(s t ,a t ) is in state s t Next, perform action a t The reward obtained, γ is the decay factor of future rewards, t is the time step, and T is the termination time;

[0026] (3) Using inverse reinforcement learning to obtain the optimal reward function

[0027] 1) First, a parameterized reward function R(s, a; θ) is preset, where θ is the parameter of the reward function;

[0028] 2) Then, through the sampled trajectory information, the maximum entropy inverse reinforcement learning method is used to reversely learn an optimal reward function R(s,a;θ); this is an optimization problem, finding a parameter vector θ that maximizes the following objective function J(θ):

[0029]

[0030] In this formula, Indicates the expectation that in state s t Take action t The rewards obtained, and the discounted sum of all accumulated rewards;

[0031] 3) By iteratively updating θ, the optimal reward function is obtained, and then value iteration is used to obtain the final optimal strategy based on the optimal reward function;

[0032] Finally, use the optimal policy π^ predicted by the model to apply the best action to the new state and thereby perform self-calibration.

[0033] In one embodiment, the self-calibration model training process includes:

[0034] (1) First, set the initial strategy π, usually a random strategy; and set the initial reward function parameter θ;

[0035] (2) Generate a block of simulation data using the current policy π, which is achieved by collecting a series of device states s, actions taken a, and device results;

[0036] (3) Based on the existing simulation data, the parameters of the reward function are optimized according to the objective function of maximum entropy inverse reinforcement learning:

[0037]

[0038] Update the reward function parameters by the gradient ascent method, namely:

[0039]

[0040] Among them, θ current is the current reward function parameter, is the objective function J with respect to the parameter θ at the current parameter θ current The gradient at, α is the learning rate, θ next is the updated parameter;

[0041] (4) With the new reward function, we can optimize the strategy. According to the principle of reinforcement learning, we can find a new strategy π. next :

[0042] π next (a|s)=argmax∑R(s,a;θ next )

[0043] (5) Then the new strategy π next As the updated strategy, the next iteration is performed until the strategy π and the reward function R(s,a;θ next ) converges, the model training is completed.

[0044] In one embodiment, the model-driven self-calibration includes:

[0045] The trained inverse reinforcement learning model acts as a “controller” to automatically calibrate the relay using the optimal policy predicted based on new observations.

[0046] (1) Input new observation data:

[0047] Including the state of the relay, working environment conditions new , and input into the trained model; (2) the model predicts the optimal calibration action

[0048] According to the model, for a set of actions a, find the optimal calibration action a best , to maximize the expected reward:

[0049] a best = argmax a ∑[π(a|s new )*R(s new ,a)]

[0050] Among them, π(a|s new ) indicates that in the current state s new The probability of taking action a, R(s new ,a) means in state s new The reward obtained by executing action a; argmax a It means finding the action a that maximizes the expected reward best ;

[0051] (3) Self-calibration of the device

[0052] With the optimal calibration action a best , set and adjust the relay to achieve an optimal working state.

[0053] Beneficial effects of the present invention:

[0054] Improve calibration accuracy: Through the learning and training of machine learning models, device parameters and status can be judged more accurately, and calibration actions can be performed accurately, thereby improving calibration accuracy.

[0055] Improve calibration efficiency: Calibration is performed in an automated manner, avoiding the tedious manual operation, saving a lot of human resources and improving calibration efficiency.

[0056] Enhance the self-adaptation ability of the device: The machine learning model can predict the optimal calibration action based on the current state of the device and environmental conditions, so that the device has the ability to self-adapt and adapt to various complex environments and conditions.

[0057] Improve equipment life: Regular self-calibration can effectively maintain the equipment in good operating condition, reduce equipment damage caused by parameter deviation, and thus improve the equipment life. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flow chart of the method of the present invention;

[0059] Figure 2 A self-calibration model flow chart is established for the present invention;

[0060] Figure 3 This is a flow chart of training the self-calibration model of the present invention. DETAILED DESCRIPTION

[0061] In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Typical embodiments of the present invention are given in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in the present invention. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0062] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used in the present invention in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. In order to facilitate the understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Typical embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in the present invention. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0063] Unless otherwise defined, all technical and scientific terms used in the present invention have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0064] like Figure 1 As shown, a self-calibration method for a relay tester based on artificial intelligence includes:

[0065] S1. Basic data collection and preprocessing. In the initial stage of equipment calibration, the calibration data, operating parameters and output results of the equipment are collected, and then these data are preprocessed.

[0066] The basic data acquisition collects the basic data of the relay tester, including the calibration data set D of the device, the operating parameter set P of the device, and the output result set O of the device; and is obtained through the built-in sensor of the device or external device.

[0067] The pretreatment comprises:

[0068] (1) Outlier detection: Use Z-Score to detect and remove outliers in the data set:

[0069] Z=(X-μ) / σ

[0070] Where X is a single data value, μ is the mean of all data, and σ is the standard deviation of all data. If the Z-Score of a data value exceeds a certain threshold (for example, 3), the data can be considered an outlier.

[0071] (2) Data cleaning: interpolation method is used for processing.

[0072] (3) Data standardization, using the maximum and minimum normalization method;

[0073] In order to eliminate the impact of different data units and inconsistent dimensions on the model, the data needs to be standardized. This is the maximum and minimum normalization, and the mathematical formula is as follows:

[0074] X'=(X-Xmin) / (Xmax-Xmin)

[0075] Among them, X is the original data, X' is the standardized data, Xmin and Xmax are the minimum and maximum values ​​in the original data set respectively.

[0076] Through the above steps, the processed calibration data set D', the operation parameter set P' and the output result set O' are obtained, preparing for the next step of model building.

[0077] S2. Establish a self-calibration model based on inverse reinforcement learning; use inverse reinforcement learning to learn the optimal calibration of the device calibration task. Inverse reinforcement learning is a scenario that learns the decision maker from the observed behavior, allowing us to reversely infer the reward mechanism of the target task and thus learn the optimal decision-making strategy.

[0078] like Figure 2 As shown, the self-calibration model includes:

[0079] S201, define state space and action space:

[0080] Before building the model, it is necessary to determine the state space S (i.e., the calibration data and operating parameters of the device) and the action space A (i.e., the actions that change the device parameters). The state s belongs to the state space S, and the action a belongs to the action space A.

[0081] S202, determine the optimization target:

[0082] The goal is to maximize the expected cumulative reward under strategy π, that is, to find the optimal strategy π*, the formula is as follows:

[0083]

[0084] E is expectation, is the expected cumulative reward under strategy π, where R(s t ,a t ) is in state s t Next, perform action a t is the reward obtained, γ is the decay factor of future rewards, t is the time step, and T is the termination time.

[0085] S203. Use inverse reinforcement learning to obtain the optimal reward function:

[0086] 1) First, a parameterized reward function R(s, a; θ) is preset, where θ is the parameter of the reward function;

[0087] 2) Then, through the sampled trajectory information, the maximum entropy inverse reinforcement learning method is used to reversely learn an optimal reward function R(s,a;θ); this is an optimization problem, finding a parameter vector θ that maximizes the following objective function J(θ):

[0088]

[0089] In this formula, Indicates the expectation that in state s t Take action t The rewards obtained are calculated by discounting all the accumulated rewards.

[0090] 3) By iteratively updating θ, the optimal reward function is obtained, and then value iteration is used to obtain the final optimal strategy based on the optimal reward function;

[0091] Finally, use the optimal policy π^ predicted by the model to apply the best action to the new state and thereby perform self-calibration.

[0092] S3. Training the self-calibration model: The model training process mainly includes two parts: learning the reward function and optimizing the strategy. In each training iteration, we first obtain experience data based on the current strategy, and then update the reward function and strategy based on the collected data.

[0093] like Figure 3 As shown, the self-calibration model training process includes:

[0094] S301. First, set the initial strategy π as a random strategy; and set the initial reward function parameter θ.

[0095] S302, using the current strategy π to generate a piece of simulation data, which is achieved through a series of device states s, actions a taken, and collection of device results.

[0096] S303, based on the existing simulation data, optimize the parameters of the reward function, according to the objective function of the maximum entropy inverse reinforcement learning:

[0097]

[0098] Update the reward function parameters by the gradient ascent method, namely:

[0099]

[0100] Among them, θ current is the current reward function parameter, is the objective function J with respect to the parameter θ at the current parameter θ current The gradient at, α is the learning rate, θ next is the updated parameter;

[0101] S304. With the new reward function, we optimize the strategy; based on the principle of reinforcement learning, we find a new strategy π next :

[0102] π next (a|s)=argmax∑R(s,a;θ next )

[0103] S305, then the new strategy π next As the updated strategy, the next iteration is performed until the strategy π and the reward function R(s,a;θ next ) converges, the model training is completed.

[0104] S4, model-driven self-calibration; use the self-calibration model as a controller to self-calibrate the device; when new observation data is input, the model predicts the optimal control strategy and self-calibrates the device.

[0105] The model driven self-calibration includes:

[0106] The trained inverse reinforcement learning model acts as a “controller” to automatically calibrate the relay using the optimal policy predicted based on new observations.

[0107] (1) Input new observation data:

[0108] Including the state of the relay, working environment conditions new , and input into the trained model; (2) the model predicts the optimal calibration action

[0109] According to the model, for a set of actions a, find the optimal calibration action a best , to maximize the expected reward:

[0110] a best = argmax a ∑[π(a|s new )*R(s new ,a)]

[0111] Among them, π(a|s new ) indicates that in the current state s new The probability of taking action a, R(s new ,a) means in state s new The reward obtained by executing action a; argmax a It means finding the action a that maximizes the expected reward best ;

[0112] (3) Self-calibration of the device

[0113] With the optimal calibration action a best , set and adjust the relay to achieve an optimal working state.

[0114] In the relay tester, the calibration action a_best includes (but is not limited to) a series of equipment parameter adjustment or calibration procedures such as modifying error tolerance, calibrating voltage and current output, changing time parameters, adjusting protection action division values, etc. Which actions are most effective in actual applications needs to be determined based on the actual data collected and the results of the inverse reinforcement learning model.

[0115] To achieve these optimal calibration actions, the following is a process:

[0116] a. Identify the best action

[0117] Through the inverse reinforcement learning model, we can know the expected reward of each possible action a in a certain state s_new. We choose the action with the largest expected reward as the optimal action a_best.

[0118] b. Mapping optimal actions to specific operations

[0119] According to different device parameters or indicators, the optimal action a_best needs to be mapped to a specific device calibration operation. For example, if the optimal action is to adjust the error tolerance, it is necessary to find which error tolerance parameter to adjust and the adjustment range.

[0120] c. Perform specific operations

[0121] Perform specific calibration operations on the device, for example, adjusting the voltage and current output or adjusting the time parameters, etc., until the device reaches the optimal working state.

[0122] Verify the working effect of the updated equipment and feed the results back to the model as the data source for the next round of iterations. This way, the model and calibration operations can be continuously optimized to keep the equipment working in the best condition.

[0123] Embodiment 1:

[0124] First, collect the basic data of a relay tester, which includes the calibration data set D of the device (for example, 1, 1.2, 0.9, 1.1...), the operating parameter set P of the device (for example, temperature 25℃, humidity 60%...), and the output result set O of the device (for example, switch action time 0.9s, 0.8s, 1s...).

[0125] Then, the data is preprocessed. First, we use the Z-Score outlier detection method to detect and remove outliers in the data set. For example, we find that data 1.2 is an outlier and remove it. Then, we use the interpolation method to clean the data. Finally, we perform data standardization and use the maximum and minimum normalization method to convert the calibration data set D, the operating parameter set P, and the output result set O into the calibration data set D', the operating parameter set P', and the output result set O'.

[0126] Next, a self-calibration model based on inverse reinforcement learning is established. First, the state space S, the calibration data and operating parameters of the device, and the action space A, the action that changes the device parameters, are defined. Then the optimization goal is determined, which is to maximize the expected cumulative reward under the strategy π. Then, an optimal reward function R(s,a;θ) is reversely learned through maximum entropy inverse reinforcement learning, and θ is iteratively updated to obtain the optimal reward function. Then, based on the optimal reward function, value iteration is used to obtain the final optimal strategy.

[0127] Then, the self-calibration model is trained. First, the initial policy π is set, usually a random policy. Then, a block of simulation data is generated using the current policy π. Parameters are optimized based on these simulation data, and the reward function parameters are updated by the gradient ascent method to obtain a new reward function. Then, the policy is optimized to obtain a new policy. Finally, the new policy is returned to iterate until the policy and reward function converge, and the model training is completed.

[0128] After the model is trained, it can be used to self-calibrate the device. For example, when new observation data, such as the working environment temperature of 28°C and humidity of 55%, is input into the model, the model will predict the optimal calibration action and then apply this action to the device for self-calibration.

[0129] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0130] It should be understood that the detailed description of the technical solutions of the present invention by means of the preferred embodiments is illustrative rather than restrictive. A person skilled in the art may modify the technical solutions described in the embodiments, or replace some of the technical features by equivalents, based on reading the specification of the present invention; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A self-calibration method for a relay tester based on artificial intelligence, characterized in that: The method includes: Basic data collection and preprocessing: In the initial stage of equipment calibration, the calibration data, operating parameters and output results of the equipment are collected, and then these data are preprocessed; Establish a self-calibration model based on inverse reinforcement learning; use inverse reinforcement learning to learn the optimal calibration strategy for equipment calibration tasks; Train a self-calibrated model; Model-driven self-calibration: Use the self-calibration model as a controller to self-calibrate the device; when new observation data is input, the model predicts the optimal control strategy and self-calibrates the device.

2. The method of self-calibration of a relay tester based on artificial intelligence according to claim 1, characterized in that: The basic data collection includes a calibration data set D of the device, an operating parameter set P of the device, and an output result set O of the device; the basic data collection is obtained through the built-in sensor of the device or an external device.

3. The method for self-calibration of a relay tester based on artificial intelligence according to claim 1, characterized in that: The pretreatment comprises: (1) Outlier detection: Use Z-Score to detect and remove outliers in the data set: (2) Data cleaning, using interpolation methods for processing; (3) Data standardization, using the maximum and minimum normalization method, Through the above steps, the processed calibration data set D', the operation parameter set P' and the output result set O' are obtained, preparing for the next step of model building.

4. The method of self-calibration of a relay tester based on artificial intelligence according to claim 1, characterized in that: The self-calibration model includes: (1) Define state space and action space Determine the state space S, i.e., the calibration data and operating parameters of the device, and the action space A, i.e., the actions that change the device parameters; the state s belongs to the state space S, and the action a belongs to the action space A; (2) Determine the optimization goal The goal is to maximize the expected cumulative reward under strategy π, that is, to find the optimal strategy π*, the formula is as follows: E is expectation, is the expected cumulative reward under strategy π, where R(s t ,a t ) is in state s t Next, perform action a t The reward obtained, γ is the decay factor of future rewards, t is the time step, and T is the termination time; (3) Using inverse reinforcement learning to obtain the optimal reward function 1) First, a parameterized reward function R(s, a; θ) is preset, where θ is the parameter of the reward function; 2) Then, through the sampled trajectory information, the maximum entropy inverse reinforcement learning method is used to reversely learn an optimal reward function R(s,a;θ); this is an optimization problem, finding a parameter vector θ that maximizes the following objective function J(θ): In this formula, Indicates the expectation that in state s t Take action t The rewards obtained, and the discounted sum of all accumulated rewards; 3) By iteratively updating θ, the optimal reward function is obtained, and then value iteration is used to obtain the final optimal strategy based on the optimal reward function; Finally, use the optimal policy π^ predicted by the model to apply the best action to the new state and thereby perform self-calibration.

5. The method for self-calibration of a relay tester based on artificial intelligence according to claim 1, characterized in that: The process of training the self-calibration model includes: (1) First, set the initial strategy π, usually a random strategy; and set the initial reward function parameter θ; (2) Generate a block of simulation data using the current policy π, which is achieved by collecting a series of device states s, actions taken a, and device results; (3) Based on the existing simulation data, the parameters of the reward function are optimized according to the objective function of maximum entropy inverse reinforcement learning: Update the reward function parameters by the gradient ascent method, namely: Among them, θ current is the current reward function parameter, is the objective function J with respect to the parameter θ at the current parameter θ current The gradient at, α is the learning rate, θ next is the updated parameter; (4) With the new reward function, we can optimize the strategy. According to the principle of reinforcement learning, we can find a new strategy π. next : p next (a|s)=argmax∑R(s,a;θ next ) (5) Then the new strategy π next As the updated strategy, the next iteration is performed until the strategy π and the reward function R(s,a;θ next ) converges, the model training is completed.

6. The method of self-calibration of a relay tester based on artificial intelligence according to claim 1, characterized in that: The model driven self-calibration includes: The trained IRL model acts as a "controller" to automatically calibrate the relays using the optimal policy predicted based on new observations. (1) Input new observation data: Including the state of the relay, working environment conditions new , and input it into the trained model; (2) Model prediction of optimal calibration action According to the model, for a set of actions a, find the optimal calibration action a bes t, to maximize the expected reward: a best =argmax a ∑[π(a|s new )*R(s new ,a)] Among them, π(a|s new ) indicates that in the current state s new The probability of taking action a, R(s new ,a) means in state s new The reward obtained by executing action a; argmax a It means finding the action a that maximizes the expected reward best ; (3) Self-calibration of the device With the optimal calibration action a best , set and adjust the relay to achieve an optimal working state.

Images