Intelligent substation with automatic control system

Abstract

The invention belongs to the technical field of substation power dispatching, and relates to an intelligent substation with an automatic control system, which comprises a data acquisition layer, a communication layer, a decision control layer and an equipment execution layer, the data acquisition layer is used for acquiring line state parameters of the operation state of the transformer substation, operation actions of load equipment and power characteristics; the decision control layer is used for analyzing the collected data; modeling a control strategy of the operation state of the transformer substation by using a Markov decision process, learning an optimal control strategy by training a deep neural network algorithm, and generating a control instruction of corresponding load equipment; the equipment execution layer comprises a circuit breaker, a transformer tap switch and reactive compensation equipment; and executing state operation of the corresponding load equipment according to the control instruction. The system can give an alarm in time and carry out corresponding maintenance and emergency operation; and reactive power balance and voltage stability of the transformer substation are optimized.

Description

Technical Field

[0001] The present invention belongs to the technical field of substation power dispatching, and in particular, relates to an intelligent substation with an automatic control system. Background Art

[0002] In order to cope with the increasingly complex power grid environment and the growing demand for electricity, especially the large-scale access of renewable energy and the widespread distribution of distributed power generation resources, the reactive power and voltage deviation characteristics of the power system have been significantly affected. Reactive power and voltage deviation are not only related to the stability and power quality of the power system, but also directly affect the transmission efficiency and economy of the power grid. The existing power grid dispatching strategy seems to be unable to cope with these new characteristics, and it is urgent to explore new dispatching strategies to adapt to this change; the automation control system of smart substations came into being. However, the traditional power dispatching scheme has the following shortcomings, which limits its application in smart substations: 1. Traditional systems are usually dispatched based on fixed models and historical data, which makes it difficult to accurately cope with the complex and changing power grid environment. This may lead to uneven dispatching and distribution of power resources and inability to achieve optimal reactive power balance and voltage stability control.

[0003] Second: Traditional substations often experience various fault events during the power dispatching process. The fault detection method often relies on manual experience and complex data analysis, which makes it difficult to accurately locate the fault point, affecting the efficiency of fault handling. Summary of the invention

[0004] In order to solve the technical problem that traditional substations are difficult to accurately cope with complex and changeable power grid environments, resulting in uneven scheduling and distribution of power and energy resources, the present invention provides a smart substation with an automatic control system; an automatic optimization control strategy based on Markov decision process (MDP) and deep neural network (DQN) algorithm.

[0005] The purpose of the present invention can be achieved through the following technical solutions: An intelligent substation with an automated control system includes a data acquisition layer, a communication layer, a decision control layer, and an equipment execution layer; The data collection layer is used to collect line status parameters, load equipment operation actions and power characteristics of the substation operation status; Preferably, the line state parameters include current, voltage, active power and reactive power of each node of the transmission line; The load equipment operation action includes the switch state of the circuit breaker of the load equipment, the position of the transformer tap and the operation state of the reactive power compensation equipment; The power characteristics include voltage deviation and reactive power deviation.

[0006] The communication layer is used for data transmission, uploading the collected operating status data to the decision control layer, and sending control instructions to the device execution layer; and sending control instructions to the device execution layer.

[0007] The decision control layer is used to analyze the collected operating status data, use the Markov decision process to model the control strategy of the substation operating status, and learn the optimal control strategy by training the deep neural network algorithm, and generate control instructions for the corresponding load equipment to optimize the reactive power balance and voltage stability of the substation; The equipment execution layer includes circuit breakers, transformer tap changers and reactive power compensation equipment; it is used to execute the state operation of the corresponding load equipment according to the control instructions; Preferably, the specific implementation process of the decision control layer includes: S1) MDP modeling: The grid dispatch problem of the substation is modeled as a Markov decision process (MDP), defining the state space, action space and reward function; Preferably, the state space includes the transmission line voltage, transmission line reactive power, transmission line active power and voltage deviation of the substation; The action space includes operation control of the compensation capacitor bank, transformer tap adjustment, transformer voltage ratio adjustment and power output control of the reactive power compensation device (SVC); The reward function includes a positive reward for reactive power balance, a positive reward for voltage stability, and a negative reward for an operating state where the reactive power deviation or the voltage deviation exceeds a threshold.

[0008] Preferably, the reward function provides a positive reward for the target state through the balance of reactive power, encouraging the system to reach or approach a state of reactive power balance; the specific calculation method includes: Define reactive power deviation: Calculate the reactive power deviation ΔQ of the substation, which is the difference between the load line power demand of the substation and the current transmission line power; Reward design: The reward value is set according to the reactive power deviation value ΔQ; the smaller the deviation, the greater the reward; the reward function of reactive power is expressed as: ; In the formula, is the maximum positive reward; is the adjustment coefficient; It is the reactive power deviation value.

[0009] Preferably, the reward function provides a positive reward for the target state through voltage stability, encouraging the system to maintain voltage stability at each node; the specific calculation method includes: Define voltage deviation: Calculate the voltage deviation ΔV of each line node in the system, that is, the difference between the load line voltage of the substation and the target voltage; Reward design: Set the reward value according to the voltage deviation ΔV; when the node voltage is close to the target value, a higher reward is given; the smaller the deviation, the greater the reward; the reward function of the voltage deviation is expressed as: ; In the formula, is the maximum positive reward; is the adjustment coefficient; is the voltage deviation value.

[0010] Preferably, the reward function calculates the overall reward by integrating reactive power balance, voltage stability and negative reward conditions; Among them, if the reactive power deviation value ΔQ or the voltage deviation ΔV exceeds the set threshold, a negative reward is given; the overall reward function is expressed as: ; In the formula, , The weight of each reward, the weight parameter is set according to the operation priority; is the positive reward value for reactive power deviation; is the positive reward value for voltage deviation; Negative reward for reactive power deviation or voltage deviation.

[0011] S2) DQN algorithm construction: Build a fully connected neural network (DQN) algorithm, and bring the reward function after MDP modeling into the Q value function in the DQN algorithm. The Q value function represents the expected cumulative reward for taking a certain action in a given state; Among them, the input layer receives the current state vector, and the number of state vectors is used as the number of neurons; Multiple fully connected layers process the input state vector and extract vector features; the number of fully connected layers is the number of actions; The output layer outputs the Q value of each possible action; S3) Training process: Through MDP modeling, samples of states, actions, rewards, and next states are collected and stored in the experience replay pool; Initialize the DQN algorithm parameters, regularly extract samples from the experience replay pool, and use these samples to train the DQN algorithm; In the specific implementation process, Get the current state s, execute action a, and get the reward R; Get the next state s' and select action a' according to the ε-greedy strategy; Use the Bellman equation to update the Q value to maximize the expected cumulative reward strategy; at the same time, update the DQN algorithm parameters.

[0012] Preferably, the calculation formula for updating the Q value of the Bellman equation is expressed as: ; In the formula, is the expected cumulative reward for taking an action in a given state; is the total reward of the current action, that is, the immediate reward of the current action a; is the discount factor, which controls the impact of future rewards; is the Q value of the optimal action a′ in the next state s′, which represents the strategy to maximize the expected reward.

[0013] S4) Application deployment: After the training is completed, the DQN algorithm is deployed to the actual power grid dispatching system, and the new state vector is input into the DQN algorithm to obtain the Q value of each action; the action with the largest Q value is selected as the control instruction.

[0014] Preferably, the control instruction is at least one action in the action space in MDP modeling.

[0015] Preferably, the step S3) further includes: DQN algorithm parameter optimization: Use mean square error (MSE) as the loss function to measure the gap between the expected Q value and the target Q value; optimize the DQN algorithm parameters through back propagation algorithm and gradient descent.

[0016] Preferably, the decision control layer is also used to identify faults and quickly locate power anomalies in substations; the specific implementation process includes: Collect historical data of the substation system in different operating states (different power loads or active power), and calculate the Q value distribution of the corresponding actions; The weighted average method is used to analyze the Q value distribution of each action and set a normal fluctuation threshold.

[0017] When new state data is input into the DQN algorithm, the Q value of each action is calculated and compared with the normal fluctuation threshold. If the Q value of an action exceeds its normal fluctuation threshold, it is marked as abnormal. Further analyze the associated actions and load equipment of the Q value abnormality, and then accurately locate the power abnormality point.

[0018] Beneficial effects of the present invention: 1. The control strategy of the substation operating status is modeled using the Markov decision process. The state-action mapping of MDP can help the system associate different operating states with corresponding control strategies. At the same time, the optimal control strategy is learned by training the deep neural network (DQN) algorithm, and the control instructions of the corresponding load equipment are generated to optimize the reactive power balance and voltage stability of the substation.

[0019] 2. The DQN algorithm introduces an experience replay mechanism, which uses random sampling to enhance the robustness of the learning process and avoid overfitting. In addition, the ε-greedy strategy strikes a balance between exploration and utilization, which helps improve training efficiency and makes the system more adaptable in practical applications.

[0020] 3. This solution sets a reasonable threshold range by analyzing the Q value distribution of different actions, and then after the new state is input, it determines in real time whether there is an action Q value that exceeds the normal range, thereby accurately locating the abnormal point. This not only improves the speed of fault detection, but also reduces the false alarm rate, making the detection results more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0022] Figure 1 This is a schematic diagram of the framework structure of an intelligent substation with an automatic control system according to the present invention; Figure 2 The present invention provides an algorithm flow chart of the decision-making control layer in a smart substation with an automated control system. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0024] See also Figure 1-Figure 2 As shown, a smart substation with an automatic control system includes a data acquisition layer, a communication layer, a decision control layer and an equipment execution layer; The data collection layer is used to collect line status parameters, load equipment operation actions and power characteristics of the substation operation status; Further, the line state parameters include current, voltage, active power and reactive power of each node of the transmission line; The load equipment operation action includes the switch state of the circuit breaker of the load equipment, the position of the transformer tap and the operation state of the reactive power compensation equipment; The power characteristics include voltage deviation and reactive power deviation.

[0025] Specifically, a smart substation includes transmission lines, load equipment, and load lines (supplying power to users) from a power station; among them, the load equipment is a means of regulating and controlling the dispatching and distribution of electric energy in the substation, which is crucial to achieving reactive power balance and voltage stability.

[0026] The data acquisition layer of the present invention serves as the "perception layer" of the entire system and is used to obtain various operating data of the substation in real time. The voltage transformer (VT) is deployed at the nodes of the transmission line to measure the transmission voltage; the current transformer (CT) measures the transmission current. At the same time, the voltage transformer (VT) is deployed at the nodes of the load line to measure the load voltage; the current transformer (CT) measures the load current.

[0027] Status indicators are deployed on load devices to collect the operating status of each load device. Among them, the main function of the circuit breaker is to control the current path of the transmission line in the power system, execute the opening and closing status of the circuit breaker, and ensure the reactive power balance of the system when the power demand changes.

[0028] The adjustment of the transformer tap position directly affects the transformer output voltage regulation capability, thereby ensuring the stability of the grid voltage.

[0029] Reactive power compensation equipment (such as static VAR compensation devices (SVCs), capacitors, transformers, etc.) is used to balance reactive power and stabilize voltage in power systems.

[0030] The programmable logic controller (PLC) compiles data acquisition tasks, collects data from various measuring devices in real time, and preliminarily processes the aggregated data to calculate the reactive power deviation and voltage deviation of each node.

[0031] The reactive power deviation is the difference between the transmission line power and the current load line power, which can be calculated by measuring the voltage and current at the same time. The voltage deviation is the difference between the current load line voltage and the target voltage after the transformation.

[0032] The power characteristics include: Reactive power balance: The balance of reactive power difference is also used to reasonably dispatch and allocate actual electricity demand, and the effect is more intuitive.

[0033] Voltage deviation: Voltage deviation refers to the difference between the voltage of a node in the power system and the rated voltage. Excessive voltage deviation will affect the normal operation of power equipment and even cause equipment damage.

[0034] As a more preferred embodiment, the power characteristics may also include line overload: line overload means that the current flowing through the power line exceeds its rated current. Line overload will cause the line to heat up or even burn the line, causing a power outage. Therefore, the power characteristics also need to collect temperature data of relevant nodes (load circuits, transformers and other load equipment) as a basis.

[0035] The communication layer is used for data transmission, uploading the collected operating status data to the decision control layer, and sending control instructions to the device execution layer; and sending control instructions to the device execution layer.

[0036] Specifically, the IEC 61850 or Modbus communication protocol is used. The EC 61850 protocol is particularly suitable for the communication needs of the power system, supports data modeling and equipment management, and helps to improve the reliability and efficiency of transmission. In terms of network transmission, it can be transmitted through industrial Ethernet or wireless sensor network (WSN) technology to ensure the reliability and efficiency of data transmission. Industrial Ethernet is suitable for high bandwidth and low latency requirements, while WSN is more flexible in remote or difficult to wire areas. By combining the two, seamless connection of substations is achieved.

[0037] The decision control layer is used to analyze the collected operating status data, use the Markov decision process to model the control strategy of the substation operating status, and learn the optimal control strategy by training the deep neural network algorithm, and generate control instructions for the corresponding load equipment to optimize the reactive power balance and voltage stability of the substation; The equipment execution layer includes circuit breakers, transformer tap changers and reactive power compensation equipment; it is used to execute the state operation of the corresponding load equipment according to the control instructions; Furthermore, the specific implementation process of the decision control layer includes: S1) MDP modeling: The grid dispatch problem of the substation is modeled as a Markov decision process (MDP), defining the state space, action space and reward function; Further, the state space includes the transmission line voltage, transmission line reactive power, transmission line active power and voltage deviation of the substation; The action space includes operation control of the compensation capacitor bank, transformer tap adjustment, transformer voltage ratio adjustment and power output control of the reactive power compensation device (SVC); The reward function includes a positive reward for reactive power balance, a positive reward for voltage stability, and a negative reward for an operating state where the reactive power deviation or the voltage deviation exceeds a threshold.

[0038] Specifically, MDP modeling (Markov decision process modeling) is used to transform the power grid scheduling problem into a state-action-reward framework so that the decision control layer can optimize the control strategy of the power system through a deep neural network algorithm.

[0039] First, according to the power dispatching requirements of the substation, the state space of the system is defined, which mainly includes the following parameters: Voltage (V): The voltage value of the transmission line, used to determine voltage stability.

[0040] Reactive power (Q): The reactive power of the transmission line, used for reactive balance monitoring.

[0041] Active power (P): The active power of the transmission line, used to determine the load line.

[0042] Assuming that the state space S includes the above-mentioned eigenvectors, the state space can be expressed as S=[V,Q,P], where t represents the current time.

[0043] Next, in each state, the action space that the system can perform includes the following: Adjust reactive power compensation status (C): By connecting, increasing or decreasing the input power of reactive power compensation equipment (such as SVC, capacitors, transformers, etc.) to meet the reactive power balance requirements of the power grid.

[0044] Adjust the tap position of the transformer (T): Change the tap position to adjust the voltage level.

[0045] Control circuit breaker status (K): Open or close the circuit breaker to adjust the distribution of power or isolate the faulty line.

[0046] Assuming that the action space a contains these operation options, the action space can be expressed as a=[C,T,K], which represents reactive compensation control, tap adjustment, and circuit breaker control, respectively.

[0047] Finally, the reward function reflects the effect of the dispatch action. If the adjusted power system achieves reactive power balance or the voltage deviation is maintained within a reasonable range, the reward function gives a positive value, indicating that the operation is effective. If the parameter exceeds the limit (such as reactive power or voltage deviation exceeds the threshold), a negative reward is given.

[0048] Furthermore, the reward function provides positive rewards for the target state through the balance of reactive power, encouraging the system to reach or approach a state of reactive power balance; the specific calculation method includes: Define reactive power deviation: Calculate the reactive power deviation ΔQ of the substation, which is the difference between the load line power demand of the substation and the current transmission line power; Reward design: The reward value is set according to the reactive power deviation value ΔQ; the smaller the deviation, the greater the reward; the reward function of reactive power is expressed as: ; In the formula, is the maximum positive reward; is the adjustment coefficient; It is the reactive power deviation value.

[0049] Furthermore, the reward function provides positive rewards for the target state through voltage stability, encouraging the system to maintain voltage stability at each node; the specific calculation method includes: Define voltage deviation: Calculate the voltage deviation ΔV of each line node in the system, that is, the difference between the load line voltage of the substation and the target voltage; Reward design: Set the reward value according to the voltage deviation ΔV; when the node voltage is close to the target value, a higher reward is given; the smaller the deviation, the greater the reward; the reward function of the voltage deviation is expressed as: ; In the formula, is the maximum positive reward; is the adjustment coefficient; is the voltage deviation value.

[0050] Furthermore, the reward function comprehensively calculates the overall reward through reactive power balance, voltage stability and negative reward conditions; Among them, if the reactive power deviation value ΔQ or the voltage deviation ΔV exceeds the set threshold, a negative reward is given; the overall reward function is expressed as: ; In the formula, , The weight of each reward, the weight parameter is set according to the operation priority; is the positive reward value for reactive power deviation; is the positive reward value for voltage deviation; Negative reward for reactive power deviation or voltage deviation.

[0051] S2) DQN algorithm construction: Build a fully connected neural network (DQN) algorithm, and bring the reward function after MDP modeling into the Q value function in the DQN algorithm. The Q value function represents the expected cumulative reward for taking a certain action in a given state; Among them, the input layer receives the current state vector, and the number of state vectors is used as the number of neurons; Multiple fully connected layers process the input state vector and extract vector features; the number of fully connected layers is the number of actions; The output layer outputs the Q value of each possible action; Specifically, DQN is a reinforcement learning algorithm that estimates the Q-value function through a fully connected neural network. It can predict the long-term cumulative reward for each possible action in a given state and then determine the best action.

[0052] In the process of building the DQN algorithm, the input layer is used to receive the state vector S of the current system, where the state vector S contains 5 characteristic parameters, and the number of neurons in the input layer is 5.

[0053] Multiple fully connected layers (usually 2-3 layers) are used to extract features from the state vector and implement nonlinear feature mapping to better learn the complex relationship between state and action. An activation function (such as ReLU) can be applied between each fully connected layer to introduce nonlinearity so that the network can express complex control strategies. The action space contains at least 3 scheduling operations: 1. Adjustment of reactive power compensation equipment: Increase the output rated increment of reactive power equipment (for example, from 20Mvar to 30Mvar).

[0054] 2. Adjust the transformer tap position: increase the tap from position 2 to position 3.

[0055] 3. Circuit breaker status control: disconnect a certain transmission line to reduce the reactive power of the transmission line.

[0056] The output layer will have at least 3 nodes, each of which outputs a Q value, which represents the expected cumulative reward for performing each action in the current state.

[0057] S3) Training process: Through MDP modeling, samples of states, actions, rewards, and next states are collected and stored in the experience replay pool; Initialize the DQN algorithm parameters, regularly extract samples from the experience replay pool, and use these samples to train the DQN algorithm; In the specific implementation process, Get the current state s, execute action a, and get the reward R; Get the next state s' and select action a' according to the ε-greedy strategy; Use the Bellman equation to update the Q value to maximize the expected cumulative reward strategy; at the same time, update the DQN algorithm parameters.

[0058] Furthermore, the calculation formula for updating the Q value of the Bellman equation is expressed as: ; In the formula, is the expected cumulative reward for taking an action in a given state; is the total reward of the current action, that is, the immediate reward of the current action a; is the discount factor, which controls the impact of future rewards; is the Q value of the optimal action a′ in the next state s′, which represents the strategy to maximize the expected reward.

[0059] S4) Application deployment: After the training is completed, the DQN algorithm is deployed to the actual power grid dispatching system, and the new state vector is input into the DQN algorithm to obtain the Q value of each action; the action with the largest Q value is selected as the control instruction.

[0060] Specifically, the decision-making control layer is the "core" of the system. The key lies in using intelligent scheduling algorithms, combining Markov decision process (MDP) and deep neural network (DQN) algorithms, to maintain the reactive power balance and voltage stability of the system under changing load line power supply requirements and environments. The specific implementation process is as follows: MDP modeling: The power dispatch problem of the substation is modeled as a Markov decision process, and the state space, action space and reward function are defined. The state space includes the state information of the transmission line and compensation equipment, the action space includes the reactive compensation equipment and transformer adjustment operation, and the reward function reflects the reactive power balance and voltage stability requirements of the system.

[0061] Deep learning training: Use the DQN algorithm (specifically a fully connected neural network) to approximate the Q-value function. The DQN algorithm gradually learns the optimal control strategy based on the action performance under different states. Here, experience replay and the Bellman equation are used to continuously optimize the Q-value function, so that the optimal control action can be accurately evaluated and selected under complex power grid conditions.

[0062] In the initial stage of training, the weight parameters of the DQN network are random values. DQN starts to interact with the system environment through these initial parameters, continuously optimizes its parameters, and finally learns the optimal scheduling strategy.

[0063] In each training iteration, an ε-greedy strategy is also used to balance exploration and exploitation during the training process: that is, an action is randomly selected with probability ϵ (exploration) to discover potential better strategies. The action with the maximum Q value in the current state is selected with probability (1−ϵ) (exploitation) to execute the known optimal strategy.

[0064] Control instruction generation and optimization: The optimal action in the current state is evaluated through the Q value function, and the action is converted into a control instruction and sent to the device execution layer to ensure the long-term stability and reliability of the control strategy.

[0065] Furthermore, the control instruction is at least one action in the action space in MDP modeling.

[0066] Furthermore, the step S3) further includes: DQN algorithm parameter optimization: Use mean square error (MSE) as the loss function to measure the gap between the expected Q value and the target Q value; optimize the DQN algorithm parameters through back propagation algorithm and gradient descent.

[0067] Furthermore, the decision control layer is also used to identify faults and quickly locate power anomalies in substations; the specific implementation process includes: Collect historical data of the substation system in different operating states (different power loads or active power), and calculate the Q value distribution of the corresponding actions; The weighted average method is used to analyze the Q value distribution of each action and set a normal fluctuation threshold.

[0068] When new state data is input into the DQN algorithm, the Q value of each action is calculated and compared with the normal fluctuation threshold. If the Q value of an action exceeds its normal fluctuation threshold, it is marked as abnormal. Further analyze the associated actions and load equipment of the Q value abnormality, and then accurately locate the power abnormality point.

[0069] Specifically, various historical data are collected under different operating conditions of the substation (such as peak load, valley load or different active power levels). These data include data of substation equipment in different states, and record the Q value of each action in these states. Specifically, the Q value distribution of each action (such as transformer tap position adjustment, reactive compensation equipment start and stop, etc.) will obtain different distribution curves according to different historical load and power conditions. These curves can reflect the normal Q value range of each action in different states.

[0070] The Q value distribution is statistically analyzed, and the normal range of the Q value of each action is obtained by weighted average method, and then a normal fluctuation threshold is set. Usually, the Q value of an action is distributed between [10,15] in most states, and [10,15] can be defined as the normal fluctuation range of the action. Q value fluctuations within this range are considered normal.

[0071] The Q value of each action output by the DQN algorithm is then compared with the normal fluctuation threshold set previously. If the Q value of an action exceeds its normal range, it is considered that the device corresponding to the action may be abnormal.

[0072] When an abnormal Q value is detected, the system will further analyze the specific affected objects of the action, such as the associated load equipment, transformers, etc. By tracing the action corresponding to the abnormal Q value and its affected area, the system can accurately locate the specific power abnormal point. For example, if the Q value of a reactive compensation device is abnormal after the output power changes, it may indicate that the device is faulty or the current reactive compensation is insufficient. By locating the abnormal point, the system can issue an alarm in time and perform corresponding maintenance and emergency operations.

[0073] In the description of the specification, the method or process can be stored in a computer-readable storage medium if it is implemented in the form of a software program. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program codes.

[0074] In the description of the specification, the description with reference to the terms "particularly", "further", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0075] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A smart substation with an automated control system, characterized in that: Includes data collection layer, communication layer, decision control layer and equipment execution layer; The data collection layer is used to collect line status parameters, load equipment operation actions and power characteristics of the substation operation status; The communication layer is used for data transmission, uploading the collected operating status data to the decision control layer, and sending control instructions to the equipment execution layer; The decision control layer is used to analyze the collected operating status data, use the Markov decision process to model the control strategy of the substation operating status, and learn the optimal control strategy by training the deep neural network algorithm, and generate control instructions for the corresponding load equipment; The specific implementation process of the decision control layer includes: S1) MDP modeling: The grid dispatch problem of the substation is modeled as a Markov decision process, defining the state space, action space and reward function; S2) DQN algorithm construction: Build a fully connected neural network algorithm and bring the reward function after MDP modeling into the Q value function in the DQN algorithm. The Q value function represents the expected cumulative reward for taking a certain action in a given state. Among them, the input layer receives the current state vector, and the number of state vectors is used as the number of neurons; Multiple fully connected layers process the input state vector and extract vector features; the number of fully connected layers is the number of actions; The output layer outputs the Q value of each possible action; S3) Training process: Through MDP modeling, samples of states, actions, rewards, and next states are collected and stored in the experience replay pool; Initialize the DQN algorithm parameters, regularly extract samples from the experience replay pool, and use these samples to train the DQN algorithm; Get the current state s, execute action a, and get the reward R; Get the next state s' and select action a' according to the ε-greedy strategy; Use the Bellman equation to update the Q value to maximize the expected cumulative reward strategy; at the same time, update the DQN algorithm parameters; S4) Application deployment: After training is completed, the DQN algorithm is deployed to the actual power grid dispatching system, and the new state vector is input into the DQN algorithm to obtain the Q value of each action; the action with the largest Q value is selected as the control instruction; The equipment execution layer includes circuit breakers, transformer tap changers and reactive power compensation equipment; it is used to execute the status operation of the corresponding load equipment according to the control instructions.

2. The intelligent substation with an automatic control system according to claim 1 is characterized in that: The line state parameters include current, voltage, active power and reactive power of each node of the transmission line; The load equipment operation action includes the switch state of the circuit breaker of the load equipment, the position of the transformer tap and the operation state of the reactive power compensation equipment; The power characteristics include voltage deviation and reactive power deviation.

3. The intelligent substation with an automatic control system according to claim 1 is characterized in that: In process S1), the state space includes the transmission line voltage, transmission line reactive power, transmission line active power and voltage deviation of the substation; The action space includes operation control of the compensation capacitor bank, transformer tap adjustment, transformer voltage ratio adjustment and power output control of the reactive compensation equipment; The reward function includes a positive reward for reactive power balance, a positive reward for voltage stability, and a negative reward for an operating state where the reactive power deviation or the voltage deviation exceeds a threshold.

4. The intelligent substation with an automatic control system according to claim 3 is characterized in that: The reward function provides positive rewards for the target state through the balance of reactive power, encouraging the system to reach or approach a state of reactive power balance. The specific calculation method includes: Define reactive power deviation: Calculate the reactive power deviation ΔQ of the substation, which is the difference between the load line power demand of the substation and the current transmission line power; Reward design: The reward value is set according to the reactive power deviation value ΔQ; the smaller the deviation, the greater the reward; the reward function of reactive power is expressed as: ； In the formula, is the maximum positive reward; is the adjustment coefficient; It is the reactive power deviation value.

5. The intelligent substation with an automatic control system according to claim 3 is characterized in that: The reward function provides positive rewards for the target state through voltage stability, encouraging the system to maintain voltage stability at each node; the specific calculation method includes: Define voltage deviation: Calculate the voltage deviation ΔV of each line node in the system, that is, the difference between the load line voltage of the substation and the target voltage; Reward design: Set the reward value according to the voltage deviation ΔV; when the node voltage is close to the target value, a higher reward is given; the smaller the deviation, the greater the reward; the reward function of the voltage deviation is expressed as: ； In the formula, is the maximum positive reward; is the adjustment coefficient; is the voltage deviation value.

6. The intelligent substation with an automatic control system according to claim 3 is characterized in that: The reward function calculates the overall reward by combining reactive power balance, voltage stability and negative reward conditions; Among them, if the reactive power deviation value ΔQ or the voltage deviation ΔV exceeds the set threshold, a negative reward is given; the overall reward function is expressed as: ； In the formula, , The weight of each reward, the weight parameter is set according to the operation priority; is the positive reward value for reactive power deviation; is the positive reward value for voltage deviation; Negative reward for reactive power deviation or voltage deviation.

7. The intelligent substation with an automatic control system according to claim 1, characterized in that: The calculation formula for updating the Q value of the Bellman equation is expressed as: ； In the formula, is the expected cumulative reward for taking an action in a given state; is the total reward of the current action, that is, the immediate reward of the current action a; is the discount factor, which controls the impact of future rewards; is the Q value of the optimal action a′ in the next state s′, which represents the strategy to maximize the expected reward.

8. The intelligent substation with an automatic control system according to claim 1, characterized in that: Process S3) also includes: DQN algorithm parameter optimization: Use mean square error as the loss function to measure the gap between the expected Q value and the target Q value; optimize the DQN algorithm parameters through back propagation algorithm and gradient descent.

9. A smart substation with an automatic control system according to any one of claims 1 to 8, characterized in that: The decision control layer is also used to identify faults and quickly locate power anomalies in substations; the specific implementation process includes: Collect historical data of the substation system in different operating states and calculate the Q value distribution of the corresponding actions; The weighted average method is used to analyze the Q value distribution of each action and set a normal fluctuation threshold; When new state data is input into the DQN algorithm, the Q value of each action is calculated and compared with the normal fluctuation threshold. If the Q value of an action exceeds its normal fluctuation threshold, it is marked as abnormal. Further analyze the associated actions and load equipment of the Q value abnormality, and then accurately locate the power abnormality point.

Images