Method for determining on-off state of interconnection switch of distribution line and product

Abstract

The embodiment of the invention provides a method and a product for determining the on-off state of a power distribution line interconnection switch, which are used for solving the problem of unbalanced photovoltaic consumption and load caused by inaccurate determination of the on-off state of the power distribution line interconnection switch in the prior art. According to the embodiment of the invention, the electronic equipment obtains the state data of the power distribution network of each node in a region; wherein the state data of the power distribution network comprises node voltage, line current, load power, photovoltaic power generation power and switch state; inputting the power distribution network state data of each node into a state determination model, and obtaining a target strategy output by the state determination model, the target strategy including a target on-off state of each switch; and controlling the switch intelligent agent to adjust the state of each switch based on the target strategy. According to the embodiment of the invention, the state determination model is adopted to analyze the state data of the power distribution network, thereby accurately determining the target strategy, and based on the target on-off state of each switch in the target strategy, the switch intelligent agent is controlled to adjust the state of each switch based on the target strategy, thereby accurately and effectively adjusting the switches, and improving the switching efficiency. The problems of photovoltaic absorption, load imbalance and the like are effectively solved, efficient operation of the power distribution network is realized, network loss is reduced, voltage quality is improved, and power supply reliability and economical efficiency are improved.

Description

Technical Field

[0001] The present application relates to the technical field of power system distribution network operation, and in particular to a method and product for determining the on / off state of a distribution line tie switch. Background Art

[0002] The access of a high proportion of new energy to the distribution network has become an important direction of energy development, and the proportion of renewable energy power generation represented by photovoltaics connected to the grid continues to increase. Due to the randomness and intermittency of distributed photovoltaic power generation, its large-scale grid connection will cause frequent fluctuations in the distribution network voltage, greatly increase the risk of system voltage exceeding the limit, and increase the operation risk and operating costs of the power system.

[0003] With the large-scale distributed photovoltaic and electric heating loads connected to the power grid during the heating period, the original voltage distribution and power flow distribution of the distribution network have changed. When photovoltaic power generation is concentrated during the day and electric heating loads are concentrated at night, it brings certain pressure to the distribution lines and transformer capacity. Therefore, it is necessary to study how to change the on-off state of the existing distribution line contact switch on the basis of the existing distribution network frame, change the distribution network topology, optimize the distribution network power flow, ensure the heating needs of users and maximize the distributed photovoltaic consumption. Research on the multi-circuit time-sharing multiplexing transformation strategy of the distribution network can effectively solve the problems of photovoltaic consumption and load imbalance, realize the efficient operation of the distribution network, and is of great significance to reducing network losses, improving voltage quality, and improving power supply reliability and economy. Summary of the invention

[0004] The present application provides a method and product for determining the on / off state of a distribution line interconnector switch, so as to solve the problem in the prior art that the on / off state of the distribution line interconnector switch is inaccurately determined, resulting in unbalanced photovoltaic consumption and load.

[0005] In a first aspect, the present application provides a method for determining an on / off state of a distribution line tie switch, the method comprising:

[0006] Obtaining distribution network status data of each node in the area; wherein the distribution network status data includes: voltage, line current, load power, photovoltaic power generation power and switch status;

[0007] Inputting the distribution network state data of each node into a state determination model, and obtaining a target strategy output by the state determination model, wherein the target strategy includes a target on / off state of each switch;

[0008] The switch agent is controlled to adjust the state of each switch based on the target strategy.

[0009] In a possible implementation manner, the state determination model is a Q network.

[0010] In a possible implementation, the state determination model is trained in the following manner:

[0011] Randomly sample target samples from the playback buffer;

[0012] Calculate a target value based on the target network and the next state in the target sample;

[0013] Use the calculated target value to update the parameters of the original state determination model, and optimize the network parameters through the back propagation algorithm so that the predicted value of the state determination model for a given state and action approaches the target value;

[0014] Based on the updated state determination model, for the sampled experience, calculate the difference between the target value and the predicted value output by the original state determination model, and determine the target loss value accordingly;

[0015] The original state determination model is trained based on the target loss value.

[0016] In a possible implementation, calculating a target value based on a target network and a next state in the target sample includes:

[0017] The target value is calculated using the following formula:

[0018] y=r i +γmaxQ target (s t+1 ,a ++1 )

[0019] Among them, r i is the reward value in the target sample, γ is the discount factor, Q target is the target network, s t+1 is the next state in the target sample, a t+1 For state s t+1 Time Q target (s t+1 ,a t+1 )The biggest move.

[0020] In a possible implementation, the samples in the playback buffer are constructed in the following manner:

[0021] In the initial state, the action is selected using the ε-greedy strategy;

[0022] Execute the action in a pre-built environment, and obtain an immediate reward value returned when the environment transitions from a current initial state to a next state;

[0023] A four-tuple consisting of the initial state, the action performed, the reward value returned, and the observed next state is stored as a sample in the replay buffer.

[0024] In a possible implementation, it is characterized in that the objective function of the environment is constructed in the following manner:

[0025] The objective function is determined based on the grid loss, balanced load, distributed photovoltaic consumption level, switch control times and the corresponding weights in the distribution network area connected to the distributed photovoltaic.

[0026] In a possible implementation manner, the grid loss in the distributed photovoltaic distribution network area is confirmed by:

[0027]

[0028] In the formula, f 1 is the sum of the active power loss of the branches throughout the day, ij is the branch connected by node i and node j; X ij is the disconnection state from node i to node j, 1 means closed, 0 means open; N is the set of all lines in the distribution network; T is the total number of time periods throughout the day; r ij is the resistance from node i to node j; t is the time period mark; I ij,t is the current from node i to node j in the tth period; P ij and Q ij are the active and reactive power from node i to node j respectively; is the voltage at the end node of branch ij;

[0029] The balanced load is confirmed by:

[0030]

[0031] Where N is the set of all lines in the distribution network; L ij is the load ratio from node i to node j; P ij and Q ij are the active and reactive power from node i to node j respectively; is the average load rate of the line;

[0032] The distributed photovoltaic consumption level is confirmed by the following methods:

[0033]

[0034] Where n pv is the total number of distributed photovoltaics; T is the total time period; is the predicted value of distributed photovoltaic active power at node i in period t, is the actual active power output value of distributed photovoltaic on node i in period t;

[0035] The switch control times are confirmed by:

[0036]

[0037] In the formula, ij represents the switch ij formed by connecting node i and node j, N b is the total number of branches; H ij (t) is the open and closed state of switch ij during period t, H ij (t-1) is the open and closed state of switch ij in the t-1 period. The open and closed state is usually represented by binary 1 and 0, 1 represents closed and 0 represents open.

[0038] In a possible implementation, the constraints satisfied by the state space of the environment are confirmed in the following manner:

[0039] Power balance constraints:

[0040]

[0041] Among them, u i with u j Represent the node voltages of node i and node j respectively; i ij represents the current of branch ij starting from node i and ending at node j; P ij With Q ij Respectively represent the active and reactive power flowing through branch ij; r ij With x ij Respectively represent the resistance and reactance of branch ij; P j With Q j They represent the net active and reactive loads injected into node j respectively; jk represents the set of branches connected to node j with node j as the starting point and node k as the end point; P jk With Q jk They represent the active and reactive power flowing through branch jk respectively;

[0042] Distributed photovoltaic power generation constraints:

[0043]

[0044] Among them, P pv,n , Q pv,n They represent the distributed photovoltaic active and reactive outputs of photovoltaic user n respectively; Indicates the maximum active output of distributed photovoltaic power generation of user n; and The sub-table indicates the upper and lower limits of distributed photovoltaic reactive power output of user n;

[0045] Node voltage constraints:

[0046] V min ≤V i≤V max

[0047] Where V min and V max are the minimum and maximum values ​​of the node voltage, V i is the voltage value of node i.

[0048] Transformer capacity constraints:

[0049] S up ≤aS rated

[0050] S down ≤aS rated

[0051] In the formula, S rated is the rated capacity of the transformer, S up is the apparent power delivered by the transformer; S down is the apparent power sent by the transformer, and a is the preset proportional value;

[0052] Branch current constraints:

[0053]

[0054] In the formula, I ij (t) is the amplitude of the current flowing through branch ij during period t, I ij，max (t) is the maximum amplitude of the current allowed to flow through branch ij during period t; G ij is the conductance of branch ij, B ij is the susceptance of branch ij; V i (t), V j (t) are the voltage values ​​of node i and node t in period t respectively; θ ij is the phase difference between nodes ij.

[0055] In one possible implementation, the reward mechanism of the environment is confirmed by:

[0056] Node voltage reward:

[0057]

[0058] Where A is the first preset value, U I,min is the voltage lower limit U I，max is the upper voltage limit, I I is the voltage value of node i at time t.

[0059] Switch Change Rewards:

[0060] r 2 =-B+B*p success

[0061] Where B is the second preset value. Each time the network switch conversion action is executed, if the power flow calculation is satisfied, then p success If the power flow calculation is not satisfied, p success is 0.

[0062] In a second aspect, an embodiment of the present application further provides a device for determining an on / off state of a distribution line tie switch, the device comprising:

[0063] An acquisition module is used to acquire the distribution network status data of each node in the area; wherein the distribution network status data includes: voltage, line current, load power, photovoltaic power generation power and switch status;

[0064] A processing module, used for inputting the distribution network state data of each node into a state determination model, and obtaining a target strategy output by the state determination model, wherein the target strategy includes a target on-off state of each switch;

[0065] The control module is used to control the switch agent to adjust the state of each switch based on the target strategy.

[0066] Because in the embodiment of the present application, the electronic device obtains the distribution network status data of each node in the area; wherein the distribution network status data includes: each node voltage, line current, load power, photovoltaic power generation power and switch status; the distribution network status data of each node is input into the state determination model, and the target strategy output by the state determination model is obtained, wherein the target strategy includes the target on-off state of each switch; the switch intelligent body is controlled to adjust the state of each switch based on the target strategy. The embodiment of the present application adopts the state determination model to analyze the distribution network status data, so as to accurately determine the target strategy, and based on the target on-off state of each switch in the target strategy, the switch intelligent body is controlled to adjust the state of each switch based on the target strategy, so as to accurately and effectively adjust the switch, effectively solve the problems of photovoltaic absorption and load imbalance, realize the efficient operation of the distribution network, reduce network loss, improve voltage quality, and improve power supply reliability and economy. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0068] Figure 1 A schematic diagram of a process for determining the on / off state of a distribution line tie switch provided in an embodiment of the present application;

[0069] Figure 2A schematic diagram of a detailed process for determining the on / off state of a distribution line tie switch provided in an embodiment of the present application;

[0070] Figure 3 A schematic diagram of a structure for determining the on / off state of a distribution line tie switch provided in an embodiment of the present application;

[0071] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0073] The following detailed description is an exemplary description, which is intended to provide further detailed description of the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present invention belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present invention.

[0074] In order to accurately and effectively adjust the switches, effectively solve problems such as photovoltaic absorption and load imbalance, achieve efficient operation of the distribution network, reduce network losses, improve voltage quality, and improve power supply reliability and economy, the embodiments of the present application provide a method and product for determining the on-off state of a distribution line interconnection switch.

[0075] In an embodiment of the present application, an electronic device obtains distribution network status data of each node in an area; wherein the distribution network status data includes: voltage of each node, line current, load power, photovoltaic power generation power and switch status; the distribution network status data of each node is input into a state determination model, and a target strategy output by the state determination model is obtained, wherein the target strategy includes a target on-off state of each switch; and the switch intelligent agent is controlled to adjust the state of each switch based on the target strategy.

[0076] Embodiment 1:

[0077] Figure 1 A schematic diagram of a process for determining the on / off state of a distribution line tie switch provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the process includes the following steps:

[0078] S101: Acquire distribution network status data of each node in the area; wherein the distribution network status data includes: voltage, line current, load power, photovoltaic power generation power and switch status.

[0079] The method for determining the on / off state of a distribution line tie switch provided in an embodiment of the present application is applied to an electronic device, which may be an intelligent device such as a PC or a server.

[0080] In order to accurately adjust each switch, the electronic device can first obtain the distribution network status data of each node in the area, wherein the distribution network status data includes voltage, line current, load power, photovoltaic power generation power and switch status. In one possible implementation, the status data of each distribution network can be obtained by deploying a data acquisition (SCADA) system, a smart meter, a distributed energy monitoring system and distribution automation equipment. Specifically, the electronic device obtains the voltage amplitude and phase angle of each node from the SCADA system; obtains the active power and reactive power of each node from the smart meter; obtains the photovoltaic power generation power of each node from the distributed energy monitoring system; obtains the voltage amplitude and phase angle of each branch of the switch status of each branch from the distribution automation equipment.

[0081] It should be noted that after obtaining the distribution network status data, the electronic device can pre-process the obtained distribution network status data, that is, to identify and repair abnormal data, remove invalid data, and fill in missing data. In addition, considering the dynamic changes of the distribution network, the electronic device needs to monitor the distribution network status data in real time. According to actual needs, the distribution network data update frequency is set to once every 10 minutes to achieve real-time data acquisition and update.

[0082] S102: Inputting the distribution network state data of each node into a state determination model, and obtaining a target strategy output by the state determination model, wherein the target strategy includes a target on / off state of each switch.

[0083] In order to determine the target power-on state of each switch, the electronic device locally stores a pre-trained state determination model. After obtaining the distribution network state data, the electronic device can input the distribution network state data of each node into the state determination model and obtain the target strategy output by the state determination model. The state determination model can be a Q network.

[0084] It should be noted that the electronic device can also obtain the distribution network status data, photovoltaic power generation and electric heating load forecast data of each node in the area, and input the distribution network status data, photovoltaic power generation and electric heating load forecast data of each node into the state determination model.

[0085] The target strategy includes the target on / off state of each switch.

[0086] In the embodiment of the present application, an improved deep Q network algorithm can be used to solve the time-division multiplexing transformation problem of the distribution network. Compared with the traditional DQN algorithm, the algorithm changes the action space. The input of the Q network is a state vector s, and the output is a Q value vector corresponding to each action. The data is denoised based on the denoising autoencoder to remove redundant information in the action space.

[0087] And we can add Gaussian noise to the input data of the switch agent. Gaussian noise is a kind of random noise that conforms to the normal distribution, and its probability density function is:

[0088]

[0089] In the formula, σ is the standard deviation; x is a random variable; μ is the mean. The autoencoder is first used for denoising, and the processed data is passed to the DQN for decision-making, so that it has stronger robustness to environmental changes and enhances the generalization ability of the switch agent.

[0090] S103: Control the switch agent to adjust the state of each switch based on the target strategy.

[0091] In an embodiment of the present application, the electronic device is connected to a switch agent. After acquiring the target strategy, the electronic device controls the switch agent to adjust the state of each switch based on the target strategy.

[0092] It should be noted that the state determination model proposed in the embodiment of the present application can be called a distribution network time-sharing multiplexing transformation model. The distribution network time-sharing multiplexing transformation model can use the distributed photovoltaic power generation power and the user's electric heating load power predicted in the future. After solving, the optimized transformation strategy of the distribution network tie line switch can be obtained. The distribution network time-sharing multiplexing transformation model is mainly used to improve the imbalance between voltage quality and line load on the basis of ensuring the heating needs of residents, and to promote the efficient consumption of distributed photovoltaics. According to the characteristics of distributed photovoltaic power generation and electric heating load, the specific application can be divided into the following two scenarios:

[0093] (1) Distributed photovoltaic centralized power generation scenario during the day: During the daytime distributed photovoltaic centralized power generation, a large amount of photovoltaic power is sent to the distribution network through the grid connection point, which is prone to problems such as line reverse flow exceeding the limit and transformer reverse heavy overload. By adopting the time-sharing multiplexing transformation strategy for the distribution network tie line switch, changing the distribution network topology, and then changing the transmission path of photovoltaic power generation, the distribution network flow distribution and voltage distribution are optimized, effectively improving the absorption level of distributed photovoltaic.

[0094] (2) Peak power supply scenario for electric heating load at night: During the winter heating period, the night is the peak period for electric heating load. During peak load, the power line flow exceeds the limit and the transformer is overloaded. In the scenario of heavy load at night and no photovoltaic power generation, the distribution network topology is changed through the time-sharing multiplexing transformation strategy of the distribution network tie line switch, the purpose of multi-circuit power supply evacuation flow is achieved, the distribution network flow distribution and voltage distribution are optimized, and the safety and reliability of residents' heating are guaranteed.

[0095] Because in the embodiment of the present application, the electronic device obtains the distribution network status data of each node in the area; wherein the distribution network status data includes: each node voltage, line current, load power, photovoltaic power generation power and switch status; the distribution network status data of each node is input into the state determination model, and the target strategy output by the state determination model is obtained, wherein the target strategy includes the target on-off state of each switch; the switch intelligent body is controlled to adjust the state of each switch based on the target strategy. The embodiment of the present application adopts the state determination model to analyze the distribution network status data, so as to accurately determine the target strategy, and based on the target on-off state of each switch in the target strategy, the switch intelligent body is controlled to adjust the state of each switch based on the target strategy, so as to accurately and effectively adjust the switch, effectively solve the problems of photovoltaic absorption and load imbalance, realize the efficient operation of the distribution network, reduce network loss, improve voltage quality, and improve power supply reliability and economy.

[0096] Embodiment 2:

[0097] In order to accurately determine the on / off state of each switch, based on the above embodiment, in the embodiment of the present application, the state determination model is trained in the following manner:

[0098] Randomly sample target samples from the playback buffer;

[0099] Calculate a target value based on the target network and the next state in the target sample;

[0100] Use the calculated target value to update the parameters of the original state determination model, and optimize the network parameters through the back propagation algorithm so that the predicted value of the state determination model for a given state and action approaches the target value;

[0101] Based on the updated state determination model, for the sampled experience, calculate the difference between the target value and the predicted value output by the original state determination model, and determine the target loss value accordingly;

[0102] The original state determination model is trained based on the target loss value.

[0103] The replay buffer plays a vital role in the model training process. This buffer stores the experience data generated during the interaction between the agent and the environment. Each experience usually consists of four parts: the current state, the action taken, the reward obtained after executing the action, and the next state. Electronic devices can randomly sample target samples from the replay buffer. The purpose of randomly sampling target samples is to break the temporal correlation between samples, thereby improving the stability and efficiency of learning.

[0104] During training, the electronic device periodically randomly draws a batch of experience samples from the playback buffer. This randomness ensures that the knowledge learned by the model not only depends on recent experience, but also contains a diverse history of past interactions, which helps the model generalize better.

[0105] For the sampled target sample, the electronic device uses the target network and the next state in the target sample to calculate the target value. The target network is a lagged version of the original state determination model used to stabilize the training process.

[0106] The target value is typically calculated by feeding the next state into the target network, obtaining a state value estimate or action value estimate, and adding the expected future reward that may be obtained from the next state to the terminal state.

[0107] The electronic device uses the calculated target value to optimize the parameters of the original state determination model through the back propagation algorithm (a gradient descent algorithm). The goal of the optimization is to make the predicted value of the original state determination model under a given state and action as close to the target value as possible. In this way, the state determination model can learn which action to take in a specific state to obtain the greatest long-term reward.

[0108] In each iteration, the electronic device determines the model based on the updated state, and for each set of experiences sampled from the playback buffer, the algorithm calculates the difference between the target value and the predicted value of the model's current output. This difference (usually through some form of loss function such as mean squared error) measures the model's predictive performance and serves as an important metric in the training process.

[0109] Finally, based on the calculated target loss value, the electronic device trains the original state determination model. This usually involves using an optimizer (such as Adam or RMSprop) to update the weights of the model to minimize the loss value. By continuously iterating this process, the model can gradually learn better strategies to make better decisions in a given environment.

[0110] The entire process is a closed-loop feedback system. By continuously learning from the environment and adjusting itself, the intelligent agent can gradually improve its performance to better adapt to and respond to complex environmental challenges.

[0111] Among them, before training, the state set s of the current environment can be obtained i , action set a i , the iteration step size is α, the exploration rate is ε, and the Q network, target network and playback buffer can be initialized.

[0112] Embodiment 3:

[0113] In order to determine the target value, based on the above embodiments, in the embodiment of the present application, the target value is calculated based on the target network and the next state in the target sample, including:

[0114] The target value is calculated using the following formula:

[0115] y=r i +γmaxQ target (s t+1 ,a t+1 )

[0116] Among them, r i is the reward value in the target sample, γ is the discount factor, Q target is the target network, s t+1 is the next state in the target sample, a t+1 For state s t+1 Time Q target (s t+1 ,a t+1 )The biggest move.

[0117] Among them, the action space is the adjustable variable of the tie-line switch agent, which is used to describe the specific actions of the time-sharing multiplexing transformation of the distribution network switch. The action space of the distribution network switch transformation is the key to determining how to optimize the operation of the distribution network. The action space defines the specific operations taken by the switch agent, usually including the opening and closing states of the section switch and the tie switch. The action space of the distribution network tie-line switch transformation is represented by a set of vectors a i To effectively express, the dimension is n; a i Indicates the operating state of the i-th switch, 1 means closed and 0 means open.

[0118] In calculating the target value, the electronic device uses a specific formula that is a common method used in reinforcement learning to estimate future cumulative rewards, especially in Q-value-based learning methods. This formula aims to evaluate the value of a state-action pair by the weighted sum of the current reward and the potential future reward.

[0119] The formula is:

[0120] y=r i +γmax Q target (s t+1,a t+1 )

[0121] The detailed explanation is as follows:

[0122] r i Represents the immediate reward obtained from the environment, that is, in the current state s t , perform an action, a t After that, the environment gives immediate feedback. In actual scenarios, i Can be any real number representing positive or negative feedback from the environment.

[0123] γ is a discount factor, a constant between 0 and 1, which is used to adjust the importance of future rewards to the current state-action pair value assessment. The closer γ is to 1, the greater the impact of future rewards on the current value; the closer γ is to 0, the smaller the impact of future rewards on the current value. This parameter helps the algorithm to make a trade-off between "immediate satisfaction" and "long-term planning". target is the Q-value function of the target network, which is a lagged version of the original Q-value network and is used to stabilize the training process and prevent training instability due to rapid changes in Q-values. The target network updates its weights regularly or slowly to track the weight changes of the original network.

[0124] s t+1 is the next state in the target sample, that is, in the current state s t Execute action a t After that, the environment transitions to a new state. t+1 This is in state s t+1 Under this condition, the strategy selected by some method (such as greedy strategy) can make Q target (s t+1 ,a t+1 ) to obtain the maximum action. This selection process reflects the agent's ability to predict and plan future behavior. maxQ target (s t+1 ,a t+1 ) calculates the next state s t+1 Next, by traversing all possible actions a t+1 , find the action that maximizes the target network Q value and calculate the corresponding Q value. This represents the maximum expected future reward that can be obtained through the optimal action given the next state.

[0125] The formula calculates the target value y by combining the immediate reward and the discounted maximum expected future reward, which is used as a target or benchmark for updating the original Q-value network during training. In this way, it is possible to learn how to choose the best action in a given state to maximize its cumulative reward.

[0126] Embodiment 4:

[0127] In order to accurately train the state determination model, based on the above embodiments, in the embodiment of the present application, the samples in the playback buffer are constructed in the following manner:

[0128] In the initial state, the action is selected using the ε-greedy strategy;

[0129] Execute the action in a pre-built environment, and obtain an immediate reward value returned when the environment transitions from a current initial state to a next state;

[0130] A four-tuple consisting of the initial state, the action performed, the reward value returned, and the observed next state is stored as a sample in the replay buffer.

[0131] In an embodiment of the present application, the electronic device may adopt an ε-greedy strategy in the initial state to select an action to be performed. The ε-greedy strategy is a strategy that strikes a balance between exploration and exploitation. Specifically, the electronic device randomly selects an action with a probability of ε, and selects the action currently considered optimal with a probability of 1-ε. Here, ε is a parameter between 0 and 1, which can be adjusted according to the learning progress.

[0132] After selecting an action, the electronic device performs the action in a pre-built environment. The environment transfers state according to certain rules based on the current state and the selected action, and returns an immediate reward value to the electronic device. This reward value is a direct feedback on the effect of the electronic device performing the selected action in the current state, which can be positive (indicating a reward), negative (indicating a penalty) or zero (indicating no effect).

[0133] The agent stores the experience data of this interaction in the replay buffer in the form of a 4-tuple. This 4-tuple includes:

[0134] The initial state, the action performed, the reward value returned, and the next state the environment transitions to after the action is performed.

[0135] This process will be repeated continuously, and as the electronic device continues to interact with the environment, a large number of experience samples will gradually accumulate in the playback buffer. These samples will then be used to train the original state determination model, and the parameters of the model will be optimized through the back propagation algorithm, so that the intelligent agent can learn how to choose the best action in different states to maximize the cumulative reward.

[0136] In short, the construction process of the replay buffer embodies the idea of ​​trial-and-error learning and experience replay in reinforcement learning. By continuously accumulating and utilizing experience data, the intelligent agent can gradually improve its decision-making ability in complex environments.

[0137] Embodiment 5:

[0138] In order to construct the environment, based on the above embodiments, in the embodiment of the present application, the objective function of the environment is constructed in the following manner:

[0139] The objective function is determined based on the grid loss, balanced load, distributed photovoltaic consumption level, switch control times and the corresponding weights in the distribution network area connected to the distributed photovoltaic.

[0140] When constructing the objective function for optimizing the operation of a distribution network area involving access to distributed photovoltaics, electronic equipment needs to comprehensively consider multiple key indicators and their corresponding weights to ensure the economy, reliability, and sustainability of the system.

[0141] Among them, the network loss in the distribution network area connected to distributed photovoltaic refers to the power loss caused by resistance, reactance and other factors during the transmission and distribution of electricity. Reducing network loss is an important goal to improve the efficiency of the distribution network and reduce energy waste. In the objective function, the network loss can be obtained by calculating the current, voltage, impedance and other parameters on each line, and it is minimized as an important cost item.

[0142] Load balancing aims to reduce the load imbalance between nodes, feeders or transformers in the distribution network to avoid adverse conditions such as overload and underload, and improve the stability and reliability of system operation. By optimizing the output plan of distributed photovoltaics, the charging and discharging strategy of the energy storage system, and the switching operation of the distribution network, the load can be evenly distributed in different times and regions. In the objective function, load rate balancing indicators such as load rate standard deviation or coefficient of variation can be introduced to evaluate the load balancing level and strive to minimize it.

[0143] The level of distributed photovoltaic absorption refers to the degree to which distributed photovoltaic power generation within the distribution network area is effectively utilized. Improving the absorption level will help improve the utilization rate of renewable energy and reduce the phenomenon of abandoned photovoltaic power generation. By optimizing the grid connection strategy of distributed photovoltaic power generation, the configuration of energy storage systems, and demand-side response measures, the local absorption capacity of photovoltaic power generation can be improved. In the objective function, the local absorption ratio of photovoltaic power generation or the abandonment rate can be set as the optimization target to strive to improve its absorption level.

[0144] Frequent operation of switches in the distribution network (such as tie switches, section switches, etc.) will not only increase equipment wear, but may also affect power supply reliability. Therefore, the number of unnecessary switch operations should be minimized during the optimization process. When constructing the objective function, the cost term of the number of switch controls can be introduced and minimized. At the same time, the constraints of switch operations, such as switch action time interval restrictions and switch state transition logic, are considered in the optimization algorithm to ensure the feasibility and practicality of the optimization results.

[0145] The allocation of weights can be considered comprehensively based on the specific needs of the distribution network, policy orientation, economic benefits and other factors. In the objective function, various cost items (such as network loss cost, load imbalance cost, abandoned light cost, switch operation cost, etc.) will be multiplied by their respective weights and summed to form the final objective function. By adjusting the value of the weight, it is possible to achieve emphasis and balance on different optimization goals.

[0146] Embodiment 6:

[0147] In order to determine the objective function, based on the above embodiments, in the embodiment of the present application, the grid loss in the distribution network area of ​​the distributed photovoltaic is confirmed by the following method:

[0148]

[0149] In the formula, f 1 is the sum of the active power loss of the branches throughout the day, ij is the branch connected by node i and node j; X ij is the disconnection state from node i to node j, 1 means closed, 0 means open; N is the set of all lines in the distribution network; T is the total number of time periods throughout the day; r ij is the resistance from node i to node j; t is the time period mark; I ij,t is the current from node i to node j in the tth period; P ij and Q ij are the active and reactive power from node i to node j respectively; is the voltage at the end node of branch ij;

[0150] The balanced load is confirmed by:

[0151]

[0152] Where N is the set of all lines in the distribution network; L ij is the load ratio from node i to node j; P ij and Q ij are the active and reactive power from node i to node j respectively; is the average load rate of the line;

[0153] The distributed photovoltaic consumption level is confirmed by the following methods:

[0154]

[0155] Where n pv is the total number of distributed photovoltaics; T is the total time period; is the predicted value of distributed photovoltaic active power at node i in period t, is the actual active power output value of distributed photovoltaic on node i in period t;

[0156] The switch control times are confirmed by:

[0157]

[0158] In the formula, ij represents the switch ij formed by connecting node i and node j, N b is the total number of branches; H ij (t) is the open and closed state of switch ij during period t, H ij (t-1) is the open and closed state of switch ij in the t-1 period. The open and closed state is usually represented by binary 1 and 0, 1 represents closed and 0 represents open.

[0159] Embodiment 7:

[0160] In order to construct the environment, based on the above embodiments, in the embodiment of the present application, the constraints satisfied by the state space of the environment are confirmed in the following manner:

[0161] Power balance constraints:

[0162]

[0163] Among them, u i with u j Represent the node voltages of node i and node j respectively; i ij represents the current of branch ij starting from node i and ending at node j; P ij With Q ij Respectively represent the active and reactive power flowing through branch ij; r ij With x ij Respectively represent the resistance and reactance of branch ij; P j With Q j They represent the net active and reactive loads injected into node j respectively; jk represents the set of branches connected to node j with node j as the starting point and node k as the end point; P jk With Q jk They represent the active and reactive power flowing through branch jk respectively;

[0164] Distributed photovoltaic power generation constraints:

[0165]

[0166] Among them, P pv,n , Q pv,n They represent the distributed photovoltaic active and reactive outputs of photovoltaic user n respectively; Indicates the maximum active output of distributed photovoltaic power generation of user n; and The sub-table indicates the upper and lower limits of distributed photovoltaic reactive power output of user n;

[0167] Node voltage constraints:

[0168] V min ≤V i ≤V max

[0169] Where V min and V max are the minimum and maximum values ​​of the node voltage, V i is the voltage value of node i.

[0170] Transformer capacity constraints:

[0171] S up ≤aS rated

[0172] S down ≤aS rated

[0173] In the formula, S rated is the rated capacity of the transformer, S up is the apparent power delivered by the transformer; S down is the apparent power sent by the transformer, and a is the preset proportional value;

[0174] In one possible implementation, in order to ensure that the transformer does not operate under heavy load during the centralized power supply of electric heating loads at night and the distributed photovoltaic centralized power generation during the day, the transformer's upstream and downstream power is limited to no more than 80% of the transformer's rated capacity, so the preset ratio value can be 80%.

[0175] Branch current constraints:

[0176]

[0177] In the formula, I ij (t) is the amplitude of the current flowing through branch ij during period t, I ij，max (t) is the maximum amplitude of the current allowed to flow through branch ij during period t; G ij is the conductance of branch ij, B ij is the susceptance of branch ij; V i (t), V j (t) are the voltage values ​​of node i and node t in period t respectively; θ ij is the phase difference between nodes ij.

[0178] Among them, branch current constraints can ensure that the distribution network branch current does not exceed the limit when the electric heating load is centrally supplied at night and the distributed photovoltaic power generation is centralized during the day.

[0179] It should be noted that the state space is one of the key steps for the intelligent agent to make decisions. The state space represents the key information of the distribution network at a certain moment and should meet the above constraints.

[0180] Embodiment 8:

[0181] In order to determine the reward value, based on the above embodiments, in the embodiment of the present application, the reward mechanism of the environment is confirmed by the following method:

[0182] Node voltage reward:

[0183]

[0184] Where A is the first preset value, U I,min is the voltage lower limit U I，max is the upper voltage limit, U I is the voltage value of node i at time t.

[0185] Switch Change Rewards:

[0186] r 2 =-B+B*p success

[0187] Where B is the second preset value. Each time the network switch conversion action is executed, if the power flow calculation is satisfied, then p success If the power flow calculation is not satisfied, p success is 0.

[0188] The basic framework of reinforcement learning in this application includes five parts: agent, environment, state, action and reward. It is a kind of learning that maps from environmental state to action, and the goal is to enable the agent to obtain the maximum cumulative reward in the process of interaction with the environment. In order to apply reinforcement learning to the efficient consumption of distributed photovoltaics, this application constructs a reinforcement learning environment model suitable for network topology optimization based on the topological characteristics of the distribution network.

[0189] In the embodiment of the present application, the electronic device may determine the reward value based on the node voltage reward and the switch change reward.

[0190] This application proposes a multi-circuit time-sharing multiplexing transformation strategy for distribution networks based on reinforcement learning. First, a mathematical modeling of the multi-circuit time-sharing multiplexing transformation problem of distribution networks is carried out: the network loss is minimized, the load is balanced, the distributed photovoltaic absorption level and the number of switch control are taken as the objective functions, and the constraints of the distribution network are established; secondly, an environment simulating the distribution network is constructed based on deep reinforcement learning, including the state space, action space and reward mechanism; finally, an improved deep Q network is used to train and solve the model, and finally the transformation strategy of the distribution network line switch is obtained to guide the time-sharing multiplexing transformation of the distribution network tie line switch.

[0191] Figure 2 A schematic diagram of a detailed process for determining the on / off state of a distribution line tie switch provided in an embodiment of the present application.

[0192] Depend on Figure 2 It can be seen that the electronic device can obtain the distribution network state data, photovoltaic power generation and electric heating load forecast data of each node in the area, input the distribution network state data, photovoltaic power generation and electric heating load forecast data of each node into the state determination model, obtain the target strategy output by the state determination model, and control the switch agent to adjust the state of each switch based on the target strategy. Among them, the state determination model is a pre-constructed distribution network time-sharing service transformation reinforcement learning environment, and the improved DQN algorithm is used to train the model.

[0193] The embodiment of the present application proposes a distribution network multi-circuit time-division multiplexing transformation strategy based on reinforcement learning, which has the following advantages:

[0194] (1) The deployed distribution network status detection equipment can monitor the status of the distribution network in real time and input the real-time acquired data into the deep reinforcement learning model after processing. Deep reinforcement learning can train the switch intelligent agent to make decisions quickly and realize the real-time optimization of the time-division multiplexing transformation of the distribution network lines based on the decisions.

[0195] (2) A time-sharing multiplexing transformation model for the distribution network is constructed based on the reinforcement learning Markov decision process. The decision-making process can obtain high-quality switching strategies for the distribution network lines through continuous search and trial in a dynamic environment, thereby improving the reliability and economy of the distribution network operation after the access of distributed photovoltaic and electric heating loads.

[0196] (3) Aiming at the research problem of time-sharing multiplexing transformation strategy of distribution network based on reinforcement learning, this patent adopts an improved deep Q network algorithm to solve it. This algorithm reduces the dimension of the action space based on the original deep Q network and introduces noise into the input sample features to learn more robust data encoding. This makes it unnecessary to sample the on and off states of all switches during the training process, avoiding local optimality and improving data utilization efficiency.

[0197] Embodiment 9:

[0198] Figure 3 A schematic diagram of the structure of a device for determining the on / off state of a distribution line tie switch provided in an embodiment of the present application, the device comprising:

[0199] The acquisition module 301 is used to acquire the distribution network status data of each node in the area; wherein the distribution network status data includes: voltage, line current, load power, photovoltaic power generation power and switch status;

[0200] The processing module 302 is used to input the distribution network state data of each node into a state determination model, and obtain a target strategy output by the state determination model, wherein the target strategy includes a target on-off state of each switch;

[0201] The control module 303 is used to control the switch agent to adjust the state of each switch based on the target strategy.

[0202] In a possible implementation manner, the processing module 302 is further configured to train the state determination model in the following manner:

[0203] Randomly sample target samples from the playback buffer;

[0204] Calculate a target value based on the target network and the next state in the target sample;

[0205] Use the calculated target value to update the parameters of the original state determination model, and optimize the network parameters through the back propagation algorithm so that the predicted value of the state determination model for a given state and action approaches the target value;

[0206] Based on the updated state determination model, for the sampled experience, calculate the difference between the target value and the predicted value output by the original state determination model, and determine the target loss value accordingly;

[0207] The original state determination model is trained based on the target loss value.

[0208] In a possible implementation, the processing module 302 is further configured to calculate the target value using the following formula:

[0209] y=r i +γmaxQ target (s t+1 ,a t+1 )

[0210] Among them, r i is the reward value in the target sample, γ is the discount factor, Q target is the target network, s t+1 is the next state in the target sample, a t+1 For state s t+1 Time Q target (s t+1 ,a t+1 )The biggest move.

[0211] In a possible implementation manner, the processing module 302 is further configured to construct the samples in the playback buffer in the following manner:

[0212] In the initial state, the action is selected using the ε-greedy strategy;

[0213] Execute the action in a pre-built environment, and obtain an immediate reward value returned when the environment transitions from a current initial state to a next state;

[0214] A four-tuple consisting of the initial state, the action performed, the reward value returned, and the observed next state is stored as a sample in the replay buffer.

[0215] In a possible implementation, the processing module 302 is further configured to construct the objective function of the environment in the following manner:

[0216] The objective function is determined based on the grid loss, balanced load, distributed photovoltaic consumption level, switch control times and the corresponding weights in the distribution network area connected to the distributed photovoltaic.

[0217] In a possible implementation manner, the processing module 302 is further configured to confirm the regional grid loss of the distributed photovoltaic distribution network by:

[0218]

[0219] In the formula, f 1 is the sum of the active power loss of the branches throughout the day, ij is the branch connected by node i and node j; X ij is the disconnection state from node i to node j, 1 means closed, 0 means open; N is the set of all lines in the distribution network; T is the total number of time periods throughout the day; r ij is the resistance from node i to node j; t is the time period mark; I ij,t is the current from node i to node j in the tth period; P ij and Q ij are the active and reactive power from node i to node j respectively; is the voltage at the end node of branch ij;

[0220] The balanced load is confirmed by:

[0221]

[0222] Where N is the set of all lines in the distribution network; L ij is the load ratio from node i to node j; P ij and Q ij are the active and reactive power from node i to node j respectively; is the average load rate of the line;

[0223] The distributed photovoltaic consumption level is confirmed by the following methods:

[0224]

[0225] Where n pv is the total number of distributed photovoltaics; T is the total time period; is the predicted value of distributed photovoltaic active power at node i in period t, is the actual active power output value of distributed photovoltaic on node i in period t;

[0226] The switch control times are confirmed by:

[0227]

[0228] In the formula, ij represents the switch ij formed by connecting node i and node j, N b is the total number of branches; H ij (t) is the open and closed state of switch ij during period t, H ij (t-1) is the open and closed state of switch ij in the t-1 period. The open and closed state is usually represented by binary 1 and 0, 1 represents closed and 0 represents open.

[0229] In a possible implementation manner, the processing module 302 is further configured to confirm the constraint condition satisfied by the state space of the environment in the following manner:

[0230] Power balance constraints:

[0231]

[0232]

[0233] Among them, u i with u j Represent the node voltages of node i and node j respectively; i ij represents the current of branch ij starting from node i and ending at node j; P ij With Q ij Respectively represent the active and reactive power flowing through branch ij; r ij With x ij Respectively represent the resistance and reactance of branch ij; P j With Q j They represent the net active and reactive loads injected into node j respectively; jk represents the set of branches connected to node j with node j as the starting point and node k as the end point; P jk With Q jk They represent the active and reactive power flowing through branch jk respectively;

[0234] Distributed photovoltaic power generation constraints:

[0235]

[0236] Among them, P pv,n , Q pv,n They represent the distributed photovoltaic active and reactive outputs of photovoltaic user n respectively; Indicates the maximum active output of distributed photovoltaic power generation of user n; and The sub-table indicates the upper and lower limits of distributed photovoltaic reactive power output of user n;

[0237] Node voltage constraints:

[0238] V min ≤V i ≤V max

[0239] Where V min and V max are the minimum and maximum values ​​of the node voltage, V i is the voltage value of node i.

[0240] Transformer capacity constraints:

[0241] S up ≤aS rated

[0242] S down ≤aS rated

[0243] In the formula, S rated is the rated capacity of the transformer, S up is the apparent power delivered by the transformer; S down is the apparent power sent by the transformer, and a is the preset proportional value;

[0244] Branch current constraints:

[0245]

[0246] In the formula, I ij (t) is the amplitude of the current flowing through branch ij during period t, I ij，max (t) is the maximum amplitude of the current allowed to flow through branch ij during period t; G ij is the conductance of branch ij, B ij is the susceptance of branch ij; V i (t), V j (t) are the voltage values ​​of node i and node t in period t respectively; θ ij is the phase difference between nodes ij.

[0247] In a possible implementation, the processing module 302 is further configured to confirm the reward mechanism of the environment in the following manner:

[0248] Node voltage reward:

[0249]

[0250] Where A is the first preset value, U I,min is the voltage lower limit U I，max is the upper voltage limit, U I is the voltage value of node i at time t.

[0251] Switch Change Rewards:

[0252] r 2 =-B+B*p success

[0253] Where B is the second preset value. Each time the network switch conversion action is executed, if the power flow calculation is satisfied, then p success If the power flow calculation is not satisfied, p success is 0.

[0254] Embodiment 7:

[0255] Figure 4 The present invention provides a schematic diagram of an electronic device structure according to an embodiment of the present invention. Based on the above embodiments, the present invention further provides an electronic device, such as Figure 4 As shown, it includes: a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404;

[0256] The memory 403 stores a computer program. When the program is executed by the processor 401, the processor 401 performs the following steps:

[0257] Obtaining distribution network status data of each node in the area; wherein the distribution network status data includes: voltage, line current, load power, photovoltaic power generation power and switch status;

[0258] Inputting the distribution network state data of each node into a state determination model, and obtaining a target strategy output by the state determination model, wherein the target strategy includes a target on / off state of each switch;

[0259] The switch agent is controlled to adjust the state of each switch based on the target strategy.

[0260] In a possible implementation manner, the state determination model is a Q network.

[0261] In a possible implementation, the state determination model is trained in the following manner:

[0262] Randomly sample target samples from the playback buffer;

[0263] Calculate a target value based on the target network and the next state in the target sample;

[0264] Use the calculated target value to update the parameters of the original state determination model, and optimize the network parameters through the back propagation algorithm so that the predicted value of the state determination model for a given state and action approaches the target value;

[0265] Based on the updated state determination model, for the sampled experience, calculate the difference between the target value and the predicted value output by the original state determination model, and determine the target loss value accordingly;

[0266] The original state determination model is trained based on the target loss value.

[0267] In a possible implementation, calculating a target value based on a target network and a next state in the target sample includes:

[0268] The target value is calculated using the following formula:

[0269] y=r i +γnax Q target (s t+1 ,a t+1 )

[0270] Among them, r i is the reward value in the target sample, γ is the discount factor, Q target is the target network, s t+1 is the next state in the target sample, a t+1 For state s t+1 Time Q target (s t+1 ,a t+1 )The biggest move.

[0271] In a possible implementation, the samples in the playback buffer are constructed in the following manner:

[0272] In the initial state, the action is selected using the ε-greedy strategy;

[0273] Execute the action in a pre-built environment, and obtain an immediate reward value returned when the environment transitions from a current initial state to a next state;

[0274] A four-tuple consisting of the initial state, the action performed, the reward value returned, and the observed next state is stored as a sample in the replay buffer.

[0275] In a possible implementation, it is characterized in that the objective function of the environment is constructed in the following manner:

[0276] The objective function is determined based on the grid loss, balanced load, distributed photovoltaic consumption level, switch control times and the corresponding weights in the distribution network area connected to the distributed photovoltaic.

[0277] In a possible implementation manner, the grid loss in the distributed photovoltaic distribution network area is confirmed by:

[0278]

[0279] In the formula, f 1 is the sum of the active power loss of the branches throughout the day, ij is the branch connected by node i and node j; X ij is the disconnection state from node i to node j, 1 means closed, 0 means open; N is the set of all lines in the distribution network; T is the total number of time periods throughout the day; r ij is the resistance from node i to node j; t is the time period mark; I ij,t is the current from node i to node j in the tth period; P ij and Q ij are the active and reactive power from node i to node j respectively; is the voltage at the end node of branch ij;

[0280] The balanced load is confirmed by:

[0281]

[0282]

[0283] Where N is the set of all lines in the distribution network; L ij is the load ratio from node i to node j; P ij and Q ij are the active and reactive power from node i to node j respectively; is the average load rate of the line;

[0284] The distributed photovoltaic consumption level is confirmed by the following methods:

[0285]

[0286] Where n pv is the total number of distributed photovoltaics; T is the total time period; is the predicted value of distributed photovoltaic active power at node i in period t, is the actual active power output value of distributed photovoltaic on node i in period t;

[0287] The switch control times are confirmed by:

[0288]

[0289] In the formula, ij represents the switch ij formed by connecting node i and node j, N b is the total number of branches; H ij (t) is the open and closed state of switch ij during period t, H ij (t-1) is the open and closed state of switch ij in the t-1 period. The open and closed state is usually represented by binary 1 and 0, 1 represents closed and 0 represents open.

[0290] In a possible implementation, the constraints satisfied by the state space of the environment are confirmed in the following manner:

[0291] Power balance constraints:

[0292]

[0293] Among them, u i with u j Represent the node voltages of node i and node j respectively; i ij represents the current of branch ij starting from node i and ending at node j; P ij With Q ij Respectively represent the active and reactive power flowing through branch ij; r ij With x ij Respectively represent the resistance and reactance of branch ij; P j With Q j They represent the net active and reactive loads injected into node j respectively; jk represents the set of branches connected to node j with node j as the starting point and node k as the end point; P jk With Q jk They represent the active and reactive power flowing through branch jk respectively;

[0294] Distributed photovoltaic power generation constraints:

[0295]

[0296] Among them, P pv,n , Q pv,n They represent the distributed photovoltaic active and reactive outputs of photovoltaic user n respectively; Indicates the maximum active output of distributed photovoltaic power generation of user n; and The sub-table indicates the upper and lower limits of distributed photovoltaic reactive power output of user n;

[0297] Node voltage constraints:

[0298] Vmin ≤V i ≤V max

[0299] Where V min and V max are the minimum and maximum values ​​of the node voltage, V i is the voltage value of node i.

[0300] Transformer capacity constraints:

[0301] S up ≤aS rated

[0302] S down ≤aS rated

[0303] In the formula, S rated is the rated capacity of the transformer, S up is the apparent power delivered by the transformer; S down is the apparent power sent by the transformer, and a is the preset proportional value;

[0304] Branch current constraints:

[0305]

[0306] In the formula, I ij (t) is the amplitude of the current flowing through branch ij during period t, I ij，max (t) is the maximum amplitude of the current allowed to flow through branch ij during period t; G ij is the conductance of branch ij, B ij is the susceptance of branch ij; V i (t), V j (t) are the voltage values ​​of node i and node t in period t respectively; θ ij is the phase difference between nodes ij.

[0307] In one possible implementation, the reward mechanism of the environment is confirmed by:

[0308] Node voltage reward:

[0309]

[0310] Where A is the first preset value, U I,min is the voltage lower limit U I，max is the upper voltage limit, U I is the voltage value of node i at time t.

[0311] Switch Change Rewards:

[0312] r2 =-B+B*p success

[0313] Where B is the second preset value. Each time the network switch conversion action is executed, if the power flow calculation is satisfied, then p success If the power flow calculation is not satisfied, p success is 0.

[0314] The communication bus mentioned in the above server can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0315] The communication interface is used for communication between the above electronic device and other devices.

[0316] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0317] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0318] Embodiment 8:

[0319] On the basis of the above embodiments, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by an electronic device, and when the program is run on the electronic device, the electronic device implements the following steps when executing:

[0320] Obtaining distribution network status data of each node in the area; wherein the distribution network status data includes: voltage, line current, load power, photovoltaic power generation power and switch status;

[0321] Inputting the distribution network state data of each node into a state determination model, and obtaining a target strategy output by the state determination model, wherein the target strategy includes a target on / off state of each switch;

[0322] The switch agent is controlled to adjust the state of each switch based on the target strategy.

[0323] In a possible implementation manner, the state determination model is a Q network.

[0324] In a possible implementation, the state determination model is trained in the following manner:

[0325] Randomly sample target samples from the playback buffer;

[0326] Calculate a target value based on the target network and the next state in the target sample;

[0327] Use the calculated target value to update the parameters of the original state determination model, and optimize the network parameters through the back propagation algorithm so that the predicted value of the state determination model for a given state and action approaches the target value;

[0328] Based on the updated state determination model, for the sampled experience, calculate the difference between the target value and the predicted value output by the original state determination model, and determine the target loss value accordingly;

[0329] The original state determination model is trained based on the target loss value.

[0330] In a possible implementation, calculating a target value based on a target network and a next state in the target sample includes:

[0331] The target value is calculated using the following formula:

[0332] y=r i +ymax Q target (s t+1 ,a t+1 )

[0333] Among them, r i is the reward value in the target sample, γ is the discount factor, Q target is the target network, s t+1 is the next state in the target sample, a t+1 For state s t+1 Time Q target (s t+1 ,a t+1 )The biggest move.

[0334] In a possible implementation, the samples in the playback buffer are constructed in the following manner:

[0335] In the initial state, the action is selected using the ε-greedy strategy;

[0336] Execute the action in a pre-built environment, and obtain an immediate reward value returned when the environment transitions from a current initial state to a next state;

[0337] A four-tuple consisting of the initial state, the action performed, the reward value returned, and the observed next state is stored as a sample in the replay buffer.

[0338] In a possible implementation, it is characterized in that the objective function of the environment is constructed in the following manner:

[0339] The objective function is determined based on the grid loss, balanced load, distributed photovoltaic consumption level, switch control times and the corresponding weights in the distribution network area connected to the distributed photovoltaic.

[0340] In a possible implementation manner, the grid loss in the distributed photovoltaic distribution network area is confirmed by:

[0341]

[0342] In the formula, f 1 is the sum of the active power loss of the branches throughout the day, ij is the branch connected by node i and node j; X ij is the disconnection state from node i to node j, 1 means closed, 0 means open; N is the set of all lines in the distribution network; T is the total number of time periods throughout the day; r ij is the resistance from node i to node j; t is the time period mark; I ij,t is the current from node i to node j in the tth period; P ij and Q ij are the active and reactive power from node i to node j respectively; is the voltage at the end node of branch ij;

[0343] The balanced load is confirmed by:

[0344]

[0345] Where N is the set of all lines in the distribution network; L ij is the load ratio from node i to node j; P ij and Q ij are the active and reactive power from node i to node j respectively; is the average load rate of the line;

[0346] The distributed photovoltaic consumption level is confirmed by the following methods:

[0347]

[0348] Where n pv is the total number of distributed photovoltaics; T is the total time period; is the predicted value of distributed photovoltaic active power at node i in period t, is the actual active power output value of distributed photovoltaic on node i in period t;

[0349] The switch control times are confirmed by:

[0350]

[0351] In the formula, ij represents the switch ij formed by connecting node i and node j, N b is the total number of branches; H ij (t) is the open and closed state of switch ij during period t, H ij (t-1) is the open and closed state of switch ij in the t-1 period. The open and closed state is usually represented by binary 1 and 0, 1 represents closed and 0 represents open.

[0352] In a possible implementation, the constraints satisfied by the state space of the environment are confirmed in the following manner:

[0353] Power balance constraints:

[0354]

[0355]

[0356] Among them, u i with u j Represent the node voltages of node i and node j respectively; i ij represents the current of branch ij starting from node i and ending at node j; P ij With Q ij Respectively represent the active and reactive power flowing through branch ij; r ij With x ij Respectively represent the resistance and reactance of branch ij; P j With Q j They represent the net active and reactive loads injected into node j respectively; jk represents the set of branches connected to node j with node j as the starting point and node k as the end point; P jk With Q jk They represent the active and reactive power flowing through branch jk respectively;

[0357] Distributed photovoltaic power generation constraints:

[0358]

[0359] Among them, P pv,n , Q pv,n They represent the distributed photovoltaic active and reactive outputs of photovoltaic user n respectively; Indicates the maximum active output of distributed photovoltaic power generation of user n; and The sub-table indicates the upper and lower limits of distributed photovoltaic reactive power output of user n;

[0360] Node voltage constraints:

[0361] V min ≤V i ≤V max

[0362] Where V min and V max are the minimum and maximum values ​​of the node voltage, V i is the voltage value of node i.

[0363] Transformer capacity constraints:

[0364] S up ≤aS rated

[0365] S down ≤aS rated

[0366] In the formula, S rated is the rated capacity of the transformer, S up is the apparent power delivered by the transformer; S down is the apparent power sent by the transformer, and a is the preset proportional value;

[0367] Branch current constraints:

[0368]

[0369]

[0370] In the formula, I ij (t) is the amplitude of the current flowing through branch ij during period t, I ij，max (t) is the maximum amplitude of the current allowed to flow through branch ij during period t; G ij is the conductance of branch ij, B ij is the susceptance of branch ij; V i (t), V j (t) are the voltage values ​​of node i and node t in period t respectively; θ ij is the phase difference between nodes ij.

[0371] In one possible implementation, the reward mechanism of the environment is confirmed by:

[0372] Node voltage reward:

[0373]

[0374] Where A is the first preset value, U I,min is the voltage lower limit U I，max is the upper voltage limit, I I is the voltage value of node i at time t.

[0375] Switch Change Rewards:

[0376] r 2 =-B+B*p success

[0377] Where B is the second preset value. Each time the network switch conversion action is executed, if the power flow calculation is satisfied, then p success If the power flow calculation is not satisfied, p success is 0.

[0378] It is known from common technical knowledge that the present application can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present application or within the scope equivalent to the present application are included in the present application.

[0379] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0380] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0381] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0382] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0383] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present application can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present application should be included in the scope of protection of the claims of the present application.

Claims

1. A method for determining the on / off state of a distribution line tie switch, characterized in that: The method comprises: Obtaining distribution network status data of each node in the area; wherein the distribution network status data includes: voltage, line current, load power, photovoltaic power generation power and switch status; Inputting the distribution network state data of each node into a state determination model, and obtaining a target strategy output by the state determination model, wherein the target strategy includes a target on / off state of each switch; The switch agent is controlled to adjust the state of each switch based on the target strategy.

2. The method according to claim 1, characterized in that The state determination model is a Q network.

3. The method according to claim 1 or 2, characterized in that: The state determination model is trained in the following way: Randomly sample target samples from the playback buffer; Calculate a target value based on the target network and the next state in the target sample; Use the calculated target value to update the parameters of the original state determination model, and optimize the network parameters through the back propagation algorithm so that the predicted value of the state determination model for a given state and action approaches the target value; Based on the updated state determination model, for the sampled experience, calculate the difference between the target value and the predicted value output by the original state determination model, and determine the target loss value accordingly; The original state determination model is trained based on the target loss value.

4. The method according to claim 3, characterized in that Calculating a target value based on the target network and the next state in the target sample includes: The target value is calculated using the following formula: y=r i +γmaxQ target (s t+1 ,to t+1 ) Among them, r i is the reward value in the target sample, γ is the discount factor, Q targct is the target network, s t+1 is the next state in the target sample, a t+1 For state s t+1 Time Q target (s t+1 ,a t+1 )The biggest move.

5. The method according to claim 3, characterized in that: The samples in the playback buffer are constructed in the following way: In the initial state, the action is selected using the ε-greedy strategy; Execute the action in a pre-built environment, and obtain an immediate reward value returned when the environment transitions from a current initial state to a next state; A four-tuple consisting of the initial state, the action performed, the reward value returned, and the observed next state is stored as a sample in the replay buffer.

6. The method according to claim 5, characterized in that The objective function of the environment is constructed in the following way: The objective function is determined based on the grid loss, balanced load, distributed photovoltaic consumption level, switch control times and the corresponding weights in the distribution network area connected to the distributed photovoltaic.

7. The method according to claim 6, characterized in that The grid loss in the distribution network area of ​​the distributed photovoltaic is confirmed by the following methods: Where f1 is the sum of the active power loss of the branches throughout the day, ij is the branch connected to node i and node j; X ij is the disconnection state from node i to node j, 1 means closed, 0 means open; N is the set of all lines in the distribution network; T is the total number of time periods throughout the day; r ij is the resistance from node i to node j; t is the time period mark; I ij,t is the current from node i to node j in the tth period; P ij and Q ij are the active and reactive power from node i to node j respectively; is the voltage at the end node of branch ij; The balanced load is confirmed by: Where N is the set of all lines in the distribution network; L ij is the load ratio from node i to node j; P ij and Q ij are the active and reactive power from node i to node j respectively; is the average load rate of the line; The distributed photovoltaic consumption level is confirmed by the following methods: Where n pv is the total number of distributed photovoltaics; T is the total time period; is the predicted value of distributed photovoltaic active power at node i in period t, is the actual active power output value of distributed photovoltaic on node i in period t; The switch control times are confirmed by: In the formula, ij represents the switch ij formed by connecting node i and node j, N b is the total number of branches; H ij (t) is the open and closed state of switch ij during period t, H ij (t-1) is the open and closed state of switch ij in time period t-1. The open and closed state is usually represented by binary 1 and 0, 1 represents closed and 0 represents open.

8. The method according to claim 6, characterized in that The constraints satisfied by the state space of the environment are confirmed by: Power balance constraints: Among them, u i with u j Represent the node voltages of node i and node j respectively; i ij represents the current of branch ij starting from node i and ending at node j; P ij With Q ij Respectively represent the active and reactive power flowing through branch ij; r ij With x ij Respectively represent the resistance and reactance of branch ij; P j With Q j They represent the net active and reactive loads injected into node j respectively; jk represents the set of branches connected to node j with node j as the starting point and node k as the end point; P jk With Q jk They represent the active and reactive power flowing through branch jk respectively; Distributed photovoltaic power generation constraints: Among them, P pv,n , Q pv,n They represent the distributed photovoltaic active and reactive outputs of photovoltaic user n respectively; Indicates the maximum active output of distributed photovoltaic power generation of user n; and The sub-table indicates the upper and lower limits of distributed photovoltaic reactive power output of user n; Node voltage constraints: In min ≤V i ≤V max Where V min and V max are the minimum and maximum values ​​of the node voltage, V i is the voltage value of node i; Transformer capacity constraints: S up ≤aS rated S down ≤aS rated In the formula, S rated is the rated capacity of the transformer, S up is the apparent power delivered by the transformer; S down is the apparent power sent by the transformer, and a is the preset proportional value; Branch current constraints: In the formula, I ij (t) is the amplitude of the current flowing through branch ij during period t, I ij，max (t) is the maximum amplitude of the current allowed to flow through branch ij during period t; G ij is the conductance of branch ij, B ij is the susceptance of branch ij; V i (t), V j (t) are the voltage values ​​of node i and node t in period t respectively; θ ij is the phase difference between nodes ij.

9. The method according to claim 6, characterized in that The reward mechanism of the environment is confirmed by: Node voltage reward: Where A is the first preset value, U I,min is the voltage lower limit U I，max is the upper voltage limit, I I is the voltage value of node i at time t; Switch Change Rewards: r2=-B+B*p success Where B is the second preset value. Each time the network switch conversion action is executed, if the power flow calculation is satisfied, then p success If the power flow calculation is not satisfied, p success is 0.

10. A product for determining the on / off status of a distribution line tie switch, characterized in that: The products include: An acquisition module is used to acquire the distribution network status data of each node in the area; wherein the distribution network status data includes: voltage, line current, load power, photovoltaic power generation power and switch status; A processing module, used for inputting the distribution network state data of each node into a state determination model, and obtaining a target strategy output by the state determination model, wherein the target strategy includes a target on-off state of each switch; The control module is used to control the switch agent to adjust the state of each switch based on the target strategy.

Images