Fault self-healing method and device, nonvolatile storage medium and electronic equipment

Abstract

The invention discloses a fault self-recovery method and device, a nonvolatile storage medium and electronic equipment. The method comprises the following steps: in response to a fault signal of a power distribution network system, obtaining fault state information of the power distribution network system in an initial topological structure; based on the fault state information, a target model is input for processing, a fault self-healing strategy is determined, the target model is obtained through decision training based on training environment state information, the training environment state information is used for simulating the fault state in a training environment, and the training environment changes based on topological structure changes; the training environment is used for restraining the power distribution network system to be in a steady state in the fault self-healing processing process, and the initial topological structure is adjusted based on the target topological structure indicated by the fault self-healing strategy, so that the power distribution network system is recovered from the fault state. The technical problem that the fault self-healing efficiency is not ideal in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of power systems and automation technologies thereof, and in particular to a fault self-healing method, device, non-volatile storage medium and electronic equipment. Background Art

[0002] The distribution network has the characteristics of closed-loop design and open-loop operation. When a distribution network operating in a radial structure fails, the loads downstream of the fault line will be interrupted from power supply. It is necessary to remove the faulty equipment as soon as possible and quickly restore the downstream power-off load users. Load transfer changes the power supply path through the operation and different combinations of tie switches and section switches. Under the premise of meeting the basic constraints of open-loop operation, voltage amplitude and current carrying capacity of the distribution network, the line load in the fault area is transferred to the remaining distribution lines, thereby achieving the effect of self-healing of the distribution network, reducing the power loss caused by the fault and the purpose of improving the reliability of the distribution network operation.

[0003] Related technologies rely on human or predetermined rules for fault recovery processing. As the scale of distribution networks continues to increase, there are differences between the actual power grid and the theoretically predetermined conditions, and the variables that affect fault recovery are of a large magnitude. Existing strategy generation methods are limited in their ability to quickly solve complex distribution network conditions and the uncertainty of network topology, resulting in an increasing risk of failure to successfully achieve self-healing after a fault.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a fault self-healing method, a device, a non-volatile storage medium and an electronic device, so as to at least solve the technical problem of unsatisfactory fault self-healing efficiency existing in the related art.

[0006] According to one aspect of an embodiment of the present invention, a fault self-healing method is provided, comprising: in response to a fault signal of a distribution network system, obtaining fault state information of the distribution network system in an initial topology structure; based on the fault state information, inputting a target model for processing, and determining a fault self-healing strategy, wherein the target model is obtained by decision training based on training environment state information, the training environment state information is used to simulate a state where a fault occurs in a training environment, the training environment changes based on a change in a topology structure, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process; based on the target topology structure indicated by the fault self-healing strategy, the initial topology structure is adjusted so that the distribution network system recovers from the fault state.

[0007] Optionally, the target model includes a main network and a target network, wherein the main network is used to select an action in a predetermined action space, the target network is used to determine the pros and cons of the action selected by the main network on fault recovery, and the main network and the target network collaborate to perform network updates.

[0008] Optionally, based on the fault status information, a target model is input for processing to determine a fault self-healing strategy, including: using a main network for processing to determine a plurality of first candidate actions corresponding to the fault status information, wherein the first candidate actions belong to an action space; determining predicted action values ​​corresponding to the plurality of first candidate actions respectively; determining a target action with the largest predicted action value among the plurality of first candidate actions; and determining a fault self-healing strategy based on the target action.

[0009] Optionally, determining a target action with the greatest predicted action value among multiple first candidate actions includes: determining a random number in response to fault status information; and determining a target action with the greatest predicted action value among multiple first candidate actions when the random number and a predetermined greediness factor satisfy a predetermined relationship; the method also includes: randomly selecting a target action from multiple first candidate actions when the random number and the greediness factor do not satisfy the predetermined relationship.

[0010] Optionally, the method also includes: using the main network to determine the intermediate action corresponding to the training environment state information, wherein the intermediate action belongs to the action space; generating training experience data based on the action reward generated by executing the intermediate action and the topological change generated by executing the intermediate action; based on the training experience data, using the target network to determine the target value corresponding to the intermediate action, wherein the target value is used to predict the action value in combination with the input training environment state information and the generated topological change; using the target value to update the network parameters of the main network to obtain an updated main network; and copying the network parameters of the main network at a predetermined update interval as an update to the target network to obtain an updated target network; determining the target model based on the updated main network and the updated target network.

[0011] Optionally, the method also includes: determining node voltages and line currents included in the distribution network system; and determining a training environment based on the node voltages, line currents, and topological changes occurring in the distribution network system.

[0012] Optionally, adjusting the initial topology based on the target topology indicated by the fault self-healing strategy includes: adjusting the initial topology based on the target topology indicated by the fault self-healing strategy by: controlling the opening and closing of switches, and / or controlling the feed circuit to perform isolation.

[0013] According to another aspect of an embodiment of the present invention, a fault self-healing device is provided, comprising: a state acquisition module, for obtaining fault state information of the distribution network system in an initial topology structure in response to a fault signal of the distribution network system; a strategy determination module, for inputting a target model for processing based on the fault state information, and determining a fault self-healing strategy, wherein the target model is obtained by decision training based on training environment state information, the training environment state information is used to simulate a state where a fault occurs in a training environment, the training environment changes based on a change in the topology structure, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process; a fault recovery module, for adjusting the initial topology structure based on a target topology structure indicated by the fault self-healing strategy, so that the distribution network system recovers from the fault state.

[0014] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided. The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing any one of the fault self-healing methods.

[0015] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the fault self-healing methods.

[0016] In an embodiment of the present invention, an offline target model training method is adopted to obtain the fault state information of the distribution network system in the initial topology structure by responding to the fault signal of the distribution network system; based on the fault state information, the target model is input for processing to determine the fault self-healing strategy, wherein the target model is obtained by decision training based on the training environment state information, the training environment state information is used to simulate the state of the fault in the training environment, the training environment changes based on the topology change, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process; based on the target topology structure indicated by the fault self-healing strategy, the initial topology structure is adjusted so that the distribution network system recovers from the fault state. The training environment can reflect the real conditions of the distribution network system, the topology changes in the fault self-healing process, and the target model can adapt to different system scenarios and fault types, and the technical effect of improving the fault self-healing efficiency is achieved, thereby solving the technical problem of unsatisfactory fault self-healing efficiency existing in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a flow chart of an optional fault self-healing method provided according to an embodiment of the present invention;

[0019] Figure 2 is a flow chart of an optional fault self-healing method provided according to an embodiment of the present invention;

[0020] Figure 3 is a topological diagram of an optional fault self-healing method provided according to an embodiment of the present invention;

[0021] Figure 4 is a schematic diagram of a model of an optional fault self-healing method provided according to an embodiment of the present invention;

[0022] Figure 5 is a rendering of an optional fault self-healing method provided according to an embodiment of the present invention;

[0023] Figure 6 is a self-healing schematic diagram of an optional fault self-healing method provided according to an embodiment of the present invention;

[0024] Figure 7 It is a schematic diagram of an optional fault self-healing device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] In response to the above problems, an embodiment of the present invention provides an embodiment of a fault handling method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0028] Figure 1 is a flow chart of an optional fault self-healing method provided according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0029] Step S102, in response to a fault signal of the distribution network system, obtaining fault state information of the distribution network system in an initial topology structure;

[0030] It can be understood that when the distribution network system detects a fault signal, the system will immediately capture the state information related to the fault. This information includes the location and type of the fault, key parameters such as the current system voltage, current, power, and the initial topology of the system (i.e., the connection method of each component in the network).

[0031] Step S104: Based on the fault state information, a target model is input for processing to determine a fault self-healing strategy, wherein the target model is obtained by decision-making training based on the training environment state information, the training environment state information is used to simulate the state of a fault in the training environment, the training environment changes based on the topology change, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process;

[0032] It can be understood that the acquired fault state information is input into a pre-trained target model. The target model is obtained by decision training in a simulation environment using a variety of training environment state information. The training environment simulates various possible fault states and generates different training scenarios based on changes in the topology. This training method enables the model to learn how to make correct decisions in various complex situations. In addition, the design of the training environment also takes into account the steady state that the distribution network system needs to maintain during the fault self-healing process, which means that the target model is constrained to generate self-healing strategies that can ensure system stability during the training process, so as to reduce training in invalid environments. Based on the input fault state information, the target model uses the knowledge and experience learned during the training process to quickly determine one or more feasible fault self-healing strategies. These strategies usually include how to adjust the switch state, how to reconfigure equipment parameters, etc., to achieve the purpose of fault isolation, load transfer and power restoration.

[0033] Optionally, the target model can be an agent, which is a concept in reinforcement learning and represents an entity that can interact with the environment and learn from experience. The function of the agent is to make decisions in the face of complex problems. A reinforcement learning problem can be represented as a five-tuple. The contents of this five-tuple represent several important elements in the training process of the agent: training environment, action space, environment state, and decision reward.

[0034] {S,A,p,r,γ}

[0035] A complete decision-making process τ (Trajectory) contains multiple steps (Step). In a learning cycle (Episode), the agent observes the training environment through learning, makes decisions through the state-action value function, obtains the action at the next moment (Step), and obtains the decision reward. At this time, the state of the training environment is transferred. In a learning cycle, the agent to be trained (represented by the symbol π) performs the above learning process again in the new environment after the transfer. After multiple cycles, the agent π completes the training. In the entire reinforcement learning process, the quality of the environment will directly affect the final learning results of the agent, and is directly related to the ability of the agent to make decisions when facing the need for fault self-healing. Therefore, the training environment should first be built according to the real conditions of the distribution system.

[0036] In the application of self-healing of distribution network system faults, the agent agent can learn how to select the optimal self-healing strategy under different fault conditions. The agent agent receives the fault state information of the distribution network system as input and regards it as the current state of the environment. It can select and execute an action (such as adjusting the switch state, configuring equipment parameters, etc.) based on this state, and then observe the impact of this action on the system state (i.e. reward or punishment). By constantly interacting with the environment and observing the results, the agent agent can learn how to select the optimal action based on the current state. Once the agent agent has learned the optimal self-healing strategy, it can automatically execute these strategies in practical applications. When a fault occurs in the distribution network system, the offline trained agent agent can quickly analyze the fault state and select the most appropriate self-healing strategy to restore the system.

[0037] In an optional embodiment, the target model includes a main network and a target network, wherein the main network is used to select an action in a predetermined action space, and the target network is used to determine the pros and cons of the action selected by the main network on fault recovery, and the main network and the target network collaborate to perform network updates.

[0038] It can be understood that the target model structure includes two parts: the main network and the target network. Each of them has different responsibilities and works together to achieve efficient determination of fault self-healing strategies and stable recovery of the system. The main network is used to select actions in a predetermined action space. The predetermined action space may include all possible switch operations, equipment configuration adjustments, etc., aiming to deal with various faults that may occur in the distribution network system. The main network processes and analyzes the input fault state information through deep learning technology to predict which actions are most likely to lead to system fault recovery under the current fault state. The target network is responsible for evaluating the actual impact of the actions selected by the main network on fault recovery. When the main network selects an action, the target network simulates the application of the action in the system and predicts the state change of the system after executing the action. By comparing the system state before and after the action is executed, the target network can determine the contribution of the action to fault recovery, that is, the quality. The main network and the target network will collaboratively perform network updates during the training process. The main network will continuously learn and optimize according to the new fault state information to improve the accuracy and efficiency of its action selection. The target network will update its parameters regularly to maintain consistency with the main network and ensure the accuracy of its evaluation results. In order to prevent overfitting and improve the stability of training, the update frequency of the target network is usually lower than that of the main network. This can maintain the stability of the target network to a certain extent and prevent the instability of the evaluation results caused by excessive changes in the main network parameters.

[0039] Through the above processing, by using the collaborative work of the main network and the target network, it is possible to more accurately predict which actions are most likely to lead to system failure recovery under the current fault state, thereby improving the accuracy of fault self-healing. The introduction of the target network can alleviate the problem of overfitting of the main network to a certain extent, making the entire system more stable and reliable when facing new faults. Through the selection of the main network in the action space and the evaluation of the target network, key actions can be located and the efficiency of fault resources can be optimized.

[0040] Optionally, the target model (such as an agent) is trained offline. For the power grid, this means that when no fault occurs, the agent can be trained in advance to have the ability to make decisions through continuous decision-making training with the help of a well-constructed training environment, so that the power grid can prepare for a rainy day.

[0041] In an optional embodiment, the method further includes: determining node voltages and line currents included in the distribution network system; and determining a training environment based on the node voltages, line currents, and topological changes occurring in the distribution network system.

[0042] It can be understood that in order to improve the accuracy and efficiency of the fault self-healing strategy of the distribution network system, the proposed method not only includes the collaborative work of the main network and the target network, but also further introduces the training environment construction process based on the real-time operating status of the distribution network system. It is necessary to monitor and record the voltage value of each node in the distribution network system and the current value on the line to evaluate the important parameters of the operating status of the distribution network system, and provide actual scenario data support for the subsequent construction of the training environment and the formulation of fault self-healing strategies.

[0043] Optionally, the components of the training environment are determined according to the distribution feeder topology, line type and fault location requirements. The training environment can be constructed, for example, by collecting historical data including the operating data of the distribution network system in the past period of time, such as node voltage, line current, equipment status, etc., covering a variety of different fault scenarios and topologies. Perform data preprocessing to clean, filter and annotate the collected data. The cleaning process can remove outliers and duplicate data, and the filtering process retains key features related to fault self-healing. The annotation process associates each data sample with the corresponding fault type and self-healing strategy.

[0044] To ensure that the network that has completed self-healing still meets the requirements of safe and stable operation of the system. Therefore, the training environment can reflect the constraints of safe and stable operation of the system. In the fault self-healing problem, the focus is on whether the voltage of the system node has exceeded the limit and whether the current carrying capacity of the system line has exceeded the limit. If the node voltage exceeds the limit, the load connected to the node will not be able to obtain normal and stable power supply, which may cause the load working capacity to decrease (beyond the lower limit) or the load device to burn out (beyond the upper limit). If the line current carrying capacity exceeds the limit, it will cause line damage and secondary equipment to burn out. Therefore, the training environment will include the system measurement information provided by the distribution automation system: node voltage and line current. The training environment also needs to reflect the topological changes of the distribution system during the fault self-healing process, such as: the opening and closing of the section switch and the contact switch, the isolation of the fault line, etc. This information can be obtained through the feeder automation system (FA). In summary, the training environment will include node voltage, line current, and line topology information that reflect the system status.

[0045] Optionally, when the training agent makes an action decision in a step, the training environment will change and transfer to a new state to provide an environment for the decision training of the next step. In order to reduce the occurrence of invalid states during the self-healing process and improve the efficiency of training, in order to control variables, it is preferred to set the changing element of each training environment to only a switch in the circuit.

[0046] In an optional embodiment, based on the fault status information, a target model is input for processing to determine a fault self-healing strategy, including: using a main network for processing to determine a plurality of first candidate actions corresponding to the fault status information, wherein the first candidate actions belong to an action space; determining predicted action values ​​corresponding to the plurality of first candidate actions respectively; determining a target action with the largest predicted action value among the plurality of first candidate actions; and determining a fault self-healing strategy based on the target action.

[0047] It can be understood that the input fault status information is passed to a main network included in a pre-trained target model. The main network outputs multiple candidate actions associated with these features based on the characteristics of the fault status information through its internal calculation logic and parameters. The candidate actions belong to a predefined action space, which contains all possible response actions that the power grid can take. After determining multiple first candidate actions, the value of each candidate action is evaluated, and the predicted action value reflects the long-term or short-term return that the system may obtain if the action is taken. For example, in a fault recovery scenario, the predicted value of an action may be related to factors such as its recovery speed, impact on system performance, stability, and secondary problems. The one with the greatest value is selected as the target action, and those actions with the highest expected return are preferred based on the optimization principle. Through the above processing, faults can be automatically detected, diagnosed, and repaired without human intervention, providing support for improving the reliability, availability, and maintainability of the power grid.

[0048] Optionally, the target model is an agent, and the agent can be a Double Deep Q Net (DDQN), which is a type of reinforcement learning algorithm. Due to its own special mechanism, it has a strong decision-making ability. The reinforcement learning algorithm based on probability theory and mathematical optimization algorithm has an inherent advantage in dealing with complex decision-making problems with randomness and uncertainty.

[0049] Optionally, at the beginning of DDQN training, two networks are initialized, namely the main network (MN) and the target network (TN), which is also the meaning of "double layer". And the parameters of MN and TN networks are made the same. In each training cycle (Episode), in each training step Step, the training environment state vector St is input (the current step environment state is represented by St), and St is input into the state action value function Q of MN. π (s,a). Q π(s,a) means the expected value of the decision reward that the agent π can obtain when the environment state is S and the agent takes state a until the current decision process τ ends. This value can be used to measure the quality of the agent π. The larger the value, the stronger the decision-making ability of the agent π, that is, the better the fault self-healing solution generated.

[0050] In an optional embodiment, determining a target action with the greatest predicted action value among multiple first candidate actions includes: determining a random number in response to fault status information; and determining a target action with the greatest predicted action value among multiple first candidate actions when the random number and a predetermined greediness factor satisfy a predetermined relationship; the method also includes: randomly selecting a target action from multiple first candidate actions when the random number and the greediness factor do not satisfy the predetermined relationship.

[0051] It can be understood that randomness and greedy strategies are introduced when determining the target action with the largest predicted action value to balance the trade-off between exploration and exploitation. When a target action needs to be selected, a random number is generated, for example, a value between 0 and 1, which is used to determine whether to select the action with the largest predicted value (exploitation) or randomly select an action. The relationship between this random number and a predetermined greedy factor is checked, and the greedy factor is used to control the ratio between exploration and exploitation. If the random number and the greedy factor meet the predetermined relationship, randomly selecting an action from a set of candidate actions instead of selecting the action with the largest predicted value helps prevent falling into a local optimal solution so that a better strategy can be found in the future. If the random number and the greedy factor do not meet the predetermined relationship, the action with the largest predicted action value will be selected as the target action from multiple first candidate actions, which can ensure that the system will not always fall into a local optimal solution, but has the opportunity to discover a better global strategy.

[0052] Through the above processing, by combining randomness and greedy strategies, more possibilities can be explored in the early stage of learning, while the known best strategies can be used more in the later stage of learning, thereby achieving continuous optimization and improvement of fault self-healing capabilities.

[0053] Optionally, after the state-action value function is calculated, the current step s will be given t The fault self-healing decision a should be made under t ,a t As shown in (1); action decision a t The acquisition mechanism combines the greedy idea, and its formula is shown in (1). ε is the greed factor, which can be set to a very small positive number in the range of [0,1]. Its purpose is to prevent the agent from falling into a local strategy during the training process. ε will gradually increase with the training process. Its purpose is to make the action decision more inclined to the state action value function Q in the early stage of the agent decision training.π (s, a), and as training proceeds, the probability of selecting other decisions will be expanded to achieve a wider range of learning in the action space.

[0054]

[0055] In an optional embodiment, the method further includes: using the main network to determine the intermediate action corresponding to the training environment state information, wherein the intermediate action belongs to the action space; generating training experience data based on the action reward generated by executing the intermediate action and the topological change generated by executing the intermediate action; based on the training experience data, using the target network to determine the target value corresponding to the intermediate action, wherein the target value is used to predict the action value in combination with the input training environment state information and the generated topological change; using the target value to update the network parameters of the main network to obtain an updated main network; and copying the network parameters of the main network at a predetermined update interval as an update to the target network to obtain an updated target network; determining the target model based on the updated main network and the updated target network.

[0056] It can be understood that the main network is used to determine an intermediate action according to the current training environment state information, that is, an action selected from the action space. After executing this intermediate action, the environment will give the corresponding action reward (feedback) and the topological change (environmental state change) caused by this action. This action, reward, state change and other information are saved as training experience data. The target network is used to evaluate the value of this intermediate action. Based on the current training environment state information, the intermediate action and the generated topological change, the target network is used to predict the cumulative reward (i.e., the target value) after executing this intermediate action. The training experience data (including action, reward, state change, etc.) and the target value given by the target network are used to update the network parameters of the main network. The update of the target network is not based on every training experience data, but is carried out according to the predetermined update interval. When this update interval is reached, the current network parameters of the main network are copied to the target network, thereby completing the update of the target network, and the update frequency is lower than that of the main network. After multiple iterations of the above steps, both the main network and the target network will gradually converge to the training completion state as the target model obtained by offline training. Through the above processing, the problem of training instability caused by rapid changes in network parameters can be effectively avoided, thereby improving the efficiency and effect of training.

[0057] Optionally, the network parameters of the main network are updated by a back-propagation algorithm and an optimizer (such as gradient descent).

[0058] Optionally, the Temporal Difference (TD) mechanism is adopted in the calculation process of updating the state-action value function. Compared with the traditional Monte-Carlo (MC) mechanism, the TD mechanism is more suitable for updating the agent intelligent body (i.e., the target model) for complex decision-making problems. The core idea of ​​TD learning is to learn to predict future rewards. The MC method updates the state value function at the beginning of each stage after executing a stage task, while the TD method updates each state encountered during the execution of any stage task, and the update of each state depends on the value function of the next state and the reward obtained by reaching the next state.

[0059] The application of TD mechanism in power grid fault self-healing can be expressed as (2):

[0060] Q π (s t+1 ,π(s t+1 ))+r t =Q π (s t ,a t+1 )(2)

[0061] After making an action decision, the action reward value r is obtained t , at this time the training environment is transferred to S t+1 So far, we have gained an experience, expressed as {s t ,a t ,r t ,s t+1}, and input this experience into the experience storage array Buffer. Set the agent learning frequency to F. After every F decisions, randomly sample from Buffer and calculate the target value y at the current step. Tar As shown in (3), the main network Q is updated by calculating the target value. π (s, a) weight; Set the TN update frequency to G, and synchronize the parameters in MN to TN after every G decisions.

[0062] y Tar =r t +Q π (s t+1 ,π(s t+1 ))(3)

[0063] Repeat the above process, and after multiple training cycles, a trained agent (i.e., target model) is obtained.

[0064] Step S106: Based on the target topology indicated by the fault self-healing strategy, the initial topology is adjusted so that the distribution network system recovers from the fault state.

[0065] It can be understood that according to the selected fault self-healing strategy, the system will instruct the relevant equipment in the distribution network system (such as switches, circuit breakers, etc.) to take action, thereby changing the system topology. The above-mentioned fault self-healing method is automatically executed without human intervention, so it can be completed in the shortest time to minimize power outage time and losses.

[0066] In an optional embodiment, the initial topology is adjusted based on the target topology indicated by the fault self-healing strategy, including: based on the target topology indicated by the fault self-healing strategy, the initial topology is adjusted in the following ways: controlling the opening and closing of switches, and / or controlling the feed circuit to perform isolation.

[0067] It can be understood that by controlling the opening and closing of the switch and / or controlling the isolation of the feeder circuit, the initial topology can be effectively adjusted to conform to the target topology indicated by the fault self-healing strategy. This adjustment method can ensure that when a network failure occurs, the fault area can be quickly and accurately isolated to protect the normal operation of the non-fault area, thereby improving the reliability and stability of the network. At the same time, it is also necessary to specify the on-demand settings in combination with the specific network environment and fault type to ensure the effectiveness and safety of the adjustment.

[0068] Optionally, in the network topology, the state of the switch (open or closed) directly affects the connection relationship between nodes and the data transmission path. Therefore, when the initial topology needs to be adjusted to adapt to the target topology indicated by the fault self-healing strategy, the switch can be directly controlled to open and close. According to the location and type of the fault, the switch position that needs to be adjusted is determined, and the state of these switches is changed by remote control or local operation. The change of the switch state can also be monitored to ensure that the network topology is adjusted as expected.

[0069] Optionally, the tie switch (LS) is a normally open switch, which is usually kept in an open state during operation and is used for emergency standby transfer. The section switch (FS) is a normally closed switch, which is usually kept in a closed state during operation. Its main function is to isolate the fault. Load transfer changes the power supply path through the operation and different combinations of tie switches and section switches. Under the premise of meeting the basic constraints of open-loop operation, voltage amplitude and current carrying capacity of the distribution network, the line load in the fault area is transferred to the remaining distribution lines, thereby achieving the effect of self-healing of the distribution network.

[0070] Optionally, in order to isolate the fault area or protect the normal operation of the non-fault area, it is necessary to control the feeder circuit to perform an isolation operation, which involves disconnecting the power connection between the fault area and the normal area to prevent the fault from spreading. Controlling the feeder circuit to perform isolation can be performed in the following ways, such as determining the fault area to be isolated, finding the feeder circuit related to the area, isolating the fault area by disconnecting the related circuit breaker or switch, verifying the isolation effect, and ensuring that the fault area does not affect the normal area.

[0071] Through the above step S102, in response to the fault signal of the distribution network system, the fault state information of the distribution network system in the initial topology structure is obtained; step S104, based on the fault state information, the target model is input for processing to determine the fault self-healing strategy, wherein the target model is obtained by decision training based on the training environment state information, the training environment state information is used to simulate the state of the fault in the training environment, the training environment changes based on the topology change, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process; step S106, based on the target topology structure indicated by the fault self-healing strategy, the initial topology structure is adjusted so that the distribution network system recovers from the fault state. It can achieve the purpose of the training environment being able to reflect the real conditions of the distribution network system and the topology changes during the fault self-healing process, so that the target model can adapt to different system scenarios and fault types, and achieve the technical effect of improving the fault self-healing efficiency, thereby solving the technical problem of unsatisfactory fault self-healing efficiency existing in the related technology.

[0072] Based on the above embodiments and optional embodiments, the present invention proposes an optional implementation mode: Figure 2 is a flow chart of an optional fault self-healing method provided according to an embodiment of the present invention, Figure 2 With the help of data provided by the distribution automation system, based on the reinforcement learning DDQN technology, the agent intelligent agent (i.e., the target model) for fault self-healing strategy generation is trained offline. By building the training environment and designing the action space, an intelligent agent with decision-making ability is successfully trained. In actual use, the state of the system after the fault is input, and the intelligent agent can quickly generate a fault self-healing strategy. The following steps may be included:

[0073] Step S1, constructing a training environment required for agent training with the help of system information provided by the distribution automation system.

[0074] Step S11, determining the components of the training environment according to the distribution feeder topology, line type and fault location requirements.

[0075] Step S12, determining the update content of the training environment according to the learning requirements of the fault self-healing problem reinforcement learning model.

[0076] Step S2: offline training of the agent.

[0077] Step S3, deploy the agent agent online.

[0078] Step S4: When a fault occurs, input system fault information and generate a self-healing strategy.

[0079] Figure 3 is a topological diagram of an optional fault self-healing method provided according to an embodiment of the present invention, Figure 3 The figure shows the line topology of the embodiment. There are 32 load nodes and 1 power bus in the network. Five tie switches are configured in the network, which are recorded as 33, 34, 36, and 37, as well as 32 operable lines (indicated by solid lines). The dotted lines represent the lines that can be switched. The line between nodes 13 and 14 fails as an example. The star represents the failure.

[0080] Figure 4 is a schematic diagram of a model of an optional fault self-healing method provided according to an embodiment of the present invention, Figure 4 As shown in , it is the variable transfer relationship during the training process of the agent intelligent body (i.e., the target model). Figure 3 It can be seen that the system state is used as input into the state-action-value function, the main network is responsible for predicting the network action, and the target network is responsible for giving the target value of the action.

[0081] Figure 5 is a rendering of an optional fault self-healing method provided according to an embodiment of the present invention, Figure 5 As shown in the figure, it is the reward value obtained by the agent after completing a complete decision-making process during the training process. It can be observed that in the early stage of training, the reward obtained by the agent when completing the decision is low. This is because the decision-making ability of the agent is still low, the value of the action made is low, and the final action reward obtained is also low; as the training process progresses, the agent learns more decision-making experience from the experience storage array, the decision-making ability is improved, and it can make more correct decisions, which also means that it can output better self-healing solutions.

[0082] Figure 6 is a self-healing schematic diagram of an optional fault self-healing method provided according to an embodiment of the present invention, Figure 6 As shown in the figure, after a fault occurs in the embodiment, the agent agent obtains the fault self-healing strategy, and the disconnection line is based on Figure 2The black line indicates the closed circuit after executing the switch action. From the network topology, we can see that the load downstream of the fault location is transferred to the 26-33 branch through the tie switches 34, 36, and 37, and the section switch disconnects the branch between 5 and 7 to ensure that there is no loop in the entire network, meeting the requirements of open-loop operation of the distribution network. The application uses the system information provided by the distribution automation system and, based on reinforcement learning technology, can generate a self-healing strategy by training the agent intelligent body when a fault occurs. The power transfer of the power-lost load is realized.

[0083] According to the above optional implementation, the fault self-healing strategy generation technology based on reinforcement learning supports offline training and online deployment. Offline training can be carried out by simulating the occurrence of faults in the training environment to perform decision-making training before the distribution system fails, so as to generate an agent that can make fault self-healing decisions. When the training is completed, it can be deployed. When the detected distribution network line fails, you only need to input the system environment under the fault condition to quickly generate a fault self-healing strategy. It solves the problem that fault self-healing strategies for complex distribution system networks are difficult to generate. By training the agent in a pre-built training environment, it can be realized in an offline state. When the trained agent is applied online, you only need to input the system status to quickly generate a fault self-healing strategy.

[0084] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0085] In this embodiment, a fault self-healing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the terms "module" and "device" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0086] According to an embodiment of the present invention, there is also provided an embodiment of a device for implementing a fault self-healing method. Figure 7 is a schematic diagram of a fault self-healing device according to an embodiment of the present invention. Figure 7 As shown, the above fault self-healing device includes: a state acquisition module 702, a strategy determination module 704, and a fault recovery module 706. The device is described below.

[0087] A state acquisition module 702 is used to obtain fault state information of the distribution network system in an initial topology structure in response to a fault signal of the distribution network system;

[0088] The strategy determination module 704 is connected to the state acquisition module 702, and is used to input the target model for processing based on the fault state information to determine the fault self-healing strategy, wherein the target model is obtained by decision-making training based on the training environment state information, the training environment state information is used to simulate the state of the fault in the training environment, the training environment changes based on the topology change, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process;

[0089] The fault recovery module 706 is connected to the strategy determination module 704 and is used to adjust the initial topology structure based on the target topology structure indicated by the fault self-healing strategy so that the distribution network system recovers from the fault state.

[0090] In a fault self-healing device provided by an embodiment of the present invention, a state acquisition module 702 is set to respond to a fault signal of a distribution network system and obtain fault state information of the distribution network system in an initial topology structure; a strategy determination module 704 is connected to the state acquisition module 702 and is used to input a target model for processing based on the fault state information to determine a fault self-healing strategy, wherein the target model is obtained by decision training based on training environment state information, the training environment state information is used to simulate the state of a fault in the training environment, the training environment changes based on the change of the topology structure, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process; a fault recovery module 706 is connected to the strategy determination module 704 and is used to adjust the initial topology structure based on the target topology structure indicated by the fault self-healing strategy, so that the distribution network system recovers from the fault state. The training environment can reflect the real conditions of the distribution network system, the topology changes during the fault self-healing process, and the target model can adapt to different system scenarios and fault types, thereby achieving the technical effect of improving the fault self-healing efficiency, thereby solving the technical problem of unsatisfactory fault self-healing efficiency existing in the related art.

[0091] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0092] It should be noted that the above-mentioned state acquisition module 702, strategy determination module 704, and fault recovery module 706 correspond to steps S102 to S106 in the embodiment, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment. It should be noted that the above-mentioned modules can be run in a computer terminal as part of the device.

[0093] It should be noted that the optional or preferred implementation of this embodiment can refer to the relevant description in the embodiment, which will not be repeated here.

[0094] The above-mentioned fault self-healing device may also include a processor and a memory, and the state acquisition module 702, the strategy determination module 704, the fault recovery module 706, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions.

[0095] The processor includes a kernel, which retrieves the corresponding program unit from the memory. There can be one or more kernels. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0096] An embodiment of the present invention provides a non-volatile storage medium on which a program is stored. When the program is executed by a processor, a fault self-healing method is implemented.

[0097] An embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the following steps are implemented: in response to a fault signal of a distribution network system, the fault state information of the distribution network system in the initial topology structure is obtained; based on the fault state information, a target model is input for processing to determine a fault self-healing strategy, wherein the target model is obtained by decision training based on training environment state information, the training environment state information is used to simulate the state of a fault in the training environment, the training environment changes based on the change of the topology structure, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process; based on the target topology structure indicated by the fault self-healing strategy, the initial topology structure is adjusted so that the distribution network system recovers from the fault state. The device in this article can be a server, a PC, etc.

[0098] The present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with the following method steps: in response to a fault signal of a distribution network system, obtaining fault state information of the distribution network system in an initial topology structure; based on the fault state information, inputting a target model for processing, and determining a fault self-healing strategy, wherein the target model is obtained by decision training based on training environment state information, the training environment state information is used to simulate a fault state in a training environment, the training environment changes based on a change in the topology structure, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process; based on the target topology structure indicated by the fault self-healing strategy, the initial topology structure is adjusted so that the distribution network system recovers from the fault state.

[0099] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0103] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0104] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0105] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0106] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0107] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0108] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A fault self-healing method, characterized in that: include: In response to a fault signal of the distribution network system, obtaining fault state information of the distribution network system in an initial topology structure; Based on the fault state information, a target model is input for processing to determine a fault self-healing strategy, wherein the target model is obtained by decision-making training based on training environment state information, the training environment state information is used to simulate a state of a fault in a training environment, the training environment changes based on a topology change, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process; Based on the target topology indicated by the fault self-healing strategy, the initial topology is adjusted so that the power distribution network system recovers from the fault state.

2. The method according to claim 1, characterized in that The target model includes a main network and a target network, wherein the main network is used to select an action in a predetermined action space, and the target network is used to determine the pros and cons of the action selected by the main network on fault recovery, and the main network and the target network collaborate to perform network updates.

3. The method according to claim 2, characterized in that The step of inputting the target model for processing based on the fault status information to determine the fault self-healing strategy includes: Using the main network for processing, determining a plurality of first candidate actions corresponding to the fault state information, wherein the first candidate actions belong to the action space; Determine predicted action values ​​corresponding to each of the plurality of first candidate actions; Determining a target action with the largest predicted action value among the plurality of first candidate actions; Based on the target action, the fault self-healing strategy is determined.

4. The method according to claim 3, characterized in that The step of determining the target action with the largest predicted action value among the plurality of first candidate actions comprises: In response to the fault status information, determining a random number; In the case where the random number and the predetermined greed factor satisfy a predetermined relationship, determining the target action with the largest predicted action value among the plurality of first candidate actions; The method further includes: randomly selecting the target action from the plurality of first candidate actions when the random number and the greed factor do not satisfy the predetermined relationship.

5. The method according to claim 2, characterized in that: The method further comprises: Using the main network, determining an intermediate action corresponding to the training environment state information, wherein the intermediate action belongs to the action space; Generate training experience data based on the action reward generated by executing the intermediate action and the topological change generated by executing the intermediate action; Based on the training experience data, the target network is used to determine the target value corresponding to the intermediate action, wherein the target value is used to predict the action value in combination with the input training environment state information and the generated topological changes; Using the target value, updating the network parameters of the main network to obtain the updated main network; and copying the network parameters of the main network at a predetermined update interval as an update process for the target network to obtain the updated target network; The target model is determined based on the updated main network and the updated target network.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Determining node voltages and line currents included in the power distribution network system; The training environment is determined based on the node voltage, the line current, and the topology change of the distribution network system.

7. The method according to any one of claims 1 to 5, characterized in that The adjusting the initial topology structure based on the target topology structure indicated by the fault self-healing strategy includes: Based on the target topology indicated by the fault self-healing strategy, the initial topology is adjusted in the following manners: controlling the opening and closing of switches, and / or controlling the feeder circuit to perform isolation.

8. A fault self-healing device, characterized in that: include: A state acquisition module, used for acquiring fault state information of the distribution network system in an initial topology structure in response to a fault signal of the distribution network system; A strategy determination module is used to input a target model for processing based on the fault state information to determine a fault self-healing strategy, wherein the target model is obtained by decision-making training based on training environment state information, the training environment state information is used to simulate a state of a fault in a training environment, the training environment changes based on a topology change, and the training environment is used to constrain the distribution network system to be in a steady state during the fault self-healing process; A fault recovery module is used to adjust the initial topology structure based on the target topology structure indicated by the fault self-healing strategy so that the distribution network system recovers from the fault state.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the fault self-healing method described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the fault self-healing method described in any one of claims 1 to 7.

Images