Power distribution protection method and device based on Q learning and block chain, and storage medium

Abstract

The invention provides a power distribution protection method and device based on Q learning and a block chain, and a storage medium. The method comprises the steps that training data are acquired, a Q learning algorithm is adopted to process the training data to obtain a Q learning decision tree model, and a preset rule is followed in the processing process; inputting the collected data of the plurality of system components obtained by the plurality of slave agent modules into a data block of the block chain, controlling the main agent module to calculate the collected data in the data block to obtain fault information in the collected data, obtaining a target slave agent module determined by the main agent module according to the fault information and a Q learning decision tree model, and sending the target slave agent module to the block chain; and controlling the master agent module to send a tripping request to the target slave agent module, so that the target slave agent module performs tripping operation according to the tripping request. Reliability and efficiency of relay agency are improved by adopting a Q learning algorithm, and data interaction is performed by adopting a block chain system, so that different types of faults are effectively eliminated.

Description

Technical Field

[0001] The present application relates to the technical field of power distribution protection, and more specifically, to a power distribution protection method, device, computer-readable storage medium and electronic device based on Q learning and blockchain. Background Art

[0002] As interest in renewable energy grows, distributed power sources such as distributed generators, energy storage and controlled loads are rapidly being integrated into the distribution system and, if managed properly, can improve grid efficiency and reliability.

[0003] However, there are still challenges in the control and operation of distributed power sources in the distribution system. The continuous increase of new generation power sources requires the design of distribution network protection schemes to be flexible and adjustable. When a fault occurs in the radial network, these power sources will add additional current to the fault point. These phenomena require more careful design of protection systems and bring new challenges. In traditional distribution systems, only one feeder can guarantee unidirectional power flow. However, with the penetration of distributed power sources, multiple power sources cause bidirectional power flow, and traditional fault detection and isolation algorithms cannot be implemented in this system. Therefore, in order to operate the system efficiently, some scholars suggest installing more relays in the system. However, introducing more relays is not cost-effective, and the system may still not be flexible enough to cope with different types of faults. Summary of the invention

[0004] The main purpose of this application is to provide a distribution protection method, device, computer-readable storage medium and electronic device based on Q learning and blockchain, so as to at least solve the problem that the distribution system of the prior art is inflexible and unable to cope with different types of faults.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a distribution protection method based on Q learning and blockchain is provided, and the method is applied to a controller of a multi-agent distribution protection system, wherein the distribution protection system also includes a master agent module, multiple slave agent modules and multiple system components, and the method includes: obtaining training data, and processing the training data using a Q learning algorithm to obtain a Q learning decision tree model, wherein the processing follows preset rules, wherein the training data includes normal data and fault data, and the preset rules at least include the corresponding relationship between the slave agent module and the system component; inputting the collected data of the multiple system components obtained by the multiple slave agent modules into a data block of the blockchain, controlling the master agent module to calculate the collected data in the data block, and obtaining fault information in the collected data, wherein the collected data includes the current value, current phase angle and power flow of the system component; obtaining the target slave agent module determined by the master agent module according to the fault information and the Q learning decision tree model, and controlling the master agent module to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request.

[0006] Optionally, the method further includes: according to the first-in-first-out rule of queuing theory, the slave proxy module located at the Mth entry line is the master proxy module at the Mth time interval, M≥1; in the Nth time interval, the slave proxy module at the Nth entry line is the master proxy module; in the N+1th time interval, the slave proxy module that first enters the line is the master proxy module, wherein N≥2, and N is the total number of the slave proxy modules.

[0007] Optionally, the collected data of the multiple system components obtained by the multiple slave agent modules are input into the data block of the blockchain, and the master agent module is controlled to calculate the collected data in the data block to obtain fault information in the collected data, including: inputting the current phase angle data of the multiple system components obtained by the multiple slave agent modules into the data block of the blockchain; using the fault model in the master agent module to calculate the current phase angle data in the data block to obtain the phase angle difference of the multiple system components, generating the fault information when the phase angle difference is greater than or less than the phase angle difference of the previous cycle, and not generating the fault information when the phase angle difference is equal to the phase angle difference of the previous cycle.

[0008] Optionally, after controlling the master agent module to send a tripping request to the target slave agent module so that the target slave agent module performs a tripping operation according to the tripping request, the method further includes: sending a feedback signal to the master agent module, wherein the feedback signal includes a reward signal and a penalty signal; when the tripping operation is correct, sending a reward signal to the master agent module, and when the tripping operation is incorrect, sending a penalty signal to the master agent module.

[0009] Optionally, in the case of a tripping operation error, after sending a penalty signal to the master agent module, the method further comprises: using a fuzzy learning algorithm to improve the Q learning decision tree model to obtain an improved Q learning decision tree model.

[0010] Optionally, the method also includes: using a Java agent development framework JADE to simulate the communication between the master agent module and the slave agent module to determine whether the communication between the master agent module and the slave agent module is normal; using a MATLAB platform to simulate the distribution protection system to determine whether the fault model identifies the fault information accurately.

[0011] Optionally, the Q-learning decision tree model is:

[0012]

[0013] Among them, newQ(S t ,A t ) is the new Q value based on state and action, α is the learning efficiency, r t is the reward based on state and action, γ is the discount factor, is the maximum future reward given a new state and action, Q(S t ,A t ) is the current Q value.

[0014] According to another aspect of the present application, a distribution protection device based on Q learning and blockchain is provided, and the device is applied to a controller of a multi-agent distribution protection system, wherein the distribution protection system further includes a master agent module, multiple slave agent modules and multiple system components, and the device includes: a first acquisition unit, used to acquire training data, and use a Q learning algorithm to process the training data to obtain a Q learning decision tree model, wherein the processing follows preset rules, wherein the training data includes normal data and fault data, and the preset rules at least include a correspondence between the slave agent module and the system component; a control unit, used to input the collected data of the multiple system components acquired by the multiple slave agent modules into a data block of the blockchain, control the master agent module to calculate the collected data in the data block, and obtain fault information in the collected data, wherein the collected data includes the current value, current phase angle and power flow of the system component; a second acquisition unit, used to acquire the target slave agent module determined by the master agent module according to the fault information and the Q learning decision tree model, and control the master agent module to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request.

[0015] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the distribution protection methods based on Q learning and blockchain.

[0016] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the distribution protection methods based on Q learning and blockchain.

[0017] Apply the technical solution of the present application, obtain training data, use Q learning algorithm to process the training data to obtain Q learning decision tree model, wherein the processing process follows preset rules, wherein the training data includes normal data and fault data, and the preset rules at least include the correspondence between the slave agent module and the system component; input the collected data of multiple system components obtained by multiple slave agent modules into the data block of the blockchain, control the main agent module to calculate the collected data in the data block, and obtain the fault information in the collected data, wherein the collected data includes the current value, current phase angle and power flow of the system component; obtain the target slave agent module determined by the main agent module according to the fault information and the Q learning decision tree model, and control the main agent module to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request. By using the Q learning algorithm to train the execution agent to make reliable decisions in the process of fault identification and clearing, the Q learning algorithm improves the reliability and efficiency of the relay agent, and the agents use a decentralized connection based on blockchain to exchange information, which solves the problem that the distribution system of the prior art is inflexible and cannot cope with different types of faults. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings constituting part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A hardware structure block diagram of a mobile terminal for executing a power distribution protection method based on Q learning and blockchain provided in an embodiment of the present application is shown;

[0020] Figure 2 A schematic diagram of a flow chart of a power distribution protection method based on Q learning and blockchain provided according to an embodiment of the present application is shown;

[0021] Figure 3 A schematic diagram of a power distribution protection system provided according to an embodiment of the present application is shown;

[0022] Figure 4 A schematic diagram of a blockchain including data blocks provided according to an embodiment of the present application is shown;

[0023] Figure 5 An interactive schematic diagram of a Q learning algorithm provided according to an embodiment of the present application is shown;

[0024] Figure 6 A schematic diagram of a graphical user interface of JADE provided according to an embodiment of the present application is shown;

[0025] Figure 7aA schematic diagram of a fault-free current waveform provided according to an embodiment of the present application is shown;

[0026] Figure 7b A schematic diagram of current waveform when phase A fails according to an embodiment of the present application is shown;

[0027] Figure 7c A schematic diagram of current waveforms when phase A and phase B are faulty according to an embodiment of the present application is shown;

[0028] Figure 7d A schematic diagram of current waveforms when all three phases are faulty according to an embodiment of the present application is shown;

[0029] Figure 8 A structural block diagram of a power distribution protection device based on Q learning and blockchain provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0034] P2P network: Peer to Peer network, that is, peer-to-peer computer network, is a distributed application architecture that distributes tasks and work responsibilities among peers. It is a networking or network form formed by the peer computing model and the application layer.

[0035] Q-Learning algorithm: Q-Learning is a reinforcement learning algorithm based on value functions, which aims to enable an intelligent agent (usually a machine or program) to learn how to make optimal decisions through interaction with the environment. The core idea of ​​the algorithm is to learn a Q-value function, which measures the expected return of taking an action in a given state. The Q-value function is the core component of the Q-Learning algorithm, which searches for the optimal strategy through continuous iteration and updating to maximize the cumulative reward.

[0036] Blockchain: (English name: blockchain or block chain) is a decentralized distributed ledger with block chain storage, non-tamperability, security and reliability. It combines distributed storage, point-to-point transmission, consensus mechanism, cryptography and other technologies to record transactions and information through a growing data block chain (Blocks) to ensure data security and transparency.

[0037] As introduced in the background technology, the power distribution system of the prior art is inflexible and cannot cope with the problem of different types of faults. In order to solve the problem that the power distribution system of the prior art is inflexible and cannot cope with different types of faults, the embodiments of the present application provide a power distribution protection method, device, computer-readable storage medium and electronic device based on Q learning and blockchain.

[0038] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0039] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a power distribution protection method based on Q learning and blockchain according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1More or fewer components as shown, or with Figure 1 Different configurations shown.

[0040] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the distribution protection method based on Q learning and blockchain in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific network examples may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0041] In this embodiment, a distribution protection method based on Q learning and blockchain is provided, which runs on a mobile terminal, a computer terminal or a similar computing device. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0042] Figure 2 The flowchart of the power distribution protection method based on Q learning and blockchain according to the embodiment of the present application is applied to the controller of the multi-agent power distribution protection system, wherein the power distribution protection system also includes a master agent module, multiple slave agent modules and multiple system components. Figure 2 As shown, the method comprises the following steps:

[0043] Step S201, obtaining training data, and processing the training data using a Q learning algorithm to obtain a Q learning decision tree model, wherein the processing follows preset rules, wherein the training data includes normal data and fault data, and the preset rules at least include a correspondence between a slave agent module and a system component;

[0044] Specifically, in order to detect faults more reliably, a Q-learning algorithm is used. In Q-learning, training data is obtained from empirical evidence of defective and non-defective conditions, such as current magnitude, phase difference between terminals, and power flow. Based on the training data, Q-learning follows preset rules to generate a Q-learning decision tree model, where the preset rules also include whether to perform tripping processing based on the data.

[0045] Step S202, inputting the collected data of multiple system components obtained from the multiple agent modules into the data block of the blockchain, controlling the main agent module to calculate the collected data in the data block, and obtaining fault information in the collected data, wherein the collected data includes the current value, current phase angle and power flow of the system components;

[0046] Specifically, the blockchain system is a distributed ledger that records data in individual blocks. Users have a copy of the blockchain, and any changes to the content or order of the blocks can be immediately canceled by checking the validity of the blocks. Such a system can also be used to improve the reliability of power system protection. In the proposed method, the blockchain data block is a table containing detailed information such as the phase angle, current magnitude, agent identification, and timestamp of the sending and receiving terminals of each line. The master agent then looks at the data in the blockchain and calculates the phase angle difference between the start and end terminals of each section. It then adds this information to its data block as a binary notification. When the data exchange is approved, the data containing the network status information is sent to the blockchain. Therefore, when the information block exchange is completed, the information of each agent's data block at any point in time can be seen by other agents. In addition, the decisions reported by the master agent on the fault conditions associated with these blocks are also stored in the data blocks.

[0047] Step S203, obtaining the target slave agent module determined by the master agent module according to the fault information and the Q learning decision tree model, and controlling the master agent module to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request.

[0048] Specifically, the blockchain consists of information about system parameters and decisions associated with the settings. The master agent CBA of the current interval deploys the protocol through the P2P network. In the proposed method, all CBAs are responsible for developing a data block D containing system parameters. i Data Block D iContains information about the current amplitude, phase angle, binary notification to the agent, and timestamp. Data block example: D3 = [60, {121, 119, 123}, 0, 1551455280], the data block can be interpreted as CBA3 reporting 60A current, 121, 119, 123 are the phase angles of the three phases, 0 means CBA3 decided not to trip, and 1551395400 is the unix timestamp.

[0049] Through this embodiment, training data is obtained, and the training data is processed by a Q learning algorithm to obtain a Q learning decision tree model, wherein the processing process follows preset rules, wherein the training data includes normal data and fault data, and the preset rules include at least the corresponding relationship between the slave agent module and the system component; the collected data of multiple system components obtained by multiple slave agent modules are input into the data block of the blockchain, and the master agent module is controlled to calculate the collected data in the data block to obtain the fault information in the collected data, wherein the collected data includes the current value, current phase angle and power flow of the system component; the target slave agent module determined by the master agent module according to the fault information and the Q learning decision tree model is obtained, and the master agent module is controlled to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request. By using the Q learning algorithm to train the execution agent to make reliable decisions in the process of fault identification and clearing, the Q learning algorithm improves the reliability and efficiency of the relay agent, and the decentralized connection based on the blockchain is used between the agents to exchange information, which solves the problem that the distribution system of the prior art is inflexible and cannot cope with different types of faults.

[0050] In the specific implementation process, the method also includes: according to the first-in-first-out rule of queuing theory, the slave proxy module located at the Mth entry line is the master proxy module in the Mth time interval, M≥1; in the Nth time interval, the slave proxy module entering the line is the master proxy module; in the N+1th time interval, the first slave proxy module entering the line is the master proxy module, where N≥2, and N is the total number of slave proxy modules.

[0051] In the power distribution system, the relay system installed on the busbar should be able to respond to different fault conditions and clear them quickly. Figure 3 As shown in Figure 1, an example of a multi-agent protection system in a distribution network with two distributed generation sources is illustrated. Figure 3In the proposed system, each circuit breaker CB is controlled by an agent CBA, which cooperates with other agent CBAs to effectively detect and handle faults. The collaboration between agents improves the reliability of the system and prevents problems associated with single point failures. The system is modeled using 6 role-based agents, which are divided into master agents and slave agents according to their current roles. According to queuing theory, at different times, the role-based agents can be slave agents or master agents and can change their positions. Each role-based agent is placed in a queue to play the role of a master agent. Queuing theory is based on the first-in-first-out (FIFO) rule, and the first agent to enter the line will become the master agent in the first time interval. In the proposed system, each CBA continuously measures the parameters of the line for fault calculation and reports to the master agent. These parameters include current magnitude, phase difference between terminals, and power flow, and the master agent calculates these parameters and detects fault conditions.

[0052] Specifically, the collected data of multiple system components obtained from multiple agent modules are input into the data block of the blockchain, and the main agent module is controlled to calculate the collected data in the data block to obtain fault information in the collected data, including: inputting the current phase angle data of multiple system components obtained from multiple agent modules into the data block of the blockchain; using the fault model in the main agent module to calculate the current phase angle data in the data block to obtain the phase angle difference of multiple system components, generating fault information when the phase angle difference is greater than or less than the phase angle difference of the previous cycle, and not generating fault information when the phase angle difference is equal to the phase angle difference of the previous cycle.

[0053] In this method, when a defect is identified, the master agent selects the corresponding agent according to the Q learning decision tree and sends a trip request to the agent. The master agent calculates the difference in the current phase angles at both ends of the line, the current phase angle θ on one side of the terminal a ,θ b ,θ c The current phase shift is 120 degrees. a′ ,θ b′ ,θ c′ The difference in phase can be calculated using the following model:

[0054] Δθ a,b,c =Σ i＝1 (θ a,b,c (i)-θ a′b′c′ (i));

[0055] View Δθ a,b,c The master agent can detect faults by using the parameters and current amplitude. The fault-free judgment condition is:

[0056]

[0057] The fault judgment conditions are:

[0058]

[0059] Among them, Δθ a,b,c is the phase angle difference of the current cycle, is the phase angle difference of the previous cycle;

[0060] If the phase angle difference between the starting end and the ending end of each line is much larger or smaller than the phase angle difference of the previous cycle, the situation is considered abnormal.

[0061] For current faults, when the current I exceeds the load current I load When it is more than 1.25 times of I, it is considered a fault, that is, |I|>I load ×1.25;

[0062] If a fault occurs in the line, the proxy will send a signal to the SBs on both sides of the section to isolate them from the fault.

[0063] More specifically, after controlling the master agent module to send a tripping request to the target slave agent module so that the target slave agent module performs a tripping operation according to the tripping request, the method further includes: sending a feedback signal to the master agent module, wherein the feedback signal includes a reward signal and a penalty signal; when the tripping operation is correct, sending a reward signal to the master agent module, and when the tripping operation is wrong, sending a penalty signal to the master agent module.

[0064] This method is Figure 4 The interactive diagram of the Q-learning algorithm shown in the figure shows that the learners are agents that understand each other and can share information. Everything outside the agent is considered an environment. The agent uses feedback from the environment to learn; in Q-learning, the model is created to generate a decision tree, in which the output is partially random under the control of the decision maker. Figure 4 In , we can observe that the agent interacts with the environment in discrete time steps. Then, at each time step, the agent receives the state S t And decide on a new operation A t In response to the agent’s actions, the environment gives the agent a reward R t+1 and a new state S t+1 , the process of giving rewards and states is repeated continuously and produces a sequence trajectory.

[0065] Furthermore, in the case of a tripping operation error, after sending a penalty signal to the main agent module, the method further includes: using a fuzzy learning algorithm to improve the Q learning decision tree model to obtain an improved Q learning decision tree model.

[0066] This method uses a fuzzy learning algorithm to repeatedly modify and punish the Q-learning decision tree model to continuously improve the Q-learning decision tree model, which can make the trip request generated by the master agent based on the Q-learning decision tree model more accurate.

[0067] Furthermore, the method also includes: using the Java agent development framework JADE to simulate the communication between the master agent module and the slave agent module to determine whether the communication between the master agent module and the slave agent module is normal; using the MATLAB platform to simulate the distribution protection system to determine whether the fault model identifies the fault information accurately.

[0068] This method uses the Java Agent Development Framework (JADE) to simulate the communication between agents. Figure 3 The proposed method is tested on the power distribution system shown in the figure. All the agents and communication rules of the circuit breakers are developed in JADE software; MATLAB is used to Figure 3 The distribution system shown in the figure is simulated. CBA controls the action of the circuit breaker and is modeled in MATLAB to record the current magnitude and phase difference. In JADE, CBA is modeled as sending information or request data blocks, where the notification data block Di contains system parameters and request data blocks to trip a specific circuit breaker. MATLAB Simulink is connected to JADE software through TCP / IP protocol. According to the following properties, the fault model is simulated in MATLAB.

[0069] Specifically, the Q-learning decision tree model is:

[0070]

[0071] Among them, newQ(S t ,A t ) is the new Q value based on state and action, α is the learning efficiency, r t is the reward based on state and action, γ is the discount factor, is the maximum future reward given a new state and action, Q(S t ,A t ) is the current Q value.

[0072] The agents in this method interact with each other at discrete time steps t = 0, 1, 2 ... n. The agents can take states S = S1, S2, S3 ... S n And action A = A1, A2, A3...A n At each discrete time, the agent gets a reward from the environment for the current state, makes a decision, and takes an appropriate action. In response to taking an action, the state of the environment will be changed to a new state S t+1 = S′. The agent will receive a new reward rt+1 If the fault is correctly identified, a reward is given.

[0073] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the distribution protection method based on Q learning and blockchain of the present application will be described in detail below in combination with specific embodiments.

[0074] This embodiment relates to a specific power distribution protection method based on Q learning and blockchain, which specifically includes the following contents:

[0075] A multi-agent system consists of intelligent agents that interact and communicate with each other to achieve a common goal. A multi-agent system makes decisions based on messages and notifications from other agents to complete complex tasks. Distribution system components such as transformers, circuit breakers, etc. contain information that can be shared through intelligent agents. In a multi-agent system, the role of the intelligent agents is to collect information from electrical equipment and share it with the master agent. The master agent is responsible for the coordination between the agents. These master agents are responsible for making decisions and notifying specific agents to take specific actions. The operation and coordination of relays should be considered while designing a protection multi-agent system. The following describes the circuit breaker agent (CBA) responsible for the relay operation. In a distribution system, the relay system installed on the busbar should be able to respond to different fault conditions and clear them quickly. As Figure 3 The figure shows an example of a multi-agent protection system in a distribution network with two distributed generation sources. Figure 3In the proposed system, each circuit breaker is controlled by a CBA, which cooperates with other CBAs to effectively detect and handle faults. The collaboration between agents improves the reliability of the system and prevents problems associated with single point failures. The system is modeled using six role-based agents, which are divided into master agents and slave agents based on their current roles. According to queuing theory, at different times, the role-based agents can be slave agents or master agents and can change their positions. Each role-based agent is placed in a queue to play the role of a master agent. Queuing theory is based on the first-in-first-out (FIFO) rule, and the first agent to enter the line will become the master agent in the first time interval. In the proposed system, each CBA continuously measures the parameters of the line for fault calculation and reports to the master agent. These parameters include current magnitude, phase difference between terminals, and power flow, and the master agent calculates these parameters and detects fault conditions. In order to detect faults more reliably, the Q-learning algorithm is used. In Q-learning, training data such as current magnitude, phase difference between terminals, and power flow are obtained from empirical evidence of defective and non-defective conditions. Based on the training data, Q-learning provides specific rules (such as trip / no trip) that should be executed by a specific CBA. In a traditional multi-agent system with only a single central master agent, if it fails, the entire system fails. In the proposed method, various agents can become the master agent at different time intervals. For example, at time t1, CBA1 can be the master agent, and at time t2, CBA2 becomes the master agent, and this process is repeated in a cycle.

[0076] The circuit breaker measures data related to power flow, such as the current magnitude and current phase angle at the branch terminal. This data is then stored and shared by the CBA. Once each current transformer shares data with the current master current transformer, it calculates the phase difference and current magnitude and detects whether a fault has occurred. In the event of a defect being identified, the master agent selects the corresponding agent based on the Q-learning decision tree and sends a trip request to that agent. The master agent calculates the difference in the current phase angles at both ends of the line, the current phase angle θ on one side of the terminal a ,θ b ,θ c The current phase shift is 120 degrees. a′ ,θ b′ ,θ c′ The difference in phase can be calculated using the following model:

[0077] Δθ a,b,c =∑ i＝1 (θ a,b,c (i)-θ a′b′c′ (i));

[0078] View Δθ a,b,cThe master agent can detect faults by using the parameters and current amplitude. The fault-free judgment condition is:

[0079]

[0080] The fault judgment conditions are:

[0081]

[0082] Among them, Δθ a,b,c is the phase angle difference of the current cycle, is the phase angle difference of the previous cycle;

[0083] If the phase angle difference between the starting end and the ending end of each line is much larger or smaller than the phase angle difference of the previous cycle, the situation is considered abnormal.

[0084] For current faults, when the current I exceeds the load current I load If it is more than 1.25 times of the

[0085] That is: |I|>I load ×1.25;

[0086] If a fault occurs in the line, the proxy will send a signal to the SBs on both sides of the section to isolate them from the fault.

[0087] The blockchain system is a distributed ledger that records data in individual blocks. Users have a copy of the blockchain, and any changes to the content or order of the blocks can be immediately canceled by checking the validity of the blocks. Such a system can also be used to improve the reliability of power system protection. In the proposed method, the blockchain data block is a table containing detailed information such as the phase angle, current magnitude, agent identification, and timestamp of the sending and receiving terminals of each line. The master agent then looks at the data in the blockchain and calculates the phase angle difference between the start and end terminals of each section. It then adds this information to its data block as a binary notification. When the data exchange is approved, the data containing the network status information is sent to the blockchain. Therefore, when the information block exchange is completed, the information of each CBA's data block at any point in time can be seen by other agents. In addition, the decisions reported by the master agent on the fault conditions associated with these blocks are also stored in the data blocks.

[0088] In the proposed system, the blockchain consists of information about the system parameters and the decisions associated with the settings. The master agent CBA of the current interval deploys the protocol through a P2P network. In the proposed method, all CBAs are responsible for developing a data block Di containing the system parameters. The data block Di contains information about the current amplitude, phase angle, binary notification to the agent and timestamp. Data block example: D3 = [60, {121, 119, 123}, 0, 1551455280], the data block can be interpreted as CBA3 reporting 60A current, 121, 119, 123 are the phase angles of the three phases, 0 means that CBA3 decided not to trip, and 1551395400 is the unix timestamp. The data block is public and can be used for training the Q learning algorithm classifier. The blockchain containing the data block is as follows: Figure 5 shown.

[0089] Q-learning has become a popular machine learning algorithm and has been widely used and paid attention to in recent years. In this embodiment, the learners are agents that know each other and can share information, and everything outside the agents is considered to be an environment. The agents use feedback from the environment to learn. Figure 4 shows a schematic diagram of the feedback loop used in Q-learning.

[0090] In Q-learning, models are created to generate decision trees in which the outputs are partially random under the control of the decision maker. Figure 4 As can be observed in , the agent interacts with the environment in discrete time steps. Then, at each time step, the agent receives a state S t And decide on a new operation A t In response to the agent’s actions, the environment gives the agent a reward R t+1 and a new state S t+1 The process of assigning rewards and states is repeated and produces a sequence trajectory.

[0091] In the proposed system, CBA interacts with each other at discrete time steps t = 0, 1, 2…n. The agent can take states S = S1, S2, S3…Sn and actions A = A1, A2, A3…An. At each discrete time, the agent obtains the reward for the current state from the environment, makes a decision and takes an appropriate action. In response to taking an action, the state of the environment will be changed to a new state. The agent will receive a new reward. If the fault is correctly identified, a reward is given. The model-level decision tree of the Q-learning algorithm is as follows:

[0092]

[0093] Among them, newQ(S t ,A t) is the new Q value based on state and action, α is the learning efficiency, r t is the reward based on state and action, γ is the discount factor, is the maximum future reward given a new state and action, Q(S t ,A t ) is the current Q value.

[0094] In the proposed protection system model, the agent learns how to isolate faults through a Q-learning algorithm. In this case, the environment is the distribution system. Empirical evidence shows that current amplitude and phase difference have a strong correlation with power system faults. Therefore, different combinations of current amplitude and phase difference are considered as a state. The current and phase difference combinations for different fault types are shown in Table 1.

[0095] Table 1 Current and phase difference combinations

[0096]

[0097] As can be seen from Table 1, higher current magnitude and higher phase difference usually indicate fault conditions. In terms of the Q-learning algorithm, a boundary is set between critical conditions and reasonable conditions to distinguish faulty conditions. For different CBAs in the system, the algorithm may set different limits, and sometimes complex decisions should be taken to isolate faults and reduce losses. When the phase difference is between 0 and 12 degrees, the state is called very low (VL), if the phase difference is between 12 and 16 degrees, it is called low (L), and if the phase difference is between 16 and 20 degrees, it is called medium (M). If it is above 20 degrees, it is severe (C). Based on the determined environmental state, including phase difference and current magnitude, appropriate actions are selected. In the proposed system, case-based reasoning is trained with the Q-learning algorithm. Then, the agents cooperate with each other and learn from each other to formulate specific actions according to the environment. In the four environments, each correct response and negative response has a reward. After eliminating the negative response impact and positive response, the model can also be applied to disaster recovery decisions. For the real decision output, a fuzzy learning algorithm can be used, and the algorithm can be continuously improved through repeated modifications and penalties.

[0098] Use the Java Agent Development Framework (JADE) to simulate the communication between agents. Figure 1 The proposed method is tested in a four-bus distribution network shown in Figure 1. All the agents and communication rules of the circuit breakers are developed in JADE software. The graphical user interface of JADE is shown in Figure 1. Figure 4As shown in the figure, the network interaction between CBAs is illustrated, where CBA1 is the master agent. In the proposed system, the distribution network can be divided into three different protection zones. In each zone, there are 2 circuit breakers located at both ends of the zone. The CBAs are trained for multiple time intervals using a Q-learning algorithm. With the help of supervised learning, the algorithm learns to distinguish between normal and fault conditions in the training data. The algorithm is then applied to multiple scenario analysis for line protection.

[0099] JADE graphical user interface diagram Figure 6 As shown:

[0100] 1) At time interval t1, CBA1 is the master agent and other CBAs are slave agents. CBA1, as the master agent in the current range, deploys the smart protocol to all other agents, requiring them to submit their parameters in the form of data blocks Di. All circuit breakers report parameters such as current magnitude and current phase angle and their decisions (trip / no trip). In this case, there is no fault and the master agent does not send request messages to other CBAs.

[0101] 2) During time interval t2, CBA2 is the second agent in the queue and becomes the master agent at this stage. Similarly, CBA2 deploys the smart protocol and submits the request for the data block. Because no failure is detected, the action request is not sent.

[0102] 3) During time interval t3, CBA3 is the master agent. Based on the Q-learning algorithm trained using historical data in the blockchain, the master agent concludes that there is a fault in the first area. Therefore, the agent sends an action request to CBA1 and CBA2 to perform a tripping operation. In this case, the fault can be isolated and the wind turbine is protected from damage.

[0103] 4) In time interval t4, CBA4 becomes the master and deploys smart channels through the P2P network to collect parameters from other CBAs. Based on the reported data and previous data, CBA4 concludes that the first area still has a fault, the fault has been isolated, and there is no fault in other areas.

[0104] 5) During time interval t5, CBA5 is the master agent. From the reported data, all areas are no longer faulty. Therefore, it requests CBA1 and CBA2 to change their status to normal shutdown.

[0105] 6) In the next interval t6, CBA6 is the master agent and concludes that there is a fault in the second zone. Therefore, it sends a trip request signal to CBA3 and CBA4 to isolate the fault.

[0106] In the next time interval, CBA1 first becomes the master agent, and the cycle continues. In the embodiment of the application, the master agent is the only decision maker. In future work, each CBA will be able to make its own decisions based on the Q-learning algorithm to significantly reduce the time required to clear different types of faults.

[0107] Using MATLAB Figure 1 The distribution system shown in the figure is simulated. The CBA controls the action of the circuit breaker and is modeled in MATLAB to record the current magnitude and phase difference. In JADE, the CBA is modeled as sending information or request data blocks, where the notification data block Di contains the system parameters and the request data block to trip a specific circuit breaker. MATLAB Simulink is connected to the JADE software through the TCP / IP protocol. The fault model is simulated in MATLAB according to the following properties:

[0108] 1) The current waveform at both ends of the three-phase line without fault is as follows Figure 7a As shown, the phase angle difference between both ends of all lines is less than 20 degrees.

[0109] 2) The current waveform at both ends of the three-phase line under single-phase fault is as follows Figure 7b As shown, it can be seen that the phase difference between the two ends of the A phase line exceeds 20 degrees, the phase difference between phases B and C is less than 20 degrees, the fault recognition time is 0.1ms, and the fault isolation time is 0.15ms.

[0110] 3) The current waveform at both ends of the three-phase line under two-phase fault is as follows Figure 7c In this case, the phase difference between phase A and phase B is greater than 20 degrees, while the phase difference of phase C is less than 20 degrees, the fault recognition time is 0.12ms, and the fault isolation time is 0.18ms.

[0111] 4) The current waveform at both ends of the three-phase line under a three-phase fault is as follows Figure 7d In this case, the phase difference of the three phases is higher than 20 degrees, the fault recognition time is 0.1ms, and the fault isolation time is 0.15ms.

[0112] In this case study, only faults related to phase difference are considered. In future work, other characteristics that have statistical correlation with the error, such as current amplitude and input impedance, will be used.

[0113] The embodiment of the present application proposes a distribution network protection method based on Q learning and blockchain, which can identify and isolate different types of faults. The Q learning algorithm is used to train the execution agent to make reliable decisions in the process of fault identification and clearance. In addition, the agents use a decentralized connection based on blockchain to exchange information. In the proposed method, the circuit breaker sends data to the circuit breaker agent, and multiple agents monitor the operation of the system in real time and coordinate with each other. The agent sends the data to the blockchain data block through an intelligent protocol, and Q learning technology is used to make correct decisions on the fault. The test system was modeled and the protection method was simulated on the MATLAB and JADE platforms. The results show that the method has high predictability and processing speed for multiple types of faults. The decision boundaries of fault and non-fault cases are formulated and tested using Q learning. The test results verify the effectiveness of the proposed method.

[0114] The embodiment of the present application also provides a distribution protection device based on Q learning and blockchain. It should be noted that the distribution protection device based on Q learning and blockchain in the embodiment of the present application can be used to execute the distribution protection method based on Q learning and blockchain provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been described will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0115] The following introduces the distribution protection device based on Q learning and blockchain provided in an embodiment of the present application.

[0116] Figure 8 is a structural block diagram of a power distribution protection device based on Q learning and blockchain according to an embodiment of the present application. Figure 8 As shown, the device includes: a first acquisition unit 81, a control unit 82 and a second acquisition unit 83;

[0117] The first acquisition unit 81 is used to acquire training data, and use the Q learning algorithm to process the training data to obtain a Q learning decision tree model, wherein the processing follows preset rules, wherein the training data includes normal data and fault data, and the preset rules at least include the correspondence between the agent module and the system component; specifically, in order to detect faults more reliably, the Q learning algorithm is used. In Q learning, training data is obtained from empirical evidence of defective and non-defective conditions, such as current magnitude, phase difference between terminals, and power flow. Based on the training data, Q learning follows preset rules to generate a Q learning decision tree model, wherein the preset rules also include whether to perform tripping processing based on the data.

[0118] The control unit 82 is used to input the collected data of multiple system components obtained from the multiple slave agent modules into the data block of the blockchain, control the master agent module to calculate the collected data in the data block, and obtain the fault information in the collected data, wherein the collected data includes the current value, current phase angle and power flow of the system components; specifically, the blockchain system is a distributed ledger that records the data of each block. The user has a copy of the blockchain, and any change in the content or order of the block can be immediately canceled by checking the validity of the block. Such a system can also be used to improve the reliability of power system protection. In the proposed method, the blockchain data block is a table containing detailed information such as the phase angle, current magnitude, agent identification and timestamp of the sending and receiving terminals of each line. Then, the master agent checks the data in the blockchain and calculates the phase angle difference between the start and end terminals of each part. Then, it adds this information to its data block as a binary notification. When the data exchange is approved, the data containing the network status information is sent to the blockchain. Therefore, when the information block exchange is completed, the information of each agent's data block at any point in time can be seen by other agents. In addition, the decisions reported by the master agent on the fault conditions associated with these blocks are also stored in the data block.

[0119] The second acquisition unit 83 is used to acquire the target slave agent module determined by the master agent module according to the fault information and the Q learning decision tree model, and control the master agent module to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request. Specifically, the blockchain consists of information about system parameters and decisions associated with the settings. The master agent CBA of the current interval deploys the protocol through the P2P network. In the proposed method, all CBAs are responsible for developing a data block D containing system parameters. i Data Block D i Contains information about the current amplitude, phase angle, binary notification to the agent, and timestamp. Data block example: D3 = [60, {121, 119, 123}, 0, 1551455280], the data block can be interpreted as CBA3 reporting 60A current, 121, 119, 123 are the phase angles of the three phases, 0 means CBA3 decided not to trip, and 1551395400 is the unix timestamp.

[0120] In this embodiment, the first acquisition unit is used to acquire training data, and the training data is processed by the Q learning algorithm to obtain the Q learning decision tree model, wherein the processing process follows the preset rules; the control unit is used to input the collected data of multiple system components obtained by multiple slave agent modules into the data block of the blockchain, control the master agent module to calculate the collected data in the data block, and obtain the fault information in the collected data; the second acquisition unit is used to acquire the target slave agent module determined by the master agent module according to the fault information and the Q learning decision tree model, and control the master agent module to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request. The Q learning algorithm is used to train the execution agent to make reliable decisions in the process of fault identification and clearing, and the Q learning algorithm improves the reliability and efficiency of the relay agent. In addition, the agents use a decentralized connection based on the blockchain to exchange information, which solves the problem that the distribution system of the prior art is inflexible and cannot cope with different types of faults.

[0121] As an optional solution, the device also includes, according to the first-in-first-out rule of queuing theory, the slave proxy module located at the Mth entry line is the master proxy module in the Mth time interval, M≥1; in the Nth time interval, the slave proxy module entering the line at the Nth time interval is the master proxy module; in the N+1th time interval, the first slave proxy module entering the line is the master proxy module, wherein N≥2, and N is the total number of slave proxy modules.

[0122] In an optional solution, the control unit includes an input module and a calculation module; the input module is used to input the current phase angle data of multiple system components obtained from the agent module into the data block of the blockchain; the calculation module is used to calculate the current phase angle data in the data block using the fault model in the main agent module to obtain the phase angle difference of the multiple system components, and generate fault information when the phase angle difference is greater than or less than the phase angle difference of the previous cycle, and do not generate fault information when the phase angle difference is equal to the phase angle difference of the previous cycle.

[0123] In an optional solution, the device also includes a first sending unit and a second sending unit; the first sending unit is used to send a feedback signal to the master agent module after controlling the master agent module to send a tripping request to the target slave agent module so that the target slave agent module performs a tripping operation according to the tripping request, wherein the feedback signal includes a reward signal and a penalty signal; the second sending unit is used to send a reward signal to the master agent module when the tripping operation is correct, and send a penalty signal to the master agent module when the tripping operation is incorrect.

[0124] An optional solution, the device includes a processing unit, which is used to use a fuzzy learning algorithm to improve the Q learning decision tree model after sending a penalty signal to the main agent module in the event of a tripping operation error, so as to obtain an improved Q learning decision tree model.

[0125] An optional solution, the device also includes a first adopting unit and a second adopting unit, the first adopting unit is used to use the Java agent development framework JADE to simulate the communication between the master agent module and the slave agent module, and determine whether the communication between the master agent module and the slave agent module is normal; the second adopting unit is used to use the MATLAB platform to simulate the distribution protection system, and determine whether the fault model accurately identifies the fault information.

[0126] An optional solution, the Q-learning decision tree model is:

[0127]

[0128] Among them, newQ(S t ,A t ) is the new Q value based on state and action, α is the learning efficiency, r t is the reward based on state and action, γ is the discount factor, is the maximum future reward given a new state and action, Q(S t ,A t ) is the current Q value.

[0129] The distribution protection device based on Q learning and blockchain includes a processor and a memory. The first acquisition unit, the control unit, the second acquisition unit, etc. are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The above modules are all located in the same processor; or, the above modules are located in different processors in any combination.

[0130] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem that the power distribution system of the prior art is inflexible and cannot cope with different types of faults can be solved by adjusting kernel parameters.

[0131] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0132] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the distribution protection method based on Q learning and blockchain.

[0133] Specifically, the distribution protection method based on Q learning and blockchain includes:

[0134] Step S201, obtaining training data, and processing the training data using a Q learning algorithm to obtain a Q learning decision tree model, wherein the processing follows preset rules, wherein the training data includes normal data and fault data, and the preset rules at least include a correspondence between a slave agent module and a system component;

[0135] Step S202, inputting the collected data of multiple system components obtained from the multiple agent modules into the data block of the blockchain, controlling the main agent module to calculate the collected data in the data block, and obtaining fault information in the collected data, wherein the collected data includes the current value, current phase angle and power flow of the system components;

[0136] Step S203, obtaining the target slave agent module determined by the master agent module according to the fault information and the Q learning decision tree model, and controlling the master agent module to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request.

[0137] An embodiment of the present invention provides a processor, which is used to run a program, wherein the distribution protection method based on Q learning and blockchain is executed when the program is running.

[0138] An embodiment of the present invention provides an electronic device, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements a power distribution protection method based on Q learning and blockchain when executing the program. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0139] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing a distribution protection method based on Q learning and blockchain.

[0140] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0141] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0143] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0145] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0146] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0147] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0148] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0149] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A distribution protection method based on Q learning and blockchain, characterized in that: The method is applied to a controller of a multi-agent power distribution protection system, wherein the power distribution protection system further includes a master agent module, multiple slave agent modules and multiple system components, and the method includes: Acquire training data, and process the training data using a Q learning algorithm to obtain a Q learning decision tree model, wherein the processing follows a preset rule, wherein the training data includes normal data and fault data, and the preset rule includes at least a corresponding relationship between the slave agent module and the system component; Inputting the collected data of the plurality of system components obtained by the plurality of slave agent modules into a data block of the blockchain, controlling the master agent module to calculate the collected data in the data block, and obtaining fault information in the collected data, wherein the collected data includes current value, current phase angle and power flow of the system components; The target slave agent module is obtained by the master agent module according to the fault information and the Q learning decision tree model, and the master agent module is controlled to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request.

2. The method according to claim 1, characterized in that The method further comprises: According to the first-in-first-out rule of queuing theory, the slave proxy module located at the Mth entry line is the master proxy module at the Mth time interval, M≥1; In the Nth time interval, the slave proxy module that enters the line the Nth time interval is the master proxy module; in the N+1th time interval, the slave proxy module that first enters the line is the master proxy module, wherein N≥2, and N is the total number of the slave proxy modules.

3. The method according to claim 1, characterized in that Inputting the collected data of the plurality of system components obtained by the plurality of slave agent modules into a data block of the blockchain, controlling the master agent module to calculate the collected data in the data block, and obtaining fault information in the collected data, including: Inputting the current phase angle data of the plurality of system components obtained from the agent module into the data block of the blockchain; The fault model in the master-agent module is used to calculate the current phase angle data in the data block to obtain the phase angle difference of the multiple system components, and the fault information is generated when the phase angle difference is greater than or less than the phase angle difference of the previous cycle, and the fault information is not generated when the phase angle difference is equal to the phase angle difference of the previous cycle.

4. The method according to claim 1, characterized in that After controlling the master agent module to send a trip request to the target slave agent module so that the target slave agent module performs a trip operation according to the trip request, the method further includes: Sending a feedback signal to the master agent module, wherein the feedback signal includes a reward signal and a penalty signal; In the case where the tripping operation is correct, a reward signal is sent to the master agent module, and in the case where the tripping operation is wrong, a penalty signal is sent to the master agent module.

5. The method according to claim 4, characterized in that In case of the tripping operation error, after sending a penalty signal to the master agent module, the method further comprises: The Q-learning decision tree model is improved by using a fuzzy learning algorithm to obtain an improved Q-learning decision tree model.

6. The method according to claim 1, characterized in that The method further comprises: Using the Java agent development framework JADE to simulate the communication between the master agent module and the slave agent module, and determine whether the communication between the master agent module and the slave agent module is normal; The power distribution protection system is simulated using the MATLAB platform to determine whether the fault model in the master-agent module accurately identifies the fault information.

7. The method according to any one of claims 1 to 6, characterized in that: The Q-learning decision tree model is: Among them, newQ(S t ,A t ) is the new Q value based on state and action, α is the learning efficiency, r t is the reward based on state and action, γ is the discount factor, is the maximum future reward given a new state and action, Q(S t ,A t ) is the current Q value.

8. A distribution protection device based on Q learning and blockchain, characterized in that: The device is applied to a controller of a multi-agent power distribution protection system, wherein the power distribution protection system further includes a master agent module, multiple slave agent modules and multiple system components, and the device includes: A first acquisition unit is used to acquire training data, and process the training data using a Q learning algorithm to obtain a Q learning decision tree model, wherein the processing follows a preset rule, wherein the training data includes normal data and fault data, and the preset rule at least includes a corresponding relationship between the slave agent module and the system component; A control unit, configured to input the collected data of the plurality of system components obtained by the plurality of slave agent modules into a data block of the blockchain, control the master agent module to calculate the collected data in the data block, and obtain fault information in the collected data, wherein the collected data includes a current value, a current phase angle, and a power flow of the system components; The second acquisition unit is used to acquire the target slave agent module determined by the master agent module according to the fault information and the Q learning decision tree model, and control the master agent module to send a trip request to the target slave agent module, so that the target slave agent module performs a trip operation according to the trip request.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the distribution protection method based on Q learning and blockchain as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing the Q-learning and blockchain-based distribution protection method described in any one of claims 1 to 7.

Images