Power system active power correction control method and device based on man-machine interaction

Abstract

The invention relates to the technical field of power system active correction control, and discloses a power system active correction control method and device based on man-machine interaction, and the method comprises the steps: building a power system dispatcher cognitive model, and determining dispatcher decision data through the power system dispatcher cognitive model; obtaining power system network topology data, and training the deep reinforcement learning model based on the power system network topology data and the dispatcher decision data to obtain a power system man-machine interaction model; and generating a man-machine co-fusion decision by using the power system man-machine interaction model, and performing power system active correction control based on the man-machine co-fusion decision. According to the invention, the reliability of man-machine co-fusion decision and the efficiency and accuracy of active correction control of the power system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of active power correction control of electric power systems, and in particular to a method and a device for active power correction control of electric power systems based on human-computer interaction. Background Art

[0002] Due to the in-depth application of artificial intelligence technology, the active power correction control of the power system has become a human-machine hybrid system. However, there is a lack of research on human-machine interaction and collaboration in the power grid control system. Therefore, how to combine the computing power of the machine with the cognitive ability of humans is a key problem that the human-machine hybrid intelligent system needs to solve. Summary of the invention

[0003] In view of this, the present invention provides a method and device for active power correction control of an electric power system based on human-computer interaction, so as to solve the problem that active power correction control of an electric power system lacks research on human-computer interaction and collaboration of a power grid regulation system.

[0004] In a first aspect, the present invention provides a method for active power correction control of an electric power system based on human-computer interaction, the method comprising:

[0005] Establish a cognitive model for power system dispatchers and use it to determine the dispatcher's decision data;

[0006] Obtain the power system network topology data, train the deep reinforcement learning model based on the power system network topology data and the dispatcher's decision data, and obtain the power system human-computer interaction model;

[0007] The human-machine interaction model of the power system is used to generate human-machine collaborative decisions, and active power correction control of the power system is performed based on the human-machine collaborative decisions.

[0008] The present embodiment provides an active power correction control method for an electric power system based on human-computer interaction. The method uses a cognitive model of an electric power system dispatcher to determine the dispatcher's decision data. A deep reinforcement learning model is trained based on the electric power system network topology data and the dispatcher's decision data to obtain an electric power system human-computer interaction model. The electric power system human-computer interaction model is used to generate a human-computer collaborative decision. Then, active power correction control is performed on the electric power system based on the human-computer collaborative decision. In the process of training the deep reinforcement learning model, human intelligence and machine intelligence are effectively integrated to achieve human-computer hybrid intelligence enhancement, thereby improving the reliability of human-computer collaborative decision-making and the efficiency and accuracy of active power correction control of the electric power system.

[0009] In an optional implementation, a power system dispatcher cognitive model is established, and the power system dispatcher cognitive model is used to determine the dispatcher decision data, including:

[0010] Establishing the cognitive model of power system dispatchers by Bayesian network;

[0011] Obtain the factors affecting the human reliability of dispatchers in the power system under the target scenario, and obtain the human reliability analysis results using the cognitive model of power system dispatchers based on the factors affecting the human reliability of dispatchers;

[0012] Determine the dispatcher's decision data based on the results of human reliability analysis.

[0013] The present embodiment provides an active power correction control method for an electric power system based on human-computer interaction. The method obtains the human reliability analysis result by using the cognitive model of the electric power system dispatcher through the influencing factors of the human reliability of the dispatcher, and then determines the decision data of the dispatcher based on the human reliability analysis result, thereby realizing accurate analysis of the human reliability, making the decision data of the dispatcher more accurate, and realizing accurate prediction of the dispatcher's behavior.

[0014] In an optional implementation, determining the dispatcher decision data based on the human reliability analysis result includes:

[0015] Obtain the dispatcher's decision set, and use the entropy weight method to determine the weight of each action in the dispatcher's decision set;

[0016] Calculate the dispatcher's decision probability based on the human reliability analysis results and the weight of each action;

[0017] The dispatcher decision data is selected based on the dispatcher decision probability.

[0018] The present embodiment provides an active power correction control method for an electric power system based on human-computer interaction. The method calculates the dispatcher's decision probability based on the human reliability analysis results and the weight of each action, thereby achieving accurate analysis of each dispatcher's decision, and selecting the dispatcher's decision data based on the dispatcher's decision probability, thereby simplifying the steps for selecting the dispatcher's decision data, realizing the reliability analysis of the cognitive model of the electric power system dispatcher, and providing a data basis for the subsequent training of the deep reinforcement learning model.

[0019] In an optional implementation, a deep reinforcement learning model is trained based on the power system network topology data and the dispatcher decision data to obtain a power system human-computer interaction model, including:

[0020] Based on the power system network topology data, the state space, action space and reward function of the active power correction control of the power system are constructed using the Markov decision process.

[0021] The state space, action space and reward function of the active power correction control of the power system and the decision data of the dispatcher are stored in the sample data pool;

[0022] Multiple sample data are randomly extracted from the sample data pool, and the deep reinforcement learning model is trained using the multiple sample data to obtain the power system human-computer interaction model.

[0023] This embodiment provides an active power correction control method for an electric power system based on human-computer interaction. The state space, action space and reward function of the active power correction control, as well as the dispatcher's decision data, are used as sample data to train a deep reinforcement learning model, thereby combining the dispatcher's professional knowledge and experience with intelligent algorithms to achieve more effective and reliable active power correction control.

[0024] In an optional implementation, a plurality of sample data are randomly extracted from a sample data pool, and a deep reinforcement learning model is trained using the plurality of sample data to obtain a power system human-computer interaction model, including:

[0025] Randomly select the state, action and reward function at a preset time step from the sample data pool, input the state, action and reward function at the preset time step into the deep reinforcement learning model, and obtain the single-step deep Q learning loss function value and the n-step deep Q learning loss function value;

[0026] The actions corresponding to the dispatcher's decision data are input into the deep reinforcement learning model to obtain the expert strategy loss function value;

[0027] Obtain the weight parameters of the deep reinforcement learning model, and calculate the L2 regularization loss function value based on the weight parameters of the deep reinforcement learning model;

[0028] Determine a total loss function value based on a single-step deep Q-learning loss function value, an n-step deep Q-learning loss function value, an expert strategy loss function value, and an L2 regularization loss function value;

[0029] The total loss function value is compared with the preset threshold, and the deep reinforcement learning model is iteratively trained based on the comparison result until the total loss function value is less than the preset threshold. The iteration is stopped to obtain the power system human-computer interaction model.

[0030] The present embodiment provides a method for active power correction control of an electric power system based on human-computer interaction, which combines the dispatcher's decision data with reinforcement learning, and uses the dispatcher's decision data for pre-training. The loss function related to deep Q learning, namely the single-step deep Q learning loss function value and the n-step deep Q learning loss function value, ensures that the intelligent agent can perform autonomous optimization from time difference learning. The expert strategy loss function value guides the intelligent agent to learn and approximate the demonstrator strategy, namely the dispatcher's decision strategy. The L2 regularization loss function value ensures the generalization ability of the intelligent agent by limiting the weight size of the Q network, thereby obtaining a better initial strategy after pre-training, which can improve the problem of low data utilization in mobile decision-making based on reinforcement learning.

[0031] In an optional implementation, the action corresponding to the dispatcher's decision data is input into the deep reinforcement learning model to obtain the expert strategy loss function value; wherein the calculation formula of the expert strategy loss function value is as follows:

[0032]

[0033] In the above formula, L E (θ) represents the expert strategy loss function value, Q(s,a) represents the Q value of action a in state s, represents the action space, l(a E ,a) represents the edge supervision loss function, a E represents the action corresponding to the dispatcher's decision data, Q(s,a E ) means that in the s state, a E The Q value of the action.

[0034] In a second aspect, the present invention provides an active power correction control device for an electric power system based on human-computer interaction, the device comprising:

[0035] A cognitive module is used to establish a cognitive model of a power system dispatcher and determine the dispatcher's decision data using the cognitive model of the power system dispatcher;

[0036] The human-computer intelligent interaction module is used to obtain the power system network topology data, train the deep reinforcement learning model based on the power system network topology data and the dispatcher's decision data, and obtain the power system human-computer interaction model;

[0037] The control module is used to generate a human-machine collaborative decision by utilizing a human-machine interactive model of the power system, and to perform active power correction control of the power system based on the human-machine collaborative decision.

[0038] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to thereby execute an active power correction control method for an electric power system based on human-computer interaction according to the first aspect or any corresponding embodiment thereof.

[0039] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute an active power correction control method for an electric power system based on human-computer interaction according to the first aspect or any corresponding embodiment thereof.

[0040] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute an active power correction control method for an electric power system based on human-computer interaction according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is a flow chart of a method for active power correction control of an electric power system based on human-computer interaction according to an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of an integrated human-machine intelligent interface and cognitive behavior experimental platform according to an embodiment of the present invention;

[0044] Figure 3 is a schematic diagram of a human-machine collaboration process for active power correction control of a power system according to an embodiment of the present invention;

[0045] Figure 4 is a schematic diagram of a basic human-machine interaction and collaboration mode framework in a power system active power correction control process according to an embodiment of the present invention;

[0046] Figure 5 is a flow chart of another method for active power correction control of an electric power system based on human-computer interaction according to an embodiment of the present invention;

[0047] Figure 6 is a schematic diagram of the overall flow of a questionnaire survey according to an embodiment of the present invention;

[0048] Figure 7 is a flow chart of another method for active power correction control of an electric power system based on human-computer interaction according to an embodiment of the present invention;

[0049] Figure 8 is a schematic diagram of a machine intelligence enhancement architecture for complex power system control that integrates dispatching rules and knowledge and experience according to an embodiment of the present invention;

[0050] Fig. 9 is a structural block diagram of an active power correction control device for an electric power system based on human-computer interaction according to an embodiment of the present invention;

[0051] Fig.10 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0053] The embodiment of the present invention provides an active power correction control method for an electric power system based on human-computer interaction. Based on the active power correction control problem of the electric power system, an integrated power grid auxiliary decision-making human-computer hybrid intelligent interaction, collaboration and cognitive behavior experimental platform under the active power correction control scenario is established. Through the effective integration of human intelligence and machine intelligence, human-computer hybrid intelligence enhancement is realized, thereby improving the reliability of human-computer collaborative decision-making and improving the efficiency and accuracy of active power correction control.

[0054] According to an embodiment of the present invention, an embodiment of an active power correction control method of an electric power system based on human-computer interaction is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0055] In this embodiment, a method for active power correction control of a power system based on human-computer interaction is provided, which can be used for an integrated human-computer intelligent interface and cognitive behavior experimental platform. Figure 1 is a flow chart of a method for active power correction control of a power system based on human-computer interaction according to an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0056] Step S101, establishing a cognitive model for a power system dispatcher, and using the cognitive model for the power system dispatcher to determine the dispatcher's decision data.

[0057] Step S102, obtaining the power system network topology data, training the deep reinforcement learning model based on the power system network topology data and the dispatcher decision data, and obtaining the power system human-computer interaction model.

[0058] Specifically, Figure 2As shown in the figure, a deep reinforcement learning model is constructed for the active power correction control process of the power system. The training environment comes from the open source power system test platform Grid2Op. Grid2Op can simulate the impact of maintenance, danger, unplanned downtime and attack, and can obtain continuous and discrete observation space; considering the fluctuation and seasonal characteristics of source and load, the data set of the target power grid system includes 48 years of power grid data, in which each scenario contains 28 consecutive days of data, and pre-dispatching has been carried out in all scenarios to meet the power and electricity balance; on this basis, considering the random disturbance of the system, each scenario will suffer from "N-1" line faults with random time and location every day. At the same time, the lines in the example system also have planned outages for inspection and maintenance. The goal of the intelligent agent is to perform sequential active power correction control in the example system, eliminate and alleviate the line over-limit situation under "N-1" faults and source and load fluctuations, and maintain the stable operation of the system.

[0059] Furthermore, taking the active power correction control scenario of the power system as a typical human-machine hybrid system, a human-machine interaction model of the power system is constructed.

[0060] Furthermore, if Figure 3 As shown in the figure, the basic mode of human-machine collaboration based on interpretable technology in the active power correction control of the power system, compared with the "open-loop" application mode of pure data-driven artificial intelligence model in training-testing-deployment, the hybrid enhanced intelligent system adopts a more intelligent and efficient application mode, namely the "closed-loop" collaborative application mode of human-machine interpretable interaction-human intervention-human-machine collaboration; in this closed-loop mode, the dispatcher and the intelligent agent can establish a close cooperative relationship: first, the dispatcher can deeply understand the decision-making process and reasoning logic of the artificial intelligence model through interpretable interaction with the intelligent agent. This interaction can help the dispatcher better understand the decision-making results of the artificial intelligence model and increase the trust in its reliability and rationality; secondly, when the dispatcher finds problems or needs to intervene in the process of active power correction control, he can adjust and optimize it by intervening in the workflow of the machine; the dispatcher's professional knowledge and experience can be combined with the artificial intelligence algorithm to achieve more effective and reliable active power correction control. This human-machine collaboration mode makes the active power correction control process more flexible and adaptable, and can cope with complex and changeable power grid conditions.

[0061] Furthermore, for the active power correction control task of the power system, a graph deep reinforcement learning model is constructed to realize the training and deployment process of the model. Through interpretable technology and virtual digital human technology, the interaction between the results of machine intelligence and the dispatcher can be further realized, such as Figure 4As shown in the figure, human-machine collaborative decision-making is the integration of data in machine intelligence and knowledge of human dispatchers. The overall active power correction control goal is achieved through the execution of human-machine collaborative decision-making. In the active power correction control task of the power system, the dispatcher uses the interpretable method of the artificial intelligence model to clarify the inherent decision-making mechanism of the model, and further obtains decision-making information by understanding the key features of the model input, so as to better evaluate the risks and potential consequences of the decision.

[0062] Furthermore, in terms of hybrid intelligence construction, deep reinforcement learning methods can learn the best decisions by interacting with the environment, that is, human experts and prior knowledge often help improve the ability of reinforcement learning to cope with complex environments, ensuring that reinforcement learning is constrained by experience knowledge during operation, so that it can interact with the environment safely and efficiently. In order to better combine human experience with machine intelligence, it is proposed to use human knowledge to guide reinforcement learning training and teaching learning, that is, to convert human experience and historical expert operations into a "scenario-decision" set, and guide the teaching learning model to integrate human experience in training by adding supervised losses related to prior strategies.

[0063] Step S103: Generate a human-machine collaborative decision using the human-machine interactive model of the power system, and perform active power correction control of the power system based on the human-machine collaborative decision.

[0064] The present embodiment provides an active power correction control method for an electric power system based on human-computer interaction. The method uses a cognitive model of an electric power system dispatcher to determine the dispatcher's decision data. A deep reinforcement learning model is trained based on the electric power system network topology data and the dispatcher's decision data to obtain an electric power system human-computer interaction model. The electric power system human-computer interaction model is used to generate a human-computer collaborative decision. Then, active power correction control is performed on the electric power system based on the human-computer collaborative decision. In the process of training the deep reinforcement learning model, human intelligence and machine intelligence are effectively integrated to achieve human-computer hybrid intelligence enhancement, thereby improving the reliability of human-computer collaborative decision-making and the efficiency and accuracy of active power correction control of the electric power system.

[0065] In this embodiment, a method for active power correction control of a power system based on human-computer interaction is provided, which can be used for the above-mentioned integrated human-computer intelligent interface and cognitive behavior experimental platform. Figure 5 is a flow chart of a method for active power correction control of a power system based on human-computer interaction according to an embodiment of the present invention, such as Figure 5 As shown, the process includes the following steps:

[0066] Step S501, establishing a cognitive model for a power system dispatcher, and using the cognitive model for the power system dispatcher to determine the dispatcher's decision data.

[0067] Specifically, the above step S501 includes:

[0068] Step S5011, establishing a cognitive model of power system dispatchers by using a Bayesian network.

[0069] Specifically, the cognitive model of power system dispatchers is used to predict the behavior of dispatchers (predicting the behavior of dispatchers in making dispatch decisions through factors such as individuals, environments, equipment, tasks, and interactive interfaces), pre-training and adaptive interactions, and mainly includes cognitive reliability and error analysis methods; among them, the cognitive model of power system dispatchers includes a dispatcher state assessment process model based on a Bayesian network and a response plan and execution process behavior model based on influencing factors; the relationship between various factors is simulated through a Bayesian network, and a conditional probability table is used to specifically describe the joint probability distribution between various factors. This structure enables the Bayesian network to efficiently estimate the probability of system failure in the case of specific variable nodes or multiple variable nodes.

[0070] Step S5012, obtaining the influencing factors of the human reliability of the dispatcher in the power system under the target scenario, and based on the influencing factors of the human reliability of the dispatcher, obtaining the human reliability analysis results by using the cognitive model of the power system dispatcher.

[0071] Specifically, various influencing factors affecting the dispatcher's decision-making state (i.e., factors affecting dispatcher human reliability) were analyzed and screened, and 5 types of first-level indicators and 17 types of second-level indicators were selected, as shown in Table 1 below.

[0072] Table 1:

[0073]

[0074]

[0075]

[0076] Furthermore, if Figure 6 As shown in Table 1, two groups of control experiments were designed based on the contents of the questionnaire. The experimenters will perform the same tasks in the active power correction control scenario. Each dispatcher will receive a separate face-to-face structured interview and semi-structured interview. The questionnaire and control experiment will help to gain a deeper understanding of the dispatchers' cognitive process, decision-making state and influencing factors when facing active power correction tasks. The specific contents of the control experiment are shown in Table 2.

[0077] Table 2:

[0078]

[0079] Furthermore, during the questionnaire survey and control experiment, there were large differences in the status of influencing factors among different experimenters. Through the interview results of the experimenters, the indicator status of each influencing factor was quantified, and the success of the power system dispatching task was judged by whether the power system flow converged after the dispatcher's decision was executed, and whether the transmission line over-limit could be eliminated in time; in the case of dispatching task failure, special attention was paid to the factors whose status was marked as "poor" among all monitored influencing factors. The influencing factors with a status of "poor" were considered to be more closely related to the failure of the dispatching task, and then the influencing factors of the human reliability of the dispatcher in the power system under the target scenario were obtained; by statistically analyzing the questionnaire results data of the experimenters, the influence weight of each influencing factor when the system failed due to human error can be obtained; a Bayesian network was established through the weights of each influencing factor, and the Bayesian network can be used to evaluate the status of the dispatcher, and then a comprehensive cognitive model of power system dispatchers is constructed.

[0080] Step S5013, determining the dispatcher decision data based on the human reliability analysis results.

[0081] In some optional implementations, the above step S5023 includes:

[0082] Step a1, obtain the dispatcher decision set, and use the entropy weight method to determine the weight of each action in the dispatcher decision set.

[0083] Specifically, the human reliability of two groups of experiments was obtained through questionnaires. The ordinary human reliability set was obtained through Experiment I, and the human reliability set with the influence of machine intelligence was obtained through Experiment II. The entropy weight method was used to obtain the control plan (i.e., the dispatcher decision set) and the action set was obtained by the entropy weight method. The weight of each plan (i.e., the dispatcher's decision data) in the .

[0084] Furthermore, since the element action index λ a is a negative indicator, the line load rate λ ρ is a positive indicator. In order to obtain the indicator proportion, the indicator in the dispatcher's decision plan is normalized. The normalized expression is as follows:

[0085]

[0086] Among them, d represents the d-th decision action, and Respectively represent the element action index λ a and line load factor λ ρ After standardization, the weight is determined according to the variability of the indicator.

[0087] Furthermore, the proportion of each indicator in the dth decision plan is calculated based on the normalized indicator value. The calculation formula is as follows:

[0088]

[0089] Among them, N d The number of actions corresponding to all dispatcher decisions, It represents the proportion of the g-th indicator in the d-th decision plan.

[0090] Furthermore, the information entropy E of each indicator is calculated according to the proportion of each indicator in the dth decision plan. g , and its calculation formula is as follows:

[0091]

[0092] Furthermore, through the information entropy E of each indicator g Get the element action index λ a and line load factor λ ρ The respective weights are expressed as:

[0093]

[0094] Among them, N g Indicates the number of target indicators. Indicates the indicator weight.

[0095] Furthermore, through the indicator weights and the specific indicator values ​​in all decision actions The proportion of actions that can make decisions is W, and the top 5 actions with the largest proportion are selected to obtain the optimal action set The calculation formula for the proportion W of decision actions is as follows:

[0096]

[0097] Step a2, calculate the dispatcher's decision probability based on the human reliability analysis results and the weight of each action.

[0098] Specifically, through the optimal action set The weight W of each action in the game is related to the reliability set of ordinary people After multiplication and normalization, we get the probability set of the dispatcher's decision, the probability of the dispatcher's decision The calculation formula is as follows:

[0099]

[0100] Step a3, selecting the dispatcher decision data based on the dispatcher decision probability.

[0101] Specifically, the dispatcher decision data are sorted from large to small according to the dispatcher decision probability set, and the dispatcher decision data within a preset ranking range (for example, the first or top three) are selected.

[0102] The present embodiment provides an active power correction control method for an electric power system based on human-computer interaction. The method calculates the dispatcher's decision probability based on the human reliability analysis results and the weight of each action, thereby achieving accurate analysis of each dispatcher's decision, and selecting the dispatcher's decision data based on the dispatcher's decision probability, thereby simplifying the steps for selecting the dispatcher's decision data, realizing the reliability analysis of the cognitive model of the electric power system dispatcher, and providing a data basis for the subsequent training of the deep reinforcement learning model.

[0103] Step S502: Obtain the power system network topology data, train the deep reinforcement learning model based on the power system network topology data and the dispatcher decision data, and obtain the power system human-computer interaction model. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0104] Step S503: Generate a human-machine collaborative decision using the human-machine interactive model of the power system, and perform active power correction control of the power system based on the human-machine collaborative decision. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0105] The present embodiment provides an active power correction control method for an electric power system based on human-computer interaction. The method obtains the human reliability analysis result by using the cognitive model of the electric power system dispatcher through the influencing factors of the human reliability of the dispatcher, and then determines the decision data of the dispatcher based on the human reliability analysis result, thereby realizing accurate analysis of the human reliability, making the decision data of the dispatcher more accurate, and realizing accurate prediction of the dispatcher's behavior.

[0106] In this embodiment, a method for active power correction control of a power system based on human-computer interaction is provided, which can be used for the above-mentioned integrated human-computer intelligent interface and cognitive behavior experimental platform. Figure 7 is a flow chart of a method for active power correction control of a power system based on human-computer interaction according to an embodiment of the present invention, such as Figure 7 As shown, the process includes the following steps:

[0107] Step S701: Establish a cognitive model for power system dispatchers, and use the cognitive model to determine the dispatcher's decision data. Figure 5 Step S501 of the illustrated embodiment will not be described in detail here.

[0108] Step S702, obtaining the power system network topology data, training the deep reinforcement learning model based on the power system network topology data and the dispatcher decision data, and obtaining the power system human-computer interaction model.

[0109] Specifically, the above step S702 includes:

[0110] Step S7021, based on the power system network topology data, using the Markov decision process to construct the state space, action space and reward function of the active power correction control of the power system.

[0111] Specifically, considering the decision sensitivity of high-dynamic systems and the rich information interaction relationship between their internal variables, a reinforcement learning method for high-dynamic system regulation combining neural networks and simulation constraints is proposed, which integrates the complex operating rules in high-dynamic systems into the "representation-exploration-decision-making" link of reinforcement learning. Specifically, the collected domain knowledge (i.e., the dispatcher's decision data) is first used for pre-training of the active correction control agent, and then the agent is further guided in the interactive learning between the agent and the environment to improve the effectiveness of the correction control strategy finally learned by the agent; the teaching learning method mainly realizes the fusion of prior knowledge in standard deep Q-learning by constructing a comprehensive loss function containing four loss functions.

[0112] Furthermore, the network topology of the power system is composed of a characteristic matrix and an adjacency matrix. The characteristic matrix is ​​used to characterize the characteristic vectors of multiple nodes, and the adjacency matrix is ​​used to characterize the association relationship between different nodes. The network topology data of the power system contains multiple meta-paths, which are used to characterize the paths connecting nodes of different categories. In the active power correction control scenario of the power system, electrical components such as generators, loads, and transmission lines can be regarded as nodes, and the connection relationship between electrical components is defined as an edge, so as to realize the graph regularization of the topologically transformed power grid state. The characteristic matrix in the proposed active power correction control scenario contains the active power of various nodes and the load rate on the transmission line; the adjacency matrix represents the connection relationship between nodes such as generators, loads, and transmission lines in the power grid.

[0113] Furthermore, the state space S is constructed based on the generator active power, load active power, AC transmission line active power, bus active power and transmission line load rate of the current time step in the power system active power correction control scenario; the action space is constructed based on the active power output adjustment action, load shedding action and power system network topology adjustment action in the target power network environment. By defining the reward function Satisfy various objectives and constraints of safety corrective control, Used to describe the reward value obtained by the agent when performing an action in the current state. The reward function of active power correction control includes the transmission line power limit penalty and the generator re-dispatching adjustment penalty.

[0114] Step S7022, storing the state space, action space and reward function of the active power correction control of the power system, as well as the dispatcher's decision data into the sample data pool.

[0115] Step S7023: randomly extract multiple sample data from the sample data pool, use the multiple sample data to train the deep reinforcement learning model, and obtain the power system human-computer interaction model.

[0116] In some optional implementations, the above step S7023 includes:

[0117] Step b1, arbitrarily select the state, action and reward function at a preset time step from the sample data pool, input the state, action and reward function at the preset time step into the deep reinforcement learning model, and obtain the single-step deep Q learning loss function value and the n-step deep Q learning loss function value.

[0118] Specifically, the single-step deep Q learning loss function value L DQ (θ) can be expressed as:

[0119]

[0120] Among them, Q(s t ,a t ; θ) represents the Q value of action a at time t when the network parameters are θ and s, γ represents the discount factor, which reflects the importance of the agent to future rewards. Indicates that at time t+1 s t+1 All possible actions in state a t+1 The maximum Q value, r t Represents the reward function, which is in the state Take action Instant rewards.

[0121] Furthermore, since basic deep Q learning often has a prominent credit assignment problem, by introducing n-step (the number of steps of input data) deep Q learning loss function L n (θ) assists the agent to better evaluate the long-term benefits of actions, thereby improving the training effect; the n-step deep Q learning loss can be calculated based on the n-step return, and the n-step deep Q learning loss function value l n The calculation formula of (θ) is as follows:

[0122]

[0123] in, represents the n-step return, that is, the n-step reward value, Indicates the state is s t+n and the discount factor is γ n The maximum value function obtained by executing action a when Represents a sample data pool.

[0124] Step b2: Input the action corresponding to the dispatcher's decision data into the deep reinforcement learning model to obtain the expert strategy loss function value.

[0125] Specifically, the expert strategy loss function value L E (θ) is based on the assumption of the superiority of the expert strategy, that is, in each target scenario, it is assumed that the actions taken by the expert strategy (i.e., the dispatcher decision) are superior to those in the action space For all other actions in the corresponding scene, the difference between the greedy action taken by the deep Q network, that is, the action with the largest Q value, and the expert action is measured by using the edge supervision loss function; if the greedy action is equivalent to the expert action, the edge supervision loss is 0, otherwise, the edge supervision loss is a positive constant, such as 0.8; therefore, assuming that the Q value Q(s,a) corresponding to the non-expert action in all target scenes is smaller than the Q value Q(s,a) of the expert action E ), the above settings can ensure that the entire training process satisfies the Bellman equation while prompting the agent to imitate the expert strategy and make a more reasonable estimate of the Q value of other actions.

[0126] Furthermore, the calculation formula of the expert strategy loss function value is as follows:

[0127]

[0128] In the above formula, L E (θ) represents the expert strategy loss function value, Q(s,a) represents the Q value of action a in state s, represents the action space, l(a E ,a) represents the edge supervision loss function, a E represents the action corresponding to the dispatcher's decision data, Q(s,a E ) means that in the s state, a E The Q value of the action.

[0129] Step b3, obtaining weight parameters of the deep reinforcement learning model, and calculating the L2 regularization loss function value based on the weight parameters of the deep reinforcement learning model.

[0130] Specifically, the L2 regularization loss function refers to a regularization method added to prevent overfitting during model training; the L2 regularization loss function value L L2(θ) mainly limits the weight of the Q network to ensure the generalization ability of the agent, and the L2 regularization loss function value L L2 The calculation formula of (θ) is as follows:

[0131]

[0132] Among them, y i represents the true value, Represents the value predicted by the deep reinforcement learning model, K represents the weights of each parameter in the deep reinforcement learning Q network, and χ represents the regularization parameter, which indicates a penalty for the complexity of the deep reinforcement learning Q network parameters.

[0133] Furthermore, the role of the L2 regularization loss function is to encourage the model to learn smoother coefficient values, thereby reducing the risk of overfitting of the model. The above method improves the generalization ability of the model without increasing the complexity of the model too much.

[0134] Step b4, determining the total loss function value based on the single-step deep Q learning loss function value, the n-step deep Q learning loss function value, the expert strategy loss function value and the L2 regularization loss function value.

[0135] Specifically, the calculation formula of the total loss function value L(θ) is as follows:

[0136] L(θ)=L DQ (θ)+α 1 L n (θ)+α 2 L E (θ)+α 3 L L2 (θ) (13)

[0137] Among them, α 1 , α 2 , α 3 They represent the weights corresponding to different loss functions.

[0138] Step b5, compare the total loss function value with the preset threshold, and iteratively train the deep reinforcement learning model based on the comparison result until the total loss function value is less than the preset threshold, stop the iteration, and obtain the power system human-computer interaction model.

[0139] Specifically, in order to make the training more stable, the deep reinforcement learning model introduces two networks: the value Q network and the target Q network. The two have the same network structure but different parameter updating methods. The value Q network updates its parameters at the time of training, and the target Q network copies the parameters of the value Q network at regular intervals for update, and then uses the trained deep reinforcement learning model as the human-computer interaction model of the power system.

[0140] Furthermore, based on the total loss function value, the training process of the deep reinforcement learning model integrates domain knowledge (i.e., dispatcher decision-making) into the pre-training and formal training stages of the agent. In the pre-training stage, the reinforcement learning agent samples from the collected teaching data and performs batch training; then, the pre-trained agent begins to interact with the environment and stores the interactively generated data in the experience pool; during the training process, the interactively generated data in the experience pool is continuously updated with the training progress, while the data samples remain unchanged to provide continuous guidance to the agent; at the same time, by controlling the proportion of data samples in the overall sample during experience playback, the self-improvement ability of the reinforcement learning agent based on interactive learning is guaranteed.

[0141] Furthermore, if Figure 8 As shown in the figure, a simulator with clear mechanism is used as a concrete representation of the rules and constraints of the power system, and an exploration and deployment mechanism guided by deep simulation results is proposed. The behavior of machine intelligence is restricted to the approximate strategy feasible domain, which improves its stability of exploration and application in highly dynamic systems with complex constraints, and constructs a stable deployment architecture of artificial intelligence in decision-sensitive complex system control business. The rule-based knowledge-data hybrid driven reinforcement learning method has achieved good results in complex system control, realizing stable exploration and efficient learning under strong system constraints.

[0142] The present embodiment provides a method for active power correction control of an electric power system based on human-computer interaction, which combines the dispatcher's decision data with reinforcement learning, and uses the dispatcher's decision data for pre-training. The loss function related to deep Q learning, namely the single-step deep Q learning loss function value and the n-step deep Q learning loss function value, ensures that the intelligent agent can perform autonomous optimization from time difference learning. The expert strategy loss function value guides the intelligent agent to learn and approximate the demonstrator strategy, namely the dispatcher's decision strategy. The L2 regularization loss function value ensures the generalization ability of the intelligent agent by limiting the weight size of the Q network, thereby obtaining a better initial strategy after pre-training, which can improve the problem of low data utilization in mobile decision-making based on reinforcement learning.

[0143] Step S703: Generate a human-machine collaborative decision using the human-machine interactive model of the power system, and perform active power correction control of the power system based on the human-machine collaborative decision. Figure 5 Step S503 of the illustrated embodiment will not be described in detail here.

[0144] This embodiment provides an active power correction control method for an electric power system based on human-computer interaction. The state space, action space and reward function of the active power correction control, as well as the dispatcher's decision data, are used as sample data to train a deep reinforcement learning model, thereby combining the dispatcher's professional knowledge and experience with intelligent algorithms to achieve more effective and reliable active power correction control.

[0145] The reliability of the power system active power correction control method based on human-computer interaction is verified by a specific embodiment below.

[0146] Embodiment 1:

[0147] The status of the power system is assessed through indicators such as the average line load rate and line voltage loss, reflecting the power system status after the dispatcher's decision and the machine intelligence decision in the active power correction control scenario of the power system (that is, the power system status after the human-machine collaborative decision is executed); after the implementation of the corrective control action, the relatively lower line load rate, smaller voltage loss, smaller voltage phase difference and smaller power generation cost in the power system can be considered as a better grid state; after different control actions, although the power system can maintain a normal operating level, it has different states. The grid status after different actions is analyzed through selected features to further verify the improvement of power system reliability by human-machine collaborative decision-making.

[0148] Whether it is the decision of the dispatcher or the deep reinforcement learning model, there are certain limitations. A single decision has its own advantages and disadvantages in the performance under various power grid situation evaluation indicators. In view of this situation, human-machine collaborative decision-making has become a decision-making method, which can combine the advantages of machine intelligence decision-making and dispatcher decision-making, thereby improving the reliability of active power correction control. In order to more comprehensively evaluate the impact of different active power correction decision-making methods on the state of the power system, a comprehensive analysis of the dispatcher's decision and the power system state under the human-machine collaborative decision is carried out based on the above indicators, and the line average load rate, line voltage loss, line average voltage phase angle loss and power generation cost are weighted separately by the entropy weight method to obtain the indicator weight.

[0149] Since the power grid situation evaluation indicators are all negative indicators, It indicates the value after the normalization of each indicator data, r indicates the rth indicator for evaluating the power grid status, Represents a decision set The power grid situation index value under the lth decision, Contains control plans and machine intelligence decision results, N l Represents a set of decision actions The number of actions in the test scenario is calculated by the entropy weight method. The comprehensive score S of the power grid status under different decisions is obtained by combining various power grid status indicators and their weights for the 100,000 decision operations performed in the test scenario. l , and its calculation formula is as follows:

[0150]

[0151] Among them, the process of human-computer interaction is completed before decision-making. Human-computer interaction is to obtain human-computer collaborative decision-making. Different decisions refer to "dispatcher decision (no interaction)", "machine intelligence decision" and "human-computer collaborative decision". Human-computer collaborative decision-making is obtained through human-computer interaction, and then the comprehensive score of the power grid status under different decisions is used to verify the effect of human-computer collaborative decision-making on improving system readiness and reliability.

[0152] Based on the integrated human-computer interaction collaboration and cognitive behavior experimental platform, active power correction control simulation verification and comparative analysis are carried out through test scenarios. Through the evaluation of the system power grid status after different active power correction controls such as dispatcher decision-making, machine intelligence decision-making, and human-computer collaborative decision-making, it can be obtained that the system status after human-computer collaborative decision-making has a relatively obvious improvement, which improves the accuracy of active power correction control.

[0153] In this embodiment, a power system active power correction control device based on human-computer interaction is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0154] This embodiment provides a power system active power correction control device based on human-computer interaction, such as Fig. 9 As shown, including:

[0155] A cognitive module 901 is used to establish a cognitive model of a power system dispatcher and determine the dispatcher's decision data using the cognitive model of the power system dispatcher;

[0156] The human-computer intelligent interaction module 902 is used to obtain the power system network topology data, train the deep reinforcement learning model based on the power system network topology data and the dispatcher decision data, and obtain the power system human-computer interaction model;

[0157] The control module 903 is used to generate a human-machine collaborative decision by using the human-machine interactive model of the power system, and perform active power correction control of the power system based on the human-machine collaborative decision.

[0158] In some optional implementations, the cognitive module 901 includes:

[0159] Establishing a unit for establishing a cognitive model of a power system dispatcher by using a Bayesian network;

[0160] An acquisition unit is used to acquire the influencing factors of human reliability of the dispatcher of the power system in the target scenario, and based on the influencing factors of human reliability of the dispatcher, a cognitive model of the power system dispatcher is used to acquire the human reliability analysis results;

[0161] The determination unit is used to determine the dispatcher's decision data based on the human reliability analysis results.

[0162] In some optional implementations, the determining unit includes:

[0163] The first determination subunit is used to obtain the dispatcher decision set and determine the weight of each action in the dispatcher decision set using the entropy weight method;

[0164] A first calculation subunit, used for calculating the dispatcher's decision probability based on the human reliability analysis result and the weight of each action;

[0165] The first selection subunit is used to select the dispatcher decision data based on the dispatcher decision probability.

[0166] In some optional implementations, the human-machine intelligent interaction module 902 includes:

[0167] A construction unit, for constructing a state space, an action space and a reward function of active power correction control of a power system by using a Markov decision process based on power system network topology data;

[0168] A storage unit, used for storing the state space, action space and reward function of the active power correction control of the power system and the decision data of the dispatcher into a sample data pool;

[0169] The training unit is used to randomly extract multiple sample data from the sample data pool, use the multiple sample data to train the deep reinforcement learning model, and obtain the power system human-computer interaction model.

[0170] In some optional implementations, the training unit includes:

[0171] The second selection subunit is used to arbitrarily select the state, action and reward function at a preset time step from the sample data pool, input the state, action and reward function at the preset time step into the deep reinforcement learning model, and obtain the single-step deep Q learning loss function value and the n-step deep Q learning loss function value;

[0172] The second computing subunit is used to input the action corresponding to the dispatcher's decision data into the deep reinforcement learning model to obtain the expert strategy loss function value;

[0173] A third calculation subunit is used to obtain weight parameters of the deep reinforcement learning model, and calculate an L2 regularization loss function value based on the weight parameters of the deep reinforcement learning model;

[0174] A second determination subunit is used to determine a total loss function value based on a single-step deep Q learning loss function value, an n-step deep Q learning loss function value, an expert strategy loss function value, and an L2 regularization loss function value;

[0175] The iterative subunit is used to compare the total loss function value with a preset threshold, and iteratively train the deep reinforcement learning model based on the comparison result until the total loss function value is less than the preset threshold, stop the iteration, and obtain the power system human-computer interaction model.

[0176] In some optional implementations, the calculation formula for the expert strategy loss function value in the second calculation subunit is as follows:

[0177]

[0178] In the above formula, L E (θ) represents the expert strategy loss function value, Q(s,a) represents the Q value of action a in state s, represents the action space, l(a E ,a) represents the edge supervision loss function, a E represents the action corresponding to the dispatcher's decision data, Q(s,a E ) means that in the s state, a E The Q value of the action.

[0179] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0180] In this embodiment, an active power correction control device for an electric power system based on human-computer interaction is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0181] The embodiment of the present invention also provides a computer device having the above Fig. 9 A power system active power correction control device based on human-computer interaction is shown.

[0182] See also Fig.10 , Fig.10 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.10As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.10 A processor 10 is taken as an example.

[0183] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0184] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0185] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0186] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0187] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0188] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0189] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0190] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0191] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for active power correction control of an electric power system based on human-computer interaction, characterized in that: include: Establishing a cognitive model for a power system dispatcher, and using the cognitive model for the power system dispatcher to determine the dispatcher's decision data; Acquire power system network topology data, train a deep reinforcement learning model based on the power system network topology data and the dispatcher decision data, and obtain a power system human-computer interaction model; The human-machine interaction model of the power system is used to generate a human-machine collaborative decision, and active power correction control of the power system is performed based on the human-machine collaborative decision.

2. The method according to claim 1, characterized in that The establishing of the power system dispatcher cognitive model and determining the dispatcher decision data using the power system dispatcher cognitive model includes: Establishing the cognitive model of power system dispatchers by Bayesian network; Obtaining the influencing factors of human reliability of the dispatcher of the power system in the target scenario, and obtaining the human reliability analysis results by using the cognitive model of the dispatcher of the power system based on the influencing factors of human reliability of the dispatcher; The dispatcher decision data is determined based on the human reliability analysis result.

3. The method according to claim 2, characterized in that The determining of the dispatcher decision data based on the human reliability analysis result includes: Obtaining a dispatcher decision set, and determining the weight of each action in the dispatcher decision set using an entropy weight method; Calculating the dispatcher's decision probability based on the human reliability analysis result and the weight of each action; The dispatcher decision data is selected based on the dispatcher decision probability.

4. The method according to claim 1, characterized in that The deep reinforcement learning model is trained based on the power system network topology data and the dispatcher decision data to obtain the power system human-computer interaction model, including: Based on the power system network topology data, a state space, an action space and a reward function of active power correction control of the power system are constructed using a Markov decision process; Storing the state space, action space and reward function of the active power correction control of the power system and the dispatcher decision data into a sample data pool; A plurality of sample data are randomly extracted from the sample data pool, and the deep reinforcement learning model is trained using the plurality of sample data to obtain the power system human-computer interaction model.

5. The method according to claim 4, characterized in that The method randomly extracts a plurality of sample data from the sample data pool, and uses the plurality of sample data to train the deep reinforcement learning model to obtain the power system human-computer interaction model, including: Randomly select the state, action and reward function at a preset time step from the sample data pool, input the state, action and reward function at the preset time step into the deep reinforcement learning model, and obtain the single-step deep Q learning loss function value and the n-step deep Q learning loss function value; Inputting the action corresponding to the dispatcher's decision data into the deep reinforcement learning model to obtain the expert strategy loss function value; Obtaining weight parameters of the deep reinforcement learning model, and calculating an L2 regularization loss function value based on the weight parameters of the deep reinforcement learning model; Determine a total loss function value based on the single-step deep Q learning loss function value, the n-step deep Q learning loss function value, the expert strategy loss function value, and the L2 regularization loss function value; The total loss function value is compared with a preset threshold, and the deep reinforcement learning model is iteratively trained based on the comparison result until the total loss function value is less than the preset threshold, and the iteration is stopped to obtain the power system human-computer interaction model.

6. The method according to claim 5, characterized in that The action corresponding to the dispatcher decision data is input into the deep reinforcement learning model to obtain the expert strategy loss function value; wherein the calculation formula of the expert strategy loss function value is as follows: In the above formula, L E (θ) represents the expert strategy loss function value, Q(s,a) represents the Q value of action a in state s, represents the action space, l(a E ,a) represents the edge supervision loss function, a E represents the action corresponding to the dispatcher's decision data, Q(s,a E ) means that in the s state, a E The Q value of the action.

7. A power system active power correction control device based on human-computer interaction, characterized in that: The device comprises: A cognitive module, used to establish a cognitive model of a power system dispatcher, and determine the dispatcher's decision data using the cognitive model of the power system dispatcher; A human-computer intelligent interaction module is used to obtain power system network topology data, train a deep reinforcement learning model based on the power system network topology data and the dispatcher decision data, and obtain a power system human-computer interaction model; The control module is used to generate a human-machine collaborative decision by using the human-machine interactive model of the power system, and perform active power correction control of the power system based on the human-machine collaborative decision.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the active power correction control method of the power system based on human-computer interaction as described in any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the power system active power correction control method based on human-computer interaction according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the power system active power correction control method based on human-computer interaction according to any one of claims 1 to 6.

Images