Power system look-ahead scheduling method and device based on evolutionary strategy and knowledge fusion
By constructing a Markov decision process model embedded with physical knowledge and training the agent using an evolutionary strategy, the problem of high-dimensional uncertainty and complexity in forward scheduling of power systems was solved, achieving efficient decision-making results.
Patent Information
- Application Number
- CN202511696391.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-19
- Publication Date
- 2026-02-27
AI Technical Summary
Existing forward-looking dispatch methods for power systems suffer from low computational efficiency and insufficient decision-making accuracy when dealing with the high-dimensional uncertainties and complexities under the high proportion of renewable energy access, making it difficult to cope with the complexity and uncertainty of the power grid.
We construct a Markov decision process model based on evolutionary strategy and knowledge fusion, embed physical knowledge into the action space, design a reward function, and train the agent through evolutionary strategy to improve decision-making efficiency and accuracy.
Without sacrificing decision-making accuracy, it significantly improves the training efficiency and performance of forward scheduling in power systems, effectively addresses high-dimensional uncertainty and complexity, and reduces computational resource requirements.
Smart Images
Figure CN121584749A_ABST
Abstract
Description
Technical Field
[0001] This application relates to a forward-looking scheduling method and device for power systems based on evolutionary strategies and knowledge fusion, belonging to the field of power system security and stability control technology. Background Technology
[0002] With the increasing proportion of new energy sources in the power system, the uncertainty of power grid operation has increased significantly. The control targets have expanded from traditional generating units to various resources such as new energy sources, loads, and energy storage, placing higher demands on the reliability and timeliness of control decisions. Forward-looking dispatch, as a crucial part of power system dispatch, identifies and addresses risks in advance by predicting future operating conditions based on short-term forecasts, ensuring the safe and stable operation of the power grid under uncertain environments.
[0003] However, traditional methods based on physical models have shortcomings in handling uncertainties and computational efficiency, especially in the intraday dispatch phase, where they still rely on dispatchers and struggle to cope with the complexity and uncertainty of new power systems. Therefore, there is an urgent need to research efficient forward dispatch methods for power systems that can adapt to high proportions of renewable energy integration. Summary of the Invention
[0004] Objective: In view of at least one of the above technical problems, this application provides a power system forward scheduling method and device based on evolutionary strategy and knowledge fusion, which solves the high-dimensional uncertainty and complex nonlinear constraints in the existing forward scheduling problem.
[0005] The technical solution adopted in this application is:
[0006] Firstly, this application provides a forward-looking scheduling method for power systems based on evolutionary strategies and knowledge fusion, including:
[0007] A Markov decision process model for look-ahead scheduling is constructed, and a state space, an action space with embedded physical knowledge, and a reward function are designed to formulate a time-series decision-making mechanism for power system scheduling problems.
[0008] Based on the aforementioned time-series decision-making mechanism, the evolutionary strategy is applied to the Markov decision process model of the prospective scheduling to obtain a prospective scheduling strategy based on the evolutionary strategy.
[0009] Secondly, this application provides a power system forward scheduling device based on evolutionary strategies and knowledge fusion, comprising:
[0010] The module is used to build a Markov decision process model for look-ahead scheduling, design the state space, the action space with embedded physical knowledge, and the reward function, so as to design the time-series decision mechanism for power system scheduling problems.
[0011] An optimization module is used to apply an evolutionary strategy to the Markov decision process model of the look-ahead scheduling based on the time-series decision mechanism, so as to obtain a look-ahead scheduling strategy based on the evolutionary strategy.
[0012] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.
[0013] Fourthly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.
[0014] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.
[0015] Beneficial Effects: The power system look-ahead scheduling method and apparatus based on evolutionary strategy and knowledge fusion provided in this application have the following advantages: they can improve decision-making efficiency without sacrificing decision accuracy, and have significant advantages in both training efficiency and performance. First, based on the mathematical model of look-ahead scheduling, a Markov decision process model with embedded physical knowledge is constructed, embedding physical knowledge such as generator output constraints and active power balance constraints into the action space. Then, an agent training method based on evolutionary strategy is proposed, which improves parallel exploration efficiency and reduces the demand for computing resources by introducing synchronous random seeds and mirror perturbation strategies. This power system look-ahead scheduling method based on evolutionary strategy and knowledge fusion has significant advantages in training efficiency and performance, and can effectively cope with the high-dimensional uncertainty and complexity of new power systems. It solves the high-dimensional uncertainty and complex nonlinear constraints existing in the look-ahead scheduling problem, and the evolutionary strategy learning algorithm, due to its global search capability, can better adapt to the complexity and uncertainty of new power systems. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating a power system forward scheduling method based on evolutionary strategy and knowledge fusion according to an embodiment of this application.
[0017] Figure 2 This is a schematic diagram comparing the training process of the method in the embodiments of this application and the deep reinforcement learning algorithm;
[0018] Figure 3 This is a schematic diagram comparing the training time of the method in the embodiments of this application and the deep reinforcement learning algorithm;
[0019] Figure 4 This is a schematic diagram comparing the training performance of the method in the embodiments of this application and the deep reinforcement learning algorithm;
[0020] Figure 5 This is a schematic diagram of a power system forward dispatch system according to an embodiment of this application. Detailed Implementation
[0021] The present application will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and should not be used to limit the scope of protection of the present application.
[0022] In the description of this application, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.
[0023] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0024] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0025] Example 1: This example provides a power system forward scheduling method based on evolutionary strategies and knowledge fusion, such as... Figure 1 As shown, it includes:
[0026] A Markov decision process model for look-ahead scheduling is constructed, and a state space, an action space with embedded physical knowledge, and a reward function are designed to formulate a time-series decision-making mechanism for power system scheduling problems.
[0027] Based on the aforementioned time-series decision-making mechanism, the evolutionary strategy is applied to the Markov decision process model of the prospective scheduling to obtain a prospective scheduling strategy based on the evolutionary strategy.
[0028] In some embodiments, designing the state space includes:
[0029] Markov decision process model State space at any given moment Represented as:
[0030] ;
[0031] in, , , They represent Real-time active power output, load demand, and AC transmission line power transmission capacity of all generator units; , They represent Real-time load forecasting and renewable energy output forecasting.
[0032] In this embodiment, information is input to the agent through the state space of the Markov decision process model, including the active power output of all generator units in the previous moment, load demand and AC transmission line transmission power, as well as prediction information (load prediction and new energy output prediction).
[0033] In some embodiments, the action space for embedding physical knowledge is designed, including:
[0034] Markov decision process model Moment of action space Represented as:
[0035] ;
[0036] in, , They represent in The active power output of all generator sets and new energy units at all times; , Indicates in , Time generator set Those who have made meritorious contributions; and Indicates generator set The upper and lower limits of contributions and efforts; and Indicates generator set Slope rate and landslide rate; For time intervals; and They are respectively Shike New Energy Unit The prediction of merit and effort output; A collection of nodes for new energy power stations;
[0037] In this embodiment, the intelligent agent determines the output of each generator set through the output of a deep neural network, and maps the output of the deep neural network to a standard range using a hyperbolic tangent activation function. Then, a linear transformation is used to map the output range of the deep neural network to the actual output range of the generator set.
[0038] In action space The active power output limit of the embedded unit is expressed as:
[0039] ;
[0040] in, and They are respectively The upper and lower limits of the action space at any given moment; The output of the agent represented by a deep neural network; and Indicates the upper and lower bounds of the output of the deep neural network;
[0041] In action space Embedded active power balance constraints are expressed as follows:
[0042] ;
[0043] in, express The sum of active power at the initial operation time; for Total load demand at any given time; This represents the active power imbalance coefficient; express The actions simulated in the real-time power grid simulation environment; This represents the power transmission loss coefficient. and express The sum of the upper and lower limits of the action space of all units at any given time.
[0044] In this embodiment, physical knowledge (generator output constraints, active power balance constraints) is embedded into the action space to limit the decision-making range of the intelligent agent and ensure that the decision conforms to the actual physical rules.
[0045] To minimize the grid operating cost over the entire scheduling cycle, a weighted sum of reward and penalty terms is used as the reward function.
[0046] More specifically, in some embodiments, the reward function is designed including:
[0047] Markov decision process model Reward function at time step Represented as:
[0048] ;
[0049] ;
[0050] ;
[0051] ;
[0052] ;
[0053] ;
[0054] in, , , , , These are the corresponding weighting coefficients; The power grid operating cost (negative value is taken as the bonus value); The number of generator sets; , , Generator sets The quadratic, linear, and constant coefficients of the power generation cost function; Indicates in Time generator set Those who have made meritorious contributions; Average cost per unit of electricity generated; for Total load demand at any given time; The ratio of the unit's over-limit capacity to its rated capacity is used as a penalty for active power imbalance. express Constantly balance the active power output of the generating units; , These are the upper and lower limits of the active power output of the balancing unit, respectively. This is the cumulative sum of penalties for exceeding the power transmission limit for all lines; A collection of AC transmission lines; This is a penalty for exceeding the line transmission power limit; Indicates the number of AC transmission lines; express Real-time communication transmission line The transmission power; and Represents AC transmission line The reverse and forward capacity limits; The proportion of insufficient rotating reserve capacity is used as a penalty. A collection of nodes for new energy power stations; and Indicates in The generator set Provided positive and negative rotational reserve capacity; and Indicates the minimum positive and negative rotating reserve capacity requirements; Incentives will be awarded based on the proportion of new energy consumption.
[0055] In some embodiments, applying an evolutionary strategy to the look-ahead scheduling Markov decision process model includes: training the agent using the evolutionary strategy, testing and applying the agent.
[0056] Furthermore, an evolutionary strategy is employed to train the agent, including:
[0057] S1: Obtain historical scheduling data and transform the historical scheduling data according to the definition of the state space, action space and reward function for offline training of the intelligent agent;
[0058] S2: Initialization Each process is assigned a unique random seed and initial policy parameters. ;
[0059] S3: The iterative optimization process of the loop strategy continues until a preset number of iterations is reached, and the policy parameters corresponding to the maximum reward function value are output as the final policy parameters; wherein, the iterative optimization process includes:
[0060] S31: Each process generates a set of parameter perturbations based on the assigned random seed. parameter perturbation Follows a standard normal distribution , It is the identity matrix;
[0061] S32: On each parallel process, based on the parameter perturbation Sample a noise vector from the noise sequence whose length matches the agent's current policy parameters. By respectively from the agent's current policy parameters Add or subtract noise vectors To create two mutated intelligent agents; The standard deviation of noise;
[0062] S33: Each mutant agent is evaluated in a corresponding parallel process environment, and the cumulative reward is used as the fitness score for each mutant agent; fitness score The calculation formula is expressed as follows:
[0063] ;
[0064] In the formula: T is the total number of time periods included in the scheduling cycle; for The reward function at each time step;
[0065] In this embodiment, T is taken as 2 to 4 hours.
[0066] S33: Based on the fitness scores and parameter perturbations of the two mutant agents Using the preset learning rate and noise standard deviation, calculate the global gradient to update the policy parameters; the update formula for the policy parameters is expressed as:
[0067] ,
[0068] In the formula: The learning rate; The standard deviation of noise; , Number of processes; , These are the policy parameters before and after the k-th iteration update, respectively. , These are the fitness scores of the two mutant agents, respectively.
[0069] S34: Calculate the reward function value based on the updated policy parameters and compare it with the historical maximum reward function value.
[0070] Furthermore, training agents using evolutionary strategies also includes:
[0071] Acquire real-time scheduling data and transform it according to the defined state space, action space, and reward function for online training of the intelligent agent;
[0072] Input a new learning rate and noise standard deviation, and use the final policy parameters obtained from offline training of the agent as the initial policy parameters for online training;
[0073] initialization Each process is assigned a unique random seed and initial policy parameters;
[0074] The policy optimization iteration process is repeated until the preset number of iterations is reached, and the policy parameters corresponding to the maximum reward function value are output as the final policy parameters for the online training of the agent.
[0075] Based on the final policy parameters trained online, a forward scheduling policy is determined in the real-time environment.
[0076] Simulation verification experiment: Figure 2 This is a schematic diagram comparing the training process of the method in the embodiments of this application and the deep reinforcement learning algorithm; Figure 3This is a schematic diagram comparing the training time of the method in the embodiments of this application and the deep reinforcement learning algorithm; Figure 4 This is a schematic diagram comparing the training performance of the method in this application embodiment and the deep reinforcement learning algorithm; by Figures 2-4 It can be seen that, compared with the deep reinforcement learning algorithms DDPG, PPO, and SAC, the method of this application embodiment has the highest cumulative reward in the next round for the same number of training rounds, the shortest training time for the same number of training rounds, and the highest cumulative reward in the next round for the same training time.
[0077] In summary, the power system look-ahead scheduling method based on evolutionary strategies and knowledge fusion provided in this application can improve decision-making efficiency without sacrificing decision accuracy, demonstrating significant advantages in both training efficiency and performance. First, based on the mathematical model of look-ahead scheduling, a Markov decision process model with embedded physical knowledge is constructed, embedding physical knowledge such as generator output constraints and active power balance constraints into the action space. Then, an agent training method based on evolutionary strategies is proposed, which improves parallel exploration efficiency and reduces the demand for computing resources by introducing synchronous random seeds and mirror perturbation strategies. This power system look-ahead scheduling method based on evolutionary strategies and knowledge fusion has significant advantages in training efficiency and performance, and can effectively cope with the high-dimensional uncertainty and complexity of new power systems.
[0078] Example 2: Based on Example 1, this example provides a power system forward scheduling device based on evolutionary strategies and knowledge fusion, such as... Figure 5 As shown, it includes:
[0079] The module is used to build a Markov decision process model for look-ahead scheduling, design the state space, the action space with embedded physical knowledge, and the reward function, so as to design the time-series decision mechanism for power system scheduling problems.
[0080] An optimization module is used to apply an evolutionary strategy to the Markov decision process model of the look-ahead scheduling based on the time-series decision mechanism, so as to obtain a look-ahead scheduling strategy based on the evolutionary strategy.
[0081] Example 3: Based on Example 1, this example provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in Example 1.
[0082] Example 4: Based on Example 1, this example provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in Example 1.
[0083] Example 5: Based on Example 1, this example provides a computer program product, including a computer program that, when executed by a processor, implements the method described in Example 1.
[0084] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0085] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0088] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. An evolutionary strategy and knowledge fusion based power system look-ahead scheduling method, characterized in that, include: A Markov decision process model for look-ahead scheduling is constructed, and a state space, an action space with embedded physical knowledge, and a reward function are designed to formulate a time-series decision-making mechanism for power system scheduling problems. Based on the aforementioned time-series decision-making mechanism, the evolutionary strategy is applied to the Markov decision process model of the prospective scheduling to obtain a prospective scheduling strategy based on the evolutionary strategy.
2. The method of claim 1, wherein, Design the state space, including: Markov decision process model state space at time is represented as: ； wherein, , , respectively represent active power output of all generator units, load demand and AC transmission line transmission power at time , respectively represent load forecast, new energy output forecast at time 3. The method of claim 1, wherein, Design the action space for embedding physical knowledge, including: Markov decision process model Action space at time is represented as: ； wherein, , represent the active power output of all generator units and new energy units at time ; , represent the active power output of generator units at time , ; ; and represent the upper limit and the lower limit of the active power output of generator units ; and represent the ramp-up rate and the ramp-down rate of generator units ; is the time interval; and are the active power output prediction and the active power output of new energy units at time ; ; is the set of new energy station nodes; In the action space The active power output limit of the embedded unit is embedded in the action space, expressed as: ； wherein, and are respectively upper and lower bounds of the action space at the moment; is an output of the agent represented by a deep neural network; and represent upper and lower bounds of the output of the deep neural network. In the action space The active power balance constraint is embedded in the action space and is expressed as: ； wherein, represents active power and at the initial operation; is total load demand at the time; represents the active power imbalance coefficient; represents the action simulated by the grid simulation environment at the time; represents the loss coefficient of power transmission; and represents the sum of the upper limits and the sum of the lower limits of the action space of all units at the time.
4. The method of claim 1, wherein, Design a reward function, including: Markov decision process model Reward function at time is represented as: ； ； ； ； ； ； in, , , , , These are the corresponding weighting coefficients; For power grid operating costs; The number of generator sets; , , Generator sets The quadratic, linear, and constant coefficients of the power generation cost function; Indicates in Time generator set Those who have made meritorious contributions; Average cost per unit of electricity generated; for Total load demand at any given time; The ratio of the unit's over-limit capacity to its rated capacity is used as a penalty for active power imbalance. express Constantly balance the active power output of the generating units; , These are the upper and lower limits of the active power output of the balancing unit, respectively. This is the cumulative sum of penalties for exceeding the power transmission limit for all lines; A collection of AC transmission lines; This is a penalty for exceeding the line transmission power limit; Indicates the number of AC transmission lines; express Real-time communication transmission line The transmission power; and Represents AC transmission line The reverse and forward capacity limits; The proportion of insufficient rotating reserve capacity is used as a penalty. A collection of nodes for new energy power stations; and Indicates in The generator set Provided positive and negative rotational reserve capacity; and Indicates the minimum positive and negative rotating reserve capacity requirements; Incentives will be awarded based on the proportion of new energy consumption.
5. The method according to claim 1, characterized in that, Applying an evolutionary strategy to the Markov decision process model of the prospective scheduling includes: training the agent using the evolutionary strategy, testing and applying the agent.
6. The method according to claim 5, characterized in that, Training agents using evolutionary strategies includes: S1: Obtain historical scheduling data and transform the historical scheduling data according to the definition of the state space, action space and reward function for offline training of the intelligent agent; S2: Initialization Each process is assigned a unique random seed and initial policy parameters. ; S3: The iterative optimization process of the loop strategy continues until a preset number of iterations is reached, and the policy parameters corresponding to the maximum reward function value are output as the final policy parameters; wherein, the iterative optimization process includes: S31: Each process generates a set of parameter perturbations based on the assigned random seed. parameter perturbation Follows a standard normal distribution , It is the identity matrix; S32: On each parallel process, based on the parameter perturbation Sample a noise vector from the noise sequence whose length matches the agent's current policy parameters. By respectively from the agent's current policy parameters Add or subtract noise vectors To create two mutated intelligent agents; The standard deviation of noise; S33: Each mutant agent is evaluated in a corresponding parallel process environment, and the cumulative reward is used as the fitness score for each mutant agent; fitness score The calculation formula is expressed as follows: ； In the formula: T is the total number of time periods included in the scheduling cycle; for The reward function at each time step; S33: Based on the fitness scores and parameter perturbations of the two mutant agents Using the preset learning rate and noise standard deviation, calculate the global gradient to update the policy parameters; the update formula for the policy parameters is expressed as: ， In the formula: The learning rate; The standard deviation of noise; , Number of processes; , These are the policy parameters before and after the k-th iteration update, respectively. , These are the fitness scores of the two mutant agents, respectively. S34: Calculate the reward function value based on the updated policy parameters and compare it with the historical maximum reward function value.
7. The method according to claim 6, characterized in that, Training agents using evolutionary strategies also includes: Acquire real-time scheduling data and transform it according to the defined state space, action space, and reward function for online training of the intelligent agent; Input a new learning rate and noise standard deviation, and use the final policy parameters obtained from offline training of the agent as the initial policy parameters for online training; initialization Each process is assigned a unique random seed and initial policy parameters; The policy optimization iteration process is repeated until the preset number of iterations is reached, and the policy parameters corresponding to the maximum reward function value are output as the final policy parameters for the online training of the agent. Based on the final policy parameters trained online, a forward scheduling policy is determined in the real-time environment.
8. A power system forward dispatching device based on evolutionary strategies and knowledge fusion, characterized in that, include: The module is used to build a Markov decision process model for look-ahead scheduling, design the state space, the action space with embedded physical knowledge, and the reward function, so as to design the time-series decision mechanism for power system scheduling problems. An optimization module is used to apply an evolutionary strategy to the Markov decision process model of the look-ahead scheduling based on the time-series decision mechanism, so as to obtain a look-ahead scheduling strategy based on the evolutionary strategy.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 7.
10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1 to 7.