Automatic power generation control method and device for distributed power system

Abstract

The embodiment of the invention provides an automatic power generation control method and device for a distributed power system, and the method comprises the steps: executing a control instruction corresponding to an action in a current system state according to a control strategy, and updating a Q value according to the current system state and the corresponding action and a linear mapping Q value function; according to the current system state, a reward value is calculated according to a preset reward function; calculating a value function error according to the reward value and the updated Q value; calculating a strategy gradient according to the value function error, the qualification trace and the state value function error; and updating the control strategy according to the control strategy and the strategy gradient, and generating a new control instruction in combination with the area control deviation of other areas. According to the invention, multi-region cooperative control can be realized, the problem of Q table storage explosion is solved, and the dynamic response speed and system robustness of cooperative control are ensured.

Description

Technical Field

[0001] This application relates to the field of power technology, and in particular to an automatic generation control method and apparatus for a distributed power system. Background Technology

[0002] Automatic Generation Control (AGC) is a core means of ensuring active power balance and frequency stability in power systems, and is also one of the basic functions of the power grid energy management system. Centralized AGC, where a control center uniformly schedules all units in the system, faces challenges due to the large-scale integration of new energy sources (wind power, photovoltaics, etc.) and the trend towards regional grid interconnection. These centralized architectures suffer from problems such as large communication delays, high risk of single-point failures, and difficulty in adapting to dynamic interactions across multiple regions. Distributed AGC, by replacing centralized decision-making with multi-regional autonomous collaborative control, improves system flexibility and reliability. However, issues such as random load fluctuations, nonlinear coupling between regions, and uncertainties in new energy output (intermittency and randomness) exist in each region. How to meet the requirements of dynamic response speed and robustness through multi-regional collaborative control is a problem that needs to be solved. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide an automatic generation control method and apparatus for a distributed power system.

[0004] To achieve the above objectives, this application provides an automatic generation control method for a distributed power system, comprising: After executing the control command corresponding to the action in the current system state according to the current control strategy, the Q value is updated according to the current system state and the corresponding action, using the linear mapping Q value function. Based on the current system state, calculate the reward value according to the preset reward function; Calculate the value function error based on the reward value and the updated Q value; The policy gradient is calculated based on the value function error, eligibility trace, and state value function error. Based on the current control strategy and strategy gradient, update the control strategy and generate new control commands by combining the regional control deviations of other regions.

[0005] Optionally, the linear mapping Q-value function is implemented based on a feedforward neural network model, and the linear mapping Q-value function is expressed as: ; in, It is a state-action characteristic, which is derived from the current system state. s k and corresponding actionsa k Low-dimensional features extracted from the constructed features. , These are the parameters of the feedforward neural network model.

[0006] Optionally, the reward function is: ; in, R k As a reward value, ACE k Current system state s k The corresponding area control deviation, For the first m The unit carbon emission intensity coefficient of each generating unit M For the number of units, These are weighting coefficients. For the first m Carbon emission intensity of each unit under current system conditions.

[0007] Optionally, the method for calculating the value function error based on the reward value and the updated Q value is as follows: ; in, For the error of the value function, As a discount factor, For mixed sampling parameters, For the state value function, Q ( s k-1 , a k-1 ) is in the system state s k-1 Next action a k-1 The Q value.

[0008] Optionally, the policy gradient is calculated based on the value function error, eligibility trace, and state value function error, using the following method: ; in, e k ( s , a ) is a qualification record, These are the weight parameters of the feedforward neural network under the current system state; v k ( s k , a kLet be the state value error function, expressed as: ; in, It is a scaling factor. e k-1 ( s , a () represents the system state s k-1 The qualification record below System status s k-1 The state value function under [condition].

[0009] Optionally, the method further includes: The weight parameters of the linear mapping Q-value function are updated based on the value function error, qualification trace, and updated Q-value, using the following method: ; in, For the updated weight parameters, Let be the learning rate of the value function.

[0010] Optionally, the method further includes: The predicted deviation is calculated based on the area control deviation predicted by the PID controller and the area control deviation corresponding to the current system state. If the predicted deviation is greater than a preset deviation threshold, the weights of the linear mapping Q-value function are adjusted according to the control parameters of the PID controller, as follows: ; in, The proportional gain of the PID controller. The integral coefficient of the PID controller. e i,k For the region i In the k Prediction bias of the wheel, t It is an integer.

[0011] Optionally, the method for generating new control commands by combining the regional control deviations of other regions is as follows: ; in, For the region i With the region j The synergy coefficient between them N It is the number of all regions. ACE i and ACE j They are respectively regions i , jRegional control deviation, ACE k For this region in the 1st k Wheel area control deviation, K To adjust the gain.

[0012] This application also provides an automatic generation control device for a distributed power system, comprising: The update module is used to update the Q value according to the linear mapping Q value function after executing the control command corresponding to the action in the current system state according to the current control strategy. The reward value calculation module is used to calculate the reward value according to the current system state and a preset reward function. The error calculation module is used to calculate the value function error based on the reward value and the updated Q value; The gradient calculation module is used to calculate the policy gradient based on the value function error, qualification trace, and state value function error. The strategy update module is used to update the control strategy based on the current control strategy and strategy gradient, and generate new control commands by combining the regional control deviations of other regions.

[0013] Optionally, the linear mapping Q-value function is implemented based on a feedforward neural network model, and the linear mapping Q-value function is expressed as: ; in, It is a state-action characteristic, which is derived from the current system state. s k and corresponding actions a k Low-dimensional features extracted from the constructed features. , These are the parameters of the feedforward neural network model.

[0014] As can be seen from the above, the distributed power system automatic generation control method and apparatus provided in this application, after executing the control command corresponding to the action in the current system state according to the current control strategy, updates the Q value according to the linear mapping Q value function based on the current system state and the corresponding action; calculates the reward value according to the preset reward function based on the current system state; calculates the value function error based on the reward value and the updated Q value; calculates the strategy gradient based on the value function error, qualification trace, and state value function error; and updates the control strategy based on the current control strategy and the strategy gradient, combined with the regional control deviations of other regions. This application can realize multi-regional collaborative control, solve the Q-table storage explosion problem, and ensure the dynamic response speed and system robustness of collaborative control. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the method flow of an embodiment of this application; Figure 2 This is a schematic diagram of a distributed power system architecture according to an embodiment of this application; Figure 3 This is a block diagram of the device structure according to an embodiment of this application; Figure 4 This is a block diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0018] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0019] like Figure 1 As shown in the figure, this application provides an automatic generation control method for a distributed power system, including: S 101: After executing the control command corresponding to the action in the current system state according to the current control strategy, update the Q value according to the linear mapping Q value function based on the current system state and the corresponding action; In this embodiment, in a distributed multi-regional interconnected power system, multiple independent power supply areas are divided according to geographical distribution, power grid topology, and energy resource allocation. Each area is equipped with complete energy production, load consumption, and control and dispatch units. The energy production units include new energy sources (wind power, photovoltaic), combined cooling, heating and power (CCHP), and traditional units (thermal power, hydropower), etc. Power exchange between areas is achieved through tie lines. Each area is equipped with an independent area control unit (i.e., intelligent agent), and multi-regional collaborative control is achieved through information exchange with intelligent agents in other areas.

[0020] For each agent in a region, the real-time system status of that region is collected, including system frequency, tie-line transmission power, total output of new energy sources (including real-time power generation from intermittent sources such as wind and solar power), output of combined cooling, heating and power (including equivalent power output corresponding to power generation, heating, and cooling), and output of traditional generating units. The regional frequency deviation is calculated based on the system frequency and rated frequency to reflect the frequency stability. The regional tie-line power deviation is calculated based on the tie-line transmission power and planned transmission power to reflect the power balance between regions. A state vector is constructed based on the collected system status, including the regional frequency deviation Δ in the k-th round. f k Regional tie line power deviation Δ P tiek Total output of new energy P nk Real-time output of combined cooling, heating and power P cchp_k Real-time active power output of traditional units P g_k The state vector is the main execution target of automatic power generation control. The constructed state vector is preprocessed by using a moving average filter to remove measurement noise and normalizing each state vector to [-1,1] to facilitate subsequent data processing.

[0021] In the current system state, the agent executes an action as the current control strategy π. For example, the control strategy π includes: when the system state is 0.02Hz lower than the system frequency, the agent executes an action to increase the unit output by 10MW. After executing the control command corresponding to this action, the system state is updated accordingly. Based on the updated system state, the control strategy for the next round is determined.

[0022] In this embodiment, considering that the Q-value (value of each state-action pair) in general reinforcement learning is stored in a table, in a distributed power system, when the area scale expands and / or the dimension of the system state increases, the number of state-action pairs grows exponentially, posing a risk of storage explosion and seriously affecting the efficiency of automatic generation control. For example, in the case of ten interconnected areas and two additional state dimensions, the table for storing Q-values ​​becomes too large, causing the control algorithm to converge for more than 10 seconds, and the dynamic response speed of multi-area control to decrease significantly.

[0023] To address this issue, this embodiment replaces the Q-value table with a function approximation technique, implements a linear mapping Q-value function based on a feedforward neural network model, and combines gradient optimization to improve the calculation accuracy of the Q-value. The linear mapping Q-value function can adapt to large-scale scenarios with multiple regions and multiple state dimensions, ensuring control efficiency and improving the dynamic response speed of multi-region control.

[0024] The linear mapping Q-value function approximates a complex Q-value table using a linear function. The linear mapping Q-value function is expressed as: (1) in, It is a state-action characteristic, which is derived from the current system state. s k and corresponding actions a k Low-dimensional features extracted from the constructed features. To approximate the Q value, These are the parameters of the feedforward neural network model.

[0025] The loss function of the feedforward neural network model is L2-constrained with the Q value, and is expressed as: (2) in, This is the actual Q value, and N is the number of historical samples.

[0026] In some approaches, to adapt to the continuous value space after function approximation, gradient optimization is used instead of tabular discretization updates to ensure that the parameters converge to the optimum. A multilayer perceptron model can be used as the feedforward neural network model.

[0027] S102: Calculate the reward value according to the preset reward function based on the current system state; In this embodiment, after performing an action in the current system state, an immediate reward value is calculated using a reward function, which is: (3) in, ACEk Current system state s k The corresponding area control deviation, For the first m The unit carbon emission intensity coefficient of each generating unit M For the number of units, These are weighting coefficients. For the first m Carbon emission intensity of each unit under current system conditions R k This is the reward value.

[0028] In some methods, the regional control deviation is calculated based on the regional frequency deviation and the regional tie-line power deviation, and is expressed as: (4) in, This is the frequency deviation coefficient.

[0029] S103: Calculate the value function error based on the reward value and the updated Q value; The method for calculating the error of the value function is as follows: (5) in, For the error of the value function, As a discount factor, For mixed sampling parameters, the value is 0 < <1; For the state value function, In system state s k-1 Next action a k-1 The Q value. In some approaches, the hybrid sampling parameter balances the online sampling and full sampling strategies of SARSA. When σ=1, it corresponds to the full sampling strategy (i.e., SARSA-λ), which relies solely on the actions and state updates of the previous round, resulting in high variance but simple computation. When σ=0, it degenerates into pure expectation (i.e., Expected-SARSA-λ), utilizing the weighted average Q-value of all possible actions (with the probability of strategy π as the weight), resulting in low variance but complex computation. When 0<σ<1, it combines the advantages of both sampling strategies, reducing random error while avoiding the high computational cost of pure expectation, and exhibiting high robustness. S104: Calculate the policy gradient based on the value function error, qualification trace, and state value function error; In this embodiment, the update of policy π follows the objective of maximizing long-term cumulative reward. By calculating the gradient of the policy objective function, policy π is guided to adjust in the optimal direction.

[0030] The method for calculating the policy gradient is as follows: (6) in, For qualification, As a discount factor, These are the weight parameters of the feedforward neural network model under the current system state; The state value error function is calculated as follows: (7) in, It is a scaling factor. System status s k-1 The qualification record below In system state s k-1 The state value function under [condition].

[0031] The eligibility trace is used to adapt to the introduction of mixed sampling in the policy probability π, and is expressed as: (8) Where I() is an indicator function, when ( s , a )=( s k-1 , a k-1 When ), I(( s , a )=( s k-1 , a k-1 The value is 1.

[0032] In each round of policy updates, the state value error function is updated according to equation (7), and the qualification trace is updated according to equation (8).

[0033] S105: Based on the current control strategy and strategy gradient, update the control strategy and generate new control commands by combining the regional control deviations of other regions.

[0034] In this embodiment, the control policy is updated based on the current control decision and the calculated policy gradient, as follows: (9) Where α is the policy learning rate, and its value ranges from 0 to 1. In system state s k Next action a k strategy, In the new state sk+1 Next action a k+1 The strategy.

[0035] Combination Figure 2 As shown, based on the updated control strategy and considering the regional control deviations of other regions, the control command for this region is generated, expressed as: (10) in, For the region i With the region j The coordination coefficient between regions is used to control the degree of influence of regional control deviations in other regions on the control commands of this region. N It is the number of all regions. ACE i and ACE j They are respectively regions i , j Regional control deviation, ACE k This region is in the 1st k Wheel area control deviation, K It adjusts the gain to balance the adjustment speed and stability.

[0036] In some embodiments, a PID controller is used to control the generator units to execute control commands, including adjusting the active power output of each generator unit within the drive area.

[0037] In some embodiments, the method further includes: updating the weight parameters of the linear mapping Q-value function based on the value function error, eligibility trace, and updated Q-value, as follows: (11) in, The weights are those given in the current system state. β The learning rate of the value function ranges from 0 to 1.

[0038] In some embodiments, the method further includes: The predicted deviation is calculated based on the area control deviation predicted by the PID controller and the area control deviation corresponding to the current system state. If the prediction deviation is greater than the preset deviation threshold, the weight of the linear mapping Q-value function is adjusted according to the control parameters of the PID controller.

[0039] In this embodiment, after executing the control command, the AGC prediction error for this region is calculated to ensure that the local approximation deviation converges quickly. The method for calculating the prediction deviation is as follows: (12) in, For the regioni The actual area control deviation, For the region i The predicted value of the regional control deviation is obtained by the PID controller.

[0040] When the prediction deviation exceeds the deviation threshold (e.g., 0.02 pu), the weighting parameters of the linear mapping Q-value function are adjusted according to the control parameters of the PID controller, as follows: (13) in, The proportional gain of the PID controller. The integral coefficient of the PID controller. For the region i In the k Prediction bias of the wheel, From the beginning to the number k The cumulative prediction error of the wheel is used to eliminate static bias. t The parameters are integers. A simple adjustment to the reinforcement learning parameters is made using a traditional PID controller, improving convergence speed and algorithm performance.

[0041] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0042] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0043] like Figure 3 As shown in the figure, this application provides an automatic generation control device for a distributed power system, comprising: The update module is used to execute the control command corresponding to the action in the current system state according to the current control strategy, and then update the Q value according to the current system state and the corresponding action, based on the linear mapping Q value function. The reward value calculation module is used to calculate the reward value according to the current system state and a preset reward function. The error calculation module is used to calculate the value function error based on the reward value and the updated Q value. The gradient calculation module is used to calculate the policy gradient based on the value function error, qualification trace, and state value function error. The strategy update module is used to update the control strategy based on the current control strategy and strategy gradient, and generate new control commands by combining the regional control deviations of other regions.

[0044] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware.

[0045] The apparatus described above is used to implement the corresponding methods in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0046] Figure 4 This embodiment illustrates a more specific hardware structure of an electronic device. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0047] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0048] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0049] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0050] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0051] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0052] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0053] The electronic devices described above are used to implement the corresponding methods in the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0054] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0055] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0056] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0057] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0058] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this disclosure.

Claims

1. An automatic generation control method for a distributed power system, characterized in that, include: After executing the control command corresponding to the action in the current system state according to the current control strategy, the Q value is updated according to the current system state and the corresponding action, using the linear mapping Q value function. Based on the current system state, calculate the reward value according to the preset reward function; Calculate the value function error based on the reward value and the updated Q value; The policy gradient is calculated based on the value function error, eligibility trace, and state value function error. Based on the current control strategy and strategy gradient, update the control strategy and generate new control commands by combining the regional control deviations of other regions.

2. The method according to claim 1, characterized in that, The linear mapping Q-value function is implemented based on a feedforward neural network model, and the linear mapping Q-value function is expressed as follows: ； in, It is a state-action characteristic, which is derived from the current system state. s k and corresponding actions a k Low-dimensional features extracted from the constructed features. , These are the parameters of the feedforward neural network model.

3. The method according to claim 2, characterized in that, The reward function is: ； in, R k As a reward value, ACE k Current system state s k The corresponding area control deviation, For the first m The unit carbon emission intensity coefficient of each generating unit M For the number of units, These are weighting coefficients. For the first m Carbon emission intensity of each unit under current system conditions.

4. The method according to claim 3, characterized in that, The method for calculating the value function error based on the reward value and the updated Q value is as follows: ； in, For the error of the value function, As a discount factor, For mixed sampling parameters, For the state value function, Q ( s k-1 , a k-1 ) is in the system state s k-1 Next action a k-1 The Q value.

5. The method according to claim 4, characterized in that, The policy gradient is calculated based on the value function error, eligibility trace, and state value function error, using the following method: ； in, e k ( s , a ) is a qualification record, These are the weight parameters of the feedforward neural network under the current system state; v k ( s k , a k Let be the state value error function, expressed as: ； in, It is a scaling factor. e k-1 ( s , a () represents the system state s k-1 The qualification record below System status s k-1 The state value function under [condition].

6. The method according to claim 5, characterized in that, Also includes: The weight parameters of the linear mapping Q-value function are updated based on the value function error, qualification trace, and updated Q-value, using the following method: ； in, For the updated weight parameters, Let be the learning rate of the value function.

7. The method according to claim 6, characterized in that, Also includes: The predicted deviation is calculated based on the area control deviation predicted by the PID controller and the area control deviation corresponding to the current system state. If the predicted deviation is greater than a preset deviation threshold, the weights of the linear mapping Q-value function are adjusted according to the control parameters of the PID controller, as follows: ； in, The proportional gain of the PID controller. The integral coefficient of the PID controller. e i,k For the region i In the k Prediction bias of the wheel, t It is an integer.

8. The method according to claim 1, characterized in that, The method for generating new control commands by combining regional control deviations from other regions is as follows: ； in, For the region i With the region j The synergy coefficient between them N It is the number of all regions. ACE i and ACE j They are respectively regions i , j Regional control deviation, ACE k For this region in the 1st k Wheel area control deviation, K To adjust the gain.

9. An automatic generation control device for a distributed power system, characterized in that, include: The update module is used to update the Q value according to the linear mapping Q value function after executing the control command corresponding to the action in the current system state according to the current control strategy. The reward value calculation module is used to calculate the reward value according to the current system state and a preset reward function. The error calculation module is used to calculate the value function error based on the reward value and the updated Q value; The gradient calculation module is used to calculate the policy gradient based on the value function error, qualification trace, and state value function error. The strategy update module is used to update the control strategy based on the current control strategy and strategy gradient, and generate new control commands by combining the regional control deviations of other regions.

10. The apparatus according to claim 9, characterized in that, The linear mapping Q-value function is implemented based on a feedforward neural network model, and the linear mapping Q-value function is expressed as follows: ； in, It is a state-action characteristic, which is derived from the current system state. s k and corresponding actions a k Low-dimensional features extracted from the constructed features. , These are the parameters of the feedforward neural network model.

Images