Power supply, load and energy storage cooperative control system and method based on reinforcement learning

Abstract

The invention discloses a power supply, load and energy storage cooperative control system and method based on reinforcement learning. The method comprises the following steps: obtaining initial state data representing the operation state of each part in a power supply, load and energy storage structure by using a power supply, load and energy storage cooperative communication mechanism; enhanced data representing independent operation states and associated operation states of all parts in the power supply, the load and the energy storage structure are obtained; and generating a corresponding power supply, load and energy storage cooperative control strategy according to the enhanced data, and completing power supply, load and energy storage cooperative control according to mutual cooperation of the control strategy and the power supply, load and energy storage cooperative communication mechanism. The method can solve the problem of decentralized and empirical management in traditional control, effectively deals with the volatility and intermittency of renewable energy sources, accurately coordinates power generation, load and energy storage, improves the power supply and demand balance capability, enhances the adaptability of the system to internal and external changes, guarantees the stable and efficient operation of the power system, and improves the resource utilization efficiency and economy.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and specifically to a power supply, load and energy storage collaborative control system and method based on reinforcement learning. Background Art

[0002] With the acceleration of energy transformation, the coordinated operation of power sources, loads and energy storage in power systems has become the key to ensuring power supply stability and efficient energy utilization. Under traditional technologies, the control of power sources, loads and energy storage is mostly decentralized and empirically managed. On the power generation side, different types of power sources (such as thermal power, hydropower, wind power, photovoltaic power, etc.) are difficult to effectively coordinate to cope with changes in power demand and the intermittent and volatile problems of renewable energy, resulting in low power generation efficiency or unstable power supply.

[0003] On the load side, there is a lack of effective control methods, and it is impossible to flexibly adjust electricity consumption according to the power generation situation, making it difficult to accurately maintain the balance between power supply and demand. Peak load periods are prone to power shortages, and low periods may cause energy waste. At the same time, the energy storage system has not been able to fully play its buffering and regulating role between the power supply and the load in traditional control. Its charging and discharging strategies are often not optimized, the utilization rate of energy storage resources is low, and it fails to effectively cooperate with the source and load in real time.

[0004] Furthermore, most existing technologies fail to take into account the combined impact of the system's macro-long-term trends and micro-real-time changes, and lack effective data processing and analysis mechanisms. They are unable to fully characterize the independent and related operating states of each part of the power supply, load and energy storage, and it is difficult to generate accurate and reasonable coordinated control strategies, resulting in challenges to the reliability, economy and stability of the power system.

[0005] In summary, a new reinforcement learning-based coordinated control strategy for power supply, load and energy storage is urgently needed to improve the comprehensive performance of power supply, load and energy storage systems. Summary of the invention

[0006] The purpose of the present invention is to provide a power supply, load and energy storage coordinated control system based on reinforcement learning on the one hand, and a power supply, load and energy storage coordinated control method based on reinforcement learning on the other hand. The system and method can solve the problems of decentralized and empirical management in traditional power supply, load and energy storage control, effectively cope with the volatility and intermittency of renewable energy, accurately coordinate power generation, load and energy storage, improve the ability to balance power supply and demand, enhance the system's adaptability to internal and external changes, ensure stable and efficient operation of the power system, and improve resource utilization efficiency and economy.

[0007] To achieve this purpose, the present invention designs a power supply, load and energy storage coordinated control system based on reinforcement learning, which is characterized by: it includes an initial data acquisition module, a reinforcement data acquisition module, and a reinforcement learning model module;

[0008] The initial data acquisition module is used to collect data on the power supply, load and energy storage structure in the power system by using the power supply, load and energy storage collaborative communication mechanism, and obtain the initial state data representing the operating state of each part of the power supply, load and energy storage structure;

[0009] The enhanced data acquisition module is used to perform data hierarchical enhancement processing based on the macro and micro levels on the initial state data representing the operating state of the power supply, load and energy storage structure, and obtain enhanced data representing the independent operating state and the associated operating state of each part of the power supply, load and energy storage structure;

[0010] The reinforcement learning model module is used to generate corresponding power supply, load and energy storage collaborative control strategies based on the reinforcement data that characterize the independent operating states and associated operating states of each part of the power supply, load and energy storage structure. The power supply, load and energy storage collaborative control strategies and the power supply, load and energy storage collaborative communication mechanism cooperate with each other to complete the collaborative control of power supply, load and energy storage.

[0011] Furthermore, a method for collecting data on the power supply, load and energy storage structure in the power system by utilizing the collaborative communication mechanism of the power supply, load and energy storage to obtain initial state data characterizing the operating state of each part of the power supply, load and energy storage structure includes: connecting with the power supply, load and energy storage structure through the power generation equipment interface of the power supply intelligent body, the power load interface of the load intelligent body and the energy storage equipment interface of the energy storage intelligent body to obtain initial state data characterizing the operating state of each part of the power supply, load and energy storage structure.

[0012] Furthermore, the initial status data characterizing the operating status of each part in the power supply, load and energy storage structure include: the power generation power, voltage, frequency, and operating status information of the power generation source collected by the power source intelligent agent; the real-time power consumption, power consumption time, and power demand change trend information of the power load collected by the load intelligent agent; the energy storage capacity, charging and discharging power, charging and discharging status, and health status information of the energy storage system collected by the energy storage intelligent agent.

[0013] Furthermore, the method for performing macro-level and micro-level data hierarchical enhanced processing on the initial state data representing the operating state of the power source, load and energy storage structure includes performing macro-level data analysis, micro-level data analysis, macro-level data processing and micro-level data processing on the initial state data:

[0014] Among them, the data analysis at the macro level includes overall balance analysis of power supply, load and energy storage system, long-term trend assessment and regional interaction analysis; among them, the overall balance analysis of power supply, load and energy storage system is to conduct power supply and demand balance analysis on the entire power supply, load and energy storage structure, based on the total power generation power on the power supply side and the total power consumption on the load side and the charge and discharge power of the energy storage system, to judge whether the system is in a balanced state at the current moment; the long-term trend assessment is based on the historical status data and current status data collected to characterize the operating status of each part of the power supply, load and energy storage structure, to obtain the change trend of the power supply, load and energy storage structure within a certain time range; the regional interaction analysis is used to analyze the power interaction between different local areas when the power supply, load and energy storage structure involve multiple local areas;

[0015] Micro-level data analysis includes equipment individual performance analysis, local interaction impact analysis and real-time state change analysis; equipment individual performance analysis analyzes the corresponding individual performance indicators for each power generation source, power load and energy storage device; local interaction impact analysis analyzes the interaction between the power source, load and energy storage structure within a local range; real-time state change analysis monitors the state changes of each part of the power source, load and energy storage in real time;

[0016] Data processing at the macro level includes overall goal setting, long-term planning revision and regional coordination strategy formulation; among them, overall goal setting is to set independent operation goals for each part of the power supply, load and energy storage system based on the results of the overall balance analysis of the power supply, load and energy storage system; long-term planning revision is based on the results of long-term trend assessment, and the long-term planning of the power supply, load and energy storage system is revised; regional coordination strategy formulation is based on the results of regional interaction analysis, and regional coordination strategy is formulated;

[0017] Data processing at the micro level includes generation of individual device control instructions, local optimization and adjustment, and real-time exception handling. Generation of individual device control instructions is based on the results of individual device performance analysis and real-time state change analysis to generate corresponding single device operation control instructions. Local optimization and adjustment is based on the results of local interaction impact analysis to optimize and adjust the control strategies of local power supplies, loads, and energy storage systems. Real-time exception handling is to initiate a real-time exception handling mechanism when micro-level analysis finds device abnormalities or local operation abnormalities.

[0018] Furthermore, the specific process of obtaining enhanced data characterizing the independent operating status and associated operating status of each part of the power supply, load and energy storage structure is as follows:

[0019] Perform data cluster analysis on the independent operation targets of each part of the power supply, load and energy storage system, the revised long-term plan of the power supply, load and energy storage system, the regional coordination strategy, the operation control instructions of a single device, and the control strategy of the optimized and adjusted local power supply and load and energy storage system, and obtain enhanced data characterizing the independent operation status and associated operation status of each part of the power supply, load and energy storage structure;

[0020] Furthermore, the enhanced data characterizing the independent and associated operating states of each part of the power supply, load and energy storage structure include:

[0021] The enhanced data that characterizes the independent operating status of each part of the power supply, load and energy storage structure include power supply data, load data and energy storage data; the power supply data specifically includes the total power generation stability index, the power supply structure rationality index and the average power supply operation efficiency; the load data specifically includes the total load demand characteristic index, the load structure composition ratio and the load forecast accuracy index; the energy storage data specifically includes the total capacity utilization rate of the energy storage system, the comprehensive health status of the energy storage system and the energy storage system response capability index;

[0022] The enhanced data that characterize the related operating status of each part of the power supply, load and energy storage structure include: power supply load related data, power supply energy storage related data and load energy storage related data; the power supply load related data specifically include power generation and power consumption matching indicators, power supply load supporting capacity indicators and load power supply feedback indicators; the power supply energy storage related data specifically include power generation and energy storage synergy indicators, power supply energy storage charging and discharging efficiency influencing indicators and energy storage power supply regulation auxiliary indicators; the load energy storage related data specifically include power consumption and energy storage interaction indicators, energy storage load power supply quality improvement indicators and load energy storage life impact indicators.

[0023] Furthermore, the method of using the reinforcement learning model to generate the corresponding power supply, load and energy storage coordinated control strategy according to the reinforcement data representing the independent operation state and the associated operation state of each part in the power supply, load and energy storage structure includes: presetting the reinforcement learning model: using the deep learning algorithm to perform the network architecture of the reinforcement learning model for the coordinated control of the power supply, load and energy storage, and obtaining the reinforcement learning model for the coordinated control of the power supply, load and energy storage;

[0024] By using the reinforcement data and corresponding historical control strategies that characterize the independent and associated operating states of each part of the power supply, load and energy storage structure, the power supply, load and energy storage collaborative control reinforcement learning model is trained and verified to generate the corresponding power supply, load and energy storage collaborative control strategy.

[0025] Furthermore, the method for coordinating the power supply, load and energy storage coordinated control strategy and the power supply, load and energy storage coordinated communication mechanism to complete the power supply, load and energy storage coordinated control includes:

[0026] The current power supply, load and energy storage structure are perceived by using the enhanced data representing the independent operation status and the associated operation status of each part in the power supply, load and energy storage structure, and the independent operation status and the associated operation status of each part in the current power supply, load and energy storage structure are obtained;

[0027] Generate a corresponding coordinated control strategy for the power supply, load and energy storage structure based on the independent operating status and associated operating status of each part of the current power supply, load and energy storage structure, and the control strategy is used to coordinate the control of the power supply, load and energy storage structure;

[0028] Based on the control strategy, the corresponding control strategy is sent to the corresponding parts of the power supply, load and energy storage structure using the power supply, load and energy storage collaborative communication mechanism.

[0029] The power supply, load and energy storage coordinated control method obtained according to the power supply, load and energy storage coordinated control system based on reinforcement learning includes:

[0030] The power supply, load and energy storage coordinated communication mechanism is used to collect data on the power supply, load and energy storage structure in the power system, and the initial state data representing the operating state of each part of the power supply, load and energy storage structure is obtained;

[0031] The initial state data representing the operating state of the power supply, load and energy storage structure are subjected to data hierarchical enhancement processing based on the macro and micro levels to obtain enhanced data representing the independent operating state and the associated operating state of each part of the power supply, load and energy storage structure;

[0032] According to the enhanced data characterizing the independent operating status and the associated operating status of each part in the power supply, load and energy storage structure, a corresponding power supply, load and energy storage collaborative control strategy is generated. The power supply, load and energy storage collaborative control strategy and the power supply, load and energy storage collaborative communication mechanism cooperate with each other to complete the power supply, load and energy storage collaborative control.

[0033] Beneficial effects of the present invention: The present invention is based on a reinforcement learning-based collaborative control strategy for power supply, load and energy storage, and utilizes a collaborative communication mechanism of power supply, load and energy storage to collect data, thereby acquiring initial data on the operating status of each part of the power supply, load and energy storage, and providing basic information for subsequent control; by processing the initial data through hierarchical reinforcement, the power supply, load and energy storage structure can be analyzed from macro and micro levels, and the reinforcement data of the overall operating status and related operating status can be mined to fully understand the system characteristics; with the help of a reinforcement learning model, the control strategy is generated using the reinforcement data, and the collaborative control of power supply, load and energy storage is realized in combination with a collaborative communication mechanism; this method solves the problem of decentralized and empirical management in traditional control, effectively copes with the volatility and intermittency of renewable energy, accurately coordinates power generation, load and energy storage, improves the ability to balance power supply and demand, enhances the system's adaptability to internal and external changes, ensures stable and efficient operation of the power system, and improves resource utilization efficiency and economy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A diagram of a method for coordinated control of power supply, load and energy storage based on reinforcement learning of the present invention;

[0035] Figure 2 It is a structural schematic diagram of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0037] Example 1

[0038] like Figure 2 As shown, a power supply, load and energy storage coordinated control system based on reinforcement learning includes an initial data acquisition module, a reinforcement data acquisition module, and a reinforcement learning model module;

[0039] The initial data acquisition module is used to collect data on the power supply, load and energy storage structure in the power system by using the power supply, load and energy storage collaborative communication mechanism, and obtain the initial state data representing the operating state of each part of the power supply, load and energy storage structure;

[0040] The enhanced data acquisition module is used to perform data hierarchical enhancement processing based on the macro and micro levels on the initial state data representing the operating state of the power supply, load and energy storage structure, and obtain enhanced data representing the independent operating state and the associated operating state of each part of the power supply, load and energy storage structure;

[0041] The reinforcement learning model module is used to generate corresponding power supply, load and energy storage collaborative control strategies based on the reinforcement data that characterize the independent operating states and associated operating states of each part of the power supply, load and energy storage structure. The power supply, load and energy storage collaborative control strategy and the power supply, load and energy storage collaborative communication mechanism cooperate with each other to complete the power supply, load and energy storage collaborative control.

[0042] The power supply, load and energy storage collaborative communication mechanism is any existing communication mechanism, such as a communication bus mechanism, which is intended to achieve communication between the power supply, load and energy storage, and is used for collaborative data collection and distribution between multiple intelligent agents obtained from the power supply, load and energy storage. For example, Ethernet communication is used to connect the power supply intelligent agent, load intelligent agent and energy storage intelligent agent with the power supply, load and energy storage system through a coaxial cable to achieve data transmission between the two.

[0043] The initial state data characterizing the operating states of each part in the power supply, load and energy storage structure refers to the initial state data of the operating states of specific and individual power generation equipment, power consumption equipment and energy storage equipment in the power supply, load and energy storage structure; the enhanced data characterizing the independent operating states and the associated operating states of each part in the power supply, load and energy storage structure refers to the independent operating state of the power supply part as a whole, the independent operating state of the load part as a whole and the independent operating state of the energy storage part as a whole, as well as the associated operating states between the power supply part, the load part and the energy storage part.

[0044] In the above technical scheme, the method of collecting data on the power supply, load and energy storage structure in the power system by utilizing the coordinated communication mechanism of power supply, load and energy storage to obtain the initial state data characterizing the operating state of each part of the power supply, load and energy storage structure includes: connecting with the power supply, load and energy storage structure through the power generation equipment interface of the power supply intelligent body, the power load interface of the load intelligent body and the energy storage equipment interface of the energy storage intelligent body to obtain the initial state data characterizing the operating state of each part of the power supply, load and energy storage structure.

[0045] The power supply, load and energy storage structure is also called an external system, and the external system includes at least a meteorological monitoring system, a power market trading system and a power grid dispatching center. The corresponding meteorological data, power trading data and power grid dispatching data are obtained by connecting the power supply, load and energy storage collaborative communication mechanism with the external system, and the corresponding external data is provided for further optimizing the power supply, load and energy storage collaborative control based on reinforcement learning. It should be noted that the initial data are all actual data required for the collaborative control of power supply, load and energy storage, and in order to optimize the collaborative control of power supply, load and energy storage, the initial data can further expand the data type.

[0046] In specific applications, the power generation equipment interface is connected to various power sources, including traditional thermal power generators, hydroelectric generators, wind turbines, solar photovoltaic panels and inverters, etc., so as to collect various data on the power generation side, such as temperature, pressure and other parameters of boilers in thermal power generation, water level information in hydropower, wind speed and wind direction data of wind power, photovoltaic light intensity and other environmental data, and integrate these data into information such as power generation power, voltage, frequency, and operating status, and transmit them through the collaborative communication mechanism of power supply, load and energy storage; the power load interface of this embodiment is connected to different types of power equipment such as industry, commerce and residence. For industrial power consumption, it can be connected to monitoring points of large motors, production lines and other equipment, Obtain its real-time power consumption, operating time, etc.; for commercial power consumption, it is connected to the lighting, air conditioning, elevators and other equipment in shopping malls and office buildings; for residential power consumption, it collects power consumption data through smart meters and other equipment, including the power consumption time and power consumption changes of various household appliances, so as to obtain information on the trend of power demand changes; the energy storage device interface of this embodiment: connected to the battery management system of the battery energy storage system, the monitoring system of the pumped storage power station, etc., so as to obtain data on the energy storage side, such as the voltage, temperature, and state of charge of each battery module in the battery energy storage, the water level of the upper and lower reservoirs of the pumped storage, and the operating status of the pump / turbine, so as to obtain information such as energy storage capacity, charging and discharging power, charging and discharging status, and health status. In addition, the existing intelligent agent technology is used to set up the power supply intelligent agent, load intelligent agent and energy storage intelligent agent, and the corresponding interfaces are combined to complete the digitization of the power supply, load and energy storage structure, providing a software and hardware foundation for the coordinated control of power supply, load and energy storage based on reinforcement learning.

[0047] In the above technical solution, the initial status data characterizing the operating status of each part of the power supply, load and energy storage structure include: the power generation power, voltage, frequency, and operating status information of the power generation source collected by the power source intelligent agent; the real-time power consumption, power consumption time, and power demand change trend information of the power load collected by the load intelligent agent; the energy storage capacity, charging and discharging power, charging and discharging status, and health status information of the energy storage system collected by the energy storage intelligent agent.

[0048] In the above technical solution, the method for performing macro-level and micro-level data hierarchical enhanced processing on the initial state data representing the operating state of the power supply, load and energy storage structure includes performing macro-level data analysis, micro-level data analysis, macro-level data processing and micro-level data processing on the initial state data:

[0049] Among them, the data analysis at the macro level includes overall balance analysis of power supply, load and energy storage system, long-term trend assessment and regional interaction analysis; among them, the overall balance analysis of power supply, load and energy storage system is to conduct power supply and demand balance analysis on the entire power supply, load and energy storage structure, based on the total power generation power on the power supply side and the total power consumption on the load side and the charge and discharge power of the energy storage system, to judge whether the system is in a balanced state at the current moment; the long-term trend assessment is based on the historical status data and current status data collected to characterize the operating status of each part of the power supply, load and energy storage structure, to obtain the change trend of the power supply, load and energy storage structure within a certain time range; the regional interaction analysis is used to analyze the power interaction between different local areas when the power supply, load and energy storage structure involve multiple local areas;

[0050] Micro-level data analysis includes equipment individual performance analysis, local interaction impact analysis and real-time state change analysis; equipment individual performance analysis analyzes the corresponding individual performance indicators for each power generation source, power load and energy storage device; local interaction impact analysis analyzes the interaction between the power source, load and energy storage structure within a local range; real-time state change analysis monitors the state changes of each part of the power source, load and energy storage in real time;

[0051] Data processing at the macro level includes overall goal setting, long-term planning revision and regional coordination strategy formulation; among them, overall goal setting is to set independent operation goals for each part of the power supply, load and energy storage system based on the results of the overall balance analysis of the power supply, load and energy storage system; long-term planning revision is based on the results of long-term trend assessment, and the long-term planning of the power supply, load and energy storage system is revised; regional coordination strategy formulation is based on the results of regional interaction analysis, and regional coordination strategy is formulated;

[0052] Data processing at the micro level includes generation of individual device control instructions, local optimization and adjustment, and real-time exception handling. Generation of individual device control instructions is based on the results of individual device performance analysis and real-time state change analysis to generate corresponding single device operation control instructions. Local optimization and adjustment is based on the results of local interaction impact analysis to optimize and adjust the control strategies of local power supplies, loads, and energy storage systems. Real-time exception handling is to initiate a real-time exception handling mechanism when micro-level analysis finds device abnormalities or local operation abnormalities. The overall balance analysis of the power supply, load and energy storage system of this embodiment includes but is not limited to judging that when the total power generation is greater than the sum of the power consumption and the energy storage charging power, there is a power surplus, and vice versa, there may be a power shortage; long-term trend assessment includes but is not limited to considering the development trends of different types of power generation sources (such as renewable energy and traditional energy) on the power supply side, such as the growth trend of solar and wind power generation capacity, the retirement plan of traditional thermal power, etc., and on the load side, analyzing the seasonal and cyclical changes in power loads in different industries, such as the peak of residential air-conditioning power consumption in summer and the increase of industrial power consumption in the peak production season; regional interaction analysis includes but is not limited to analyzing whether the excess power generation in a certain area can be effectively transmitted to the power shortage area through the power grid, and the impact of such transmission on the coordination of power supply, load and energy storage at the regional level.

[0053] The analysis of individual equipment performance includes the analysis of power generation equipment, such as the thermal efficiency of thermal power generators, the power curve characteristics of wind turbines, and the conversion efficiency of photovoltaic cells; in terms of power load, the power factor and power efficiency of various electrical equipment are analyzed; in terms of energy storage equipment, the focus is on the changes in the internal resistance of the battery and the charging and discharging efficiency under different working conditions. The analysis of local interactive impacts includes but is not limited to analyzing the matching of local photovoltaic power generation and the load of surrounding residents in a distributed energy community, as well as the role of local energy storage equipment in smoothing the impact of photovoltaic power generation fluctuations on local loads. Real-time state change analysis includes focusing on the real-time fluctuations of power generation for power sources, such as power changes caused by sudden changes in wind speed in wind power generation, and power adjustments caused by fuel supply adjustments in thermal power generation; the load side includes capturing the start, stop, and power mutation of electrical equipment; the energy storage system focuses on analyzing the real-time switching of charging and discharging states and the dynamic changes of energy storage capacity, so as to promptly detect abnormal states and potential control needs.

[0054] The overall goal setting adjustment includes if it is predicted that renewable energy generation will increase significantly in the future, the overall goal can be adjusted to improve the energy storage utilization rate of the energy storage system to better absorb excess electricity, while optimizing the demand response strategy on the load side to encourage more adjustable loads to use electricity during the peak period of renewable energy generation; long-term planning optimization includes in terms of power planning, deciding whether to add new power generation capacity and what type of power generation source (such as adding photovoltaic and wind power generation projects in areas rich in renewable energy resources); in load management, formulating long-term strategies to guide users in different industries to use electricity reasonably (such as promoting energy-saving equipment, adjusting industrial electricity use time, etc.); for energy storage planning, planning the additional capacity and layout of energy storage equipment to meet the long-term development needs of the system; regional coordination strategy formulation includes but is not limited to optimizing the dispatching plan of inter-regional transmission lines, determining inter-regional power trading strategies (such as formulating inter-regional electricity price mechanisms to promote the rational flow of electricity), and coordinating the power, load and energy storage collaborative control strategies of different regions to ensure that the coordinated operation between regions contributes to the stability and efficiency of the overall system. The generation of individual control instructions for the equipment in this embodiment includes, for power generation equipment, such as adjusting the fuel supply of thermal power generating sets and the pitch angle of wind turbines to optimize the power generation. On the load side, the start and stop and power of individual power consumption equipment are controlled by means of intelligent switches, electricity price incentives, etc.; for energy storage equipment, charging and discharging power instructions are issued to ensure that it operates in a safe and efficient state, while extending the life of the equipment; the local optimization adjustment of this embodiment includes, in the distributed energy system, adjusting the control parameters of the local photovoltaic inverter to better match the local load characteristics, optimizing the charging and discharging strategy of the local energy storage system to reduce the impact on the upper power grid, and coordinating the load response in the local range through local communication means to improve the reliability and quality of local power supply; the real-time abnormal processing of this embodiment includes, for power generation equipment failure, quickly adjusting the power generation of other power sources or enabling the backup power source; when problems such as overload occur on the load side, the system safety is ensured by cutting off non-critical loads or adjusting other adjustable loads; when an abnormality occurs in the energy storage device, its charging and discharging operation is stopped in time, and fault diagnosis and repair are performed, and the system operation strategy is adjusted to compensate for the impact of the lack of energy storage.

[0055] In the above technical solution, the specific process of obtaining the enhanced data characterizing the independent operation status and the associated operation status of each part in the power supply, load and energy storage structure is as follows:

[0056] Perform data cluster analysis on the independent operation targets of each part of the power supply, load and energy storage system, the revised long-term plan of the power supply, load and energy storage system, the regional coordination strategy, the operation control instructions of a single device, and the control strategy of the optimized and adjusted local power supply and load and energy storage system, and obtain enhanced data characterizing the independent operation status and associated operation status of each part of the power supply, load and energy storage structure;

[0057] The data clustering analysis divides the data into different groups or clusters based on the similarity between the data, so that the data points in the same cluster have high similarity, while the data points between different clusters have great differences. The data clustering analysis algorithm is used to clean and preprocess the data contained in the independent operation targets of each part of the power supply, load and energy storage system, the revised long-term planning of the power supply, load and energy storage system, the regional coordination strategy, the operation control instructions of a single device, and the control strategy of the optimized and adjusted local power supply and load and energy storage system. The independent operation data and associated data related to the power supply, load and energy storage contained in the data are identified. According to the similarity and difference between the data, the enhanced data characterizing the independent operation status and the associated operation status of each part in the power supply, load and energy storage structure are obtained, and the clustering results are verified by using cross-validation or hold-out method to ensure the reliability and accuracy of the results.

[0058] In the above technical solution, the enhanced data characterizing the independent operation status and associated operation status of each part of the power supply, load and energy storage structure include:

[0059] The enhanced data that characterizes the independent operating status of each part of the power supply, load and energy storage structure include power supply data, load data and energy storage data; power supply data specifically include total power generation stability index, power supply structure rationality index and power supply average operating efficiency; load data specifically include total load demand characteristic index, load structure composition ratio and load forecast accuracy index; energy storage data specifically include total capacity utilization rate of energy storage system, comprehensive health status of energy storage system and energy storage system response capability index; total power generation stability index is used to reflect the fluctuation of power generation of all power generation sources within a certain time range, for example, by calculating the standard deviation or coefficient of variation of power generation. If the index is low, it means that the overall power generation of the power source is relatively stable; otherwise, it means that the power generation fluctuates greatly, and the control strategy may need to be further adjusted.

[0060] The power structure rationality index evaluates the rationality of the power structure based on the proportion of installed capacity of various types of power generation and their power generation contribution ratio at different times (such as day, month, and year). For example, in areas rich in renewable energy, when the proportion of renewable energy power generation is high and the power supply can be stable, this indicator is more ideal; if there is over-reliance on a single type of power source (such as traditional thermal power), it may be necessary to consider optimizing the power structure.

[0061] The average operating efficiency of power supply is calculated by weighted average by comprehensively considering the operating efficiency of all power generation equipment. The operating efficiency can include the thermal efficiency of thermal power generation, the water energy utilization efficiency of hydropower, the conversion efficiency of new energy power generation, etc. This indicator reflects the economic efficiency and energy utilization level of the overall operation of the power supply part.

[0062] The total load demand characteristic index is used to describe the overall demand characteristics of all power loads in terms of time and power. For example, the peak and valley characteristics of the load are reflected by the shape of the load curve (including the daily load curve and the annual load curve), and the power factor of the load is calculated to evaluate the independent power quality of each part of the load.

[0063] The load structure composition ratio is to analyze the proportion of different types of electricity loads (such as industrial, commercial, and residential) in the total load, which helps to understand the composition characteristics of the load and its impact on the system. For example, in a system where industrial load accounts for a large proportion, its load fluctuation may be closely related to the industrial production cycle; while in a system where residential load is dominant, its load changes may be more affected by residents' living habits and seasonal factors.

[0064] The load forecast accuracy index is based on the comparison between historical load data and actual load data, and calculates the accuracy of load forecasting, which is used to measure the accuracy of the forecast of load demand change trend. This indicator is of great significance for the advance arrangement of power generation plans and energy storage scheduling. A higher accuracy rate means that load demand can be better met and the imbalance between power supply and demand can be reduced.

[0065] The total capacity utilization rate of the energy storage system is the ratio of the energy storage capacity actually used by the energy storage system in a certain period to the total energy storage capacity, reflecting the utilization of energy storage resources. For example, if this indicator is low, it may mean that the energy storage system is over-configured or the energy storage scheduling strategy is not reasonable and needs further optimization.

[0066] The comprehensive health status of the energy storage system is a comprehensive consideration of the health status indicators of the energy storage equipment (such as the battery pack in the battery energy storage), such as the change in the internal resistance of the battery, the degree of capacity attenuation, etc., and the independent health status of each part of the energy storage system is obtained through a certain weighted algorithm. This helps to discover potential problems of the energy storage system in advance, reasonably arrange maintenance and replacement plans, and ensure the long-term stable operation of the energy storage system.

[0067] The energy storage system response capability index is used to measure the response speed and capability of the energy storage system to the power system demand (such as frequency regulation and power compensation). For example, it calculates the time delay from receiving the charge and discharge instruction to actually reaching the specified charge and discharge power, and the maximum power change rate that the energy storage system can provide. This index reflects the important role of the energy storage system in maintaining the stability of the power system.

[0068] The enhanced data that characterize the related operating status of each part of the power supply, load and energy storage structure include: power supply load related data, power supply energy storage related data and load energy storage related data; the power supply load related data specifically include power generation and power consumption matching indicators, power supply load supporting capacity indicators and load power supply feedback indicators; the power supply energy storage related data specifically include power generation and energy storage synergy indicators, power supply energy storage charging and discharging efficiency influencing indicators and energy storage power supply regulation auxiliary indicators; the load energy storage related data specifically include power consumption and energy storage interaction indicators, energy storage load power supply quality improvement indicators and load energy storage life impact indicators.

[0069] It should be noted that the power-load correlation data, power-energy-storage correlation data and load-energy-storage correlation data are obtained by correlation analysis based on the aforementioned power data, load data and energy storage data, and the method used for the correlation analysis is any one of the existing correlation analysis methods.

[0070] In a specific application, the power generation and power consumption matching index of this embodiment calculates the integral of the absolute value of the difference between the power generation and the power consumption over a period of time, and the smaller the integral value, the higher the matching degree. At the same time, considering the consistency of the power generation change trend and the power demand change trend, when the power generation can increase (or decrease) accordingly with the increase (or decrease) of the power demand, the matching degree is better.

[0071] In a specific application, the power supply load support index of this embodiment evaluates the degree of effective support that the power supply can provide under load fluctuation conditions.

[0072] In a specific application, the load power supply feedback index of this embodiment analyzes the feedback effect of load changes (such as sudden increase or decrease of load, change of power factor, etc.) on the power supply operation state (such as frequency and voltage of power generation equipment, etc.). For example, when a large number of inductive loads are connected, the grid voltage will drop. It is observed how the power supply responds to this effect through self-regulation (such as excitation regulation of the generator), and the degree of influence of this regulation on the power supply operation efficiency and stability.

[0073] In a specific application, the power generation and energy storage coordination index of this embodiment measures the coordinated working between the power generation power and the charging and discharging power of the energy storage system.

[0074] In a specific application, the power supply energy storage charging and discharging efficiency influencing index of this embodiment analyzes the impact of different power generation states (such as different power generation powers, different power supply types) on the charging and discharging efficiency of the energy storage system.

[0075] In a specific application, the energy storage power supply regulation auxiliary index of this embodiment evaluates the role of the energy storage system in regulating the power supply output, such as the energy storage system smoothes the power fluctuations of the power supply (especially the renewable energy power supply) by absorbing or releasing electrical energy, thereby reducing the impact of sudden changes in power generation on the power grid.

[0076] In a specific application, the power consumption and energy storage interaction index of this embodiment observes the interaction between load changes and the charging and discharging behavior of the energy storage system.

[0077] In a specific application, the energy storage load power supply quality improvement index of this embodiment analyzes the role of the energy storage system in improving the load power supply quality. For example, when the power grid experiences voltage sag, frequency fluctuation, etc., the energy storage system can maintain the stability of the load terminal voltage and frequency by quickly releasing or absorbing electrical energy.

[0078] In a specific application, the load energy storage life impact index of this embodiment studies the impact of the load working mode (such as frequent high-power impacts, irregular power demand, etc.) on the life of the energy storage device.

[0079] Based on the above, this embodiment uses power-load correlation data, power-storage correlation data, and load-storage correlation data to provide accurate correlation data for the operation of the reinforcement learning model.

[0080] In the above technical solution, the method of using the reinforcement learning model to generate the corresponding power supply, load and energy storage coordinated control strategy according to the reinforcement data includes: presetting the reinforcement learning model: presetting the reinforcement learning model: using the deep learning algorithm to perform the network architecture of the reinforcement learning model for the coordinated control of power supply, load and energy storage, and obtaining the reinforcement learning model for the coordinated control of power supply, load and energy storage;

[0081] By using the reinforcement data and corresponding historical control strategies that characterize the independent and associated operating states of each part of the power supply, load and energy storage structure, the power supply, load and energy storage collaborative control reinforcement learning model is trained and verified to generate the corresponding power supply, load and energy storage collaborative control strategy.

[0082] In the above technical solution, the method for completing the coordinated control of power supply, load and energy storage by cooperating with the coordinated communication mechanism of power supply, load and energy storage includes:

[0083] The current power supply, load and energy storage structure are perceived by using the enhanced data representing the independent operation status and the associated operation status of each part in the power supply, load and energy storage structure, and the independent operation status and the associated operation status of each part in the current power supply, load and energy storage structure are obtained;

[0084] Generate a corresponding coordinated control strategy for the power supply, load and energy storage structure based on the independent operating status and associated operating status of each part of the current power supply, load and energy storage structure, and the control strategy is used to coordinate the control of the power supply, load and energy storage structure;

[0085] Based on the control strategy, the corresponding control strategy is sent to the corresponding parts of the power supply, load and energy storage structure using the power supply, load and energy storage collaborative communication mechanism.

[0086] The power supply, load and energy storage collaborative communication mechanism is responsible for information sharing and coordination within the power supply, load and energy storage structure, as well as responding to external systems;

[0087] The information sharing and coordination within the power supply, load and energy storage structure specifically means that during the control process, the power supply, load and energy storage collaborative communication mechanism continuously monitors the execution status of the power supply, load and energy storage structure, and shares the execution status; responding to the external system specifically means that the power supply, load and energy storage collaborative communication mechanism receives information from the external system, uses the reinforcement learning model to integrate the information and reinforcement information, and generates the corresponding control strategy.

[0088] Example 2

[0089] like Figure 1 As shown, the power supply, load and energy storage coordinated control method obtained according to the power supply, load and energy storage coordinated control system based on reinforcement learning includes:

[0090] The power supply, load and energy storage coordinated communication mechanism is used to collect data on the power supply, load and energy storage structure in the power system, and the initial state data representing the operating state of each part of the power supply, load and energy storage structure is obtained;

[0091] The initial state data representing the operating state of the power supply, load and energy storage structure are subjected to data hierarchical enhancement processing based on the macro and micro levels to obtain enhanced data representing the independent operating state and the associated operating state of each part of the power supply, load and energy storage structure;

[0092] According to the enhanced data characterizing the independent operating status and the associated operating status of each part in the power supply, load and energy storage structure, a corresponding power supply, load and energy storage collaborative control strategy is generated. The power supply, load and energy storage collaborative control strategy and the power supply, load and energy storage collaborative communication mechanism cooperate with each other to complete the power supply, load and energy storage collaborative control.

[0093] Example 3

[0094] A computer program product comprises a computer program, wherein when the computer program is executed by a processor, the steps of the method described in Embodiment 2 are implemented.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0096] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0097] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0098] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0100] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

Claims

1. A collaborative control system of power supply, load and energy storage based on reinforcement learning, characterized by: It includes an initial data acquisition module, a reinforcement data acquisition module, and a reinforcement learning model module; The initial data acquisition module is used to collect data on the power supply, load and energy storage structure in the power system by using the power supply, load and energy storage collaborative communication mechanism, and obtain the initial state data representing the operating state of each part of the power supply, load and energy storage structure; The enhanced data acquisition module is used to perform data hierarchical enhancement processing based on the macro and micro levels on the initial state data representing the operating state of the power supply, load and energy storage structure, and obtain enhanced data representing the independent operating state and the associated operating state of each part of the power supply, load and energy storage structure; The reinforcement learning model module is used to generate corresponding power supply, load and energy storage collaborative control strategies based on the reinforcement data that characterize the independent operating states and associated operating states of each part of the power supply, load and energy storage structure. The power supply, load and energy storage collaborative control strategies and the power supply, load and energy storage collaborative communication mechanism cooperate with each other to complete the collaborative control of power supply, load and energy storage.

2. The power supply, load and energy storage coordinated control system based on reinforcement learning according to claim 1 is characterized in that: The method of collecting data on the power supply, load and energy storage structure in the power system by utilizing the coordinated communication mechanism of the power supply, load and energy storage to obtain the initial state data characterizing the operating state of each part of the power supply, load and energy storage structure includes: connecting with the power supply, load and energy storage structure through the power generation equipment interface of the power supply intelligent body, the power load interface of the load intelligent body and the energy storage equipment interface of the energy storage intelligent body to obtain the initial state data characterizing the operating state of each part of the power supply, load and energy storage structure.

3. The power supply, load and energy storage coordinated control system based on reinforcement learning according to claim 2 is characterized in that: The initial status data characterizing the operating status of each part of the power supply, load and energy storage structure include: the power generation power, voltage, frequency, and operating status information of the power generation source collected by the power source intelligent agent; the real-time power consumption, power consumption time, and power demand change trend information of the power load collected by the load intelligent agent; the energy storage capacity, charging and discharging power, charging and discharging status, and health status information of the energy storage system collected by the energy storage intelligent agent.

4. The power supply, load and energy storage coordinated control system based on reinforcement learning according to claim 1, characterized in that: The method for performing macro-level and micro-level data hierarchical intensification processing on the initial state data representing the operating state of the power supply, load and energy storage structure includes macro-level data analysis, micro-level data analysis, macro-level data processing and micro-level data processing on the initial state data: Among them, the data analysis at the macro level includes overall balance analysis of power supply, load and energy storage system, long-term trend assessment and regional interaction analysis; among them, the overall balance analysis of power supply, load and energy storage system is to conduct power supply and demand balance analysis on the entire power supply, load and energy storage structure, based on the total power generation power on the power supply side and the total power consumption on the load side and the charge and discharge power of the energy storage system, to judge whether the system is in a balanced state at the current moment; the long-term trend assessment is based on the historical status data and current status data collected to characterize the operating status of each part of the power supply, load and energy storage structure, to obtain the change trend of the power supply, load and energy storage structure within a certain time range; the regional interaction analysis is used to analyze the power interaction between different local areas when the power supply, load and energy storage structure involve multiple local areas; Micro-level data analysis includes equipment individual performance analysis, local interaction impact analysis and real-time state change analysis; equipment individual performance analysis analyzes the corresponding individual performance indicators for each power generation source, power load and energy storage device; local interaction impact analysis analyzes the interaction between the power source, load and energy storage structure within a local range; real-time state change analysis monitors the state changes of each part of the power source, load and energy storage in real time; Data processing at the macro level includes overall goal setting, long-term planning revision and regional coordination strategy formulation; among them, overall goal setting is to set independent operation goals for each part of the power supply, load and energy storage system based on the results of the overall balance analysis of the power supply, load and energy storage system; long-term planning revision is based on the results of long-term trend assessment, and the long-term planning of the power supply, load and energy storage system is revised; regional coordination strategy formulation is based on the results of regional interaction analysis, and regional coordination strategy is formulated; Data processing at the micro level includes generation of individual device control instructions, local optimization and adjustment, and real-time exception handling. Generation of individual device control instructions is based on the results of individual device performance analysis and real-time state change analysis to generate corresponding single device operation control instructions. Local optimization and adjustment is based on the results of local interaction impact analysis to optimize and adjust the control strategies of local power supplies, loads, and energy storage systems. Real-time exception handling is to initiate a real-time exception handling mechanism when micro-level analysis finds device abnormalities or local operation abnormalities.

5. The power supply, load and energy storage coordinated control system based on reinforcement learning according to claim 4 is characterized in that: The specific process of obtaining enhanced data characterizing the independent operating status and associated operating status of each part of the power supply, load and energy storage structure is as follows: Data clustering analysis is performed on the independent operation targets of each part of the power supply, load and energy storage system, the revised long-term planning of the power supply, load and energy storage system, the regional coordination strategy, the operation control instructions of a single device, and the control strategy of the optimized and adjusted local power supply and load and energy storage system to obtain enhanced data characterizing the independent operation status and associated operation status of each part of the power supply, load and energy storage structure.

6. The power supply, load and energy storage coordinated control system based on reinforcement learning according to claim 5 is characterized in that: The enhanced data that characterizes the independent and associated operating states of each part of the power supply, load and energy storage structure include: The enhanced data that characterizes the independent operating status of each part of the power supply, load and energy storage structure include power supply data, load data and energy storage data; the power supply data specifically includes the total power generation stability index, the power supply structure rationality index and the average power supply operation efficiency; the load data specifically includes the total load demand characteristic index, the load structure composition ratio and the load forecast accuracy index; the energy storage data specifically includes the total capacity utilization rate of the energy storage system, the comprehensive health status of the energy storage system and the energy storage system response capability index; The enhanced data that characterize the related operating status of each part of the power supply, load and energy storage structure include: power supply load related data, power supply energy storage related data and load energy storage related data; the power supply load related data specifically include power generation and power consumption matching indicators, power supply load supporting capacity indicators and load power supply feedback indicators; the power supply energy storage related data specifically include power generation and energy storage synergy indicators, power supply energy storage charging and discharging efficiency influencing indicators and energy storage power supply regulation auxiliary indicators; the load energy storage related data specifically include power consumption and energy storage interaction indicators, energy storage load power supply quality improvement indicators and load energy storage life impact indicators.

7. The power supply, load and energy storage coordinated control system based on reinforcement learning according to claim 1, characterized in that: The method of using a reinforcement learning model to generate a corresponding power supply, load and energy storage coordinated control strategy according to reinforcement data representing the independent operating state and the associated operating state of each part in the power supply, load and energy storage structure includes: presetting the reinforcement learning model: using a deep learning algorithm to perform a network architecture of a reinforcement learning model for the coordinated control of power supply, load and energy storage, and obtaining a reinforcement learning model for the coordinated control of power supply, load and energy storage; By using the reinforcement data and corresponding historical control strategies that characterize the independent and associated operating states of each part of the power supply, load and energy storage structure, the power supply, load and energy storage collaborative control reinforcement learning model is trained and verified to generate the corresponding power supply, load and energy storage collaborative control strategy.

8. The power supply, load and energy storage coordinated control system based on reinforcement learning according to claim 1, characterized in that: The method for completing the coordinated control of power supply, load and energy storage by cooperating with the coordinated control strategy of power supply, load and energy storage and the coordinated communication mechanism of power supply, load and energy storage includes: The current power supply, load and energy storage structure are perceived by using the enhanced data representing the independent operation status and the associated operation status of each part in the power supply, load and energy storage structure, and the independent operation status and the associated operation status of each part in the current power supply, load and energy storage structure are obtained; Generate a corresponding coordinated control strategy for the power supply, load and energy storage structure based on the independent operating status and associated operating status of each part of the current power supply, load and energy storage structure, and the control strategy is used to coordinate the control of the power supply, load and energy storage structure; Based on the control strategy, the corresponding control strategy is sent to the corresponding parts of the power supply, load and energy storage structure using the power supply, load and energy storage collaborative communication mechanism.

9. A method for coordinated control of power supply, load and energy storage based on reinforcement learning according to the system of claim 1, comprising: The power supply, load and energy storage coordinated communication mechanism is used to collect data on the power supply, load and energy storage structure in the power system, and the initial state data representing the operating state of each part of the power supply, load and energy storage structure is obtained; The initial state data representing the operating state of the power supply, load and energy storage structure are subjected to data hierarchical enhancement processing based on the macro and micro levels to obtain enhanced data representing the independent operating state and the associated operating state of each part of the power supply, load and energy storage structure; According to the enhanced data characterizing the independent operating status and the associated operating status of each part in the power supply, load and energy storage structure, a corresponding power supply, load and energy storage collaborative control strategy is generated. The power supply, load and energy storage collaborative control strategy and the power supply, load and energy storage collaborative communication mechanism cooperate with each other to complete the power supply, load and energy storage collaborative control.

10. A computer program product, comprising a computer program / instruction, which implements the steps of the method according to claim 9 when executed by a processor.

Images