A method and system for dispatch control of a wind-solar-storage system and a computing device

Abstract

The application provides a method and system for scheduling control of a wind-solar-storage system and a computing device, the method comprising: obtaining and fusing real-time operation state data and source-load prediction data of the wind-solar-storage system to construct a comprehensive state vector; inputting the comprehensive state vector into a pre-trained hierarchical deep reinforcement learning intelligent agent strategy model to generate a scheduling reference trajectory; performing action optimization based on the scheduling reference trajectory and system safety constraints to output a scheduling instruction; controlling the operation of equipment in the wind-solar-storage system according to the scheduling instruction, and collecting the system response state after the scheduling instruction is executed in real time as state update input of the next scheduling period to form a data-driven closed-loop adaptive control. According to the technical scheme of the application, the rigid problem of the traditional scheduling strategy can be effectively solved, the scheduling strategy generation considering long-term economy and real-time safety can be realized, and adaptive evolution and high-reliable closed-loop control of the scheduling strategy can be realized.

Description

Technical Field

[0001] This invention relates to the fields of energy management and artificial intelligence technology, specifically to a method, system, and computing device for scheduling and control of wind, solar, and energy storage systems. Background Technology

[0002] Wind-solar-storage systems are energy systems that include wind power generation, photovoltaic power generation, and energy storage systems. As a key technological means to mitigate the fluctuations in new energy sources and improve the absorption capacity, the scientific and real-time operation and scheduling of wind-solar-storage systems are directly related to the safety, stability, and economic benefits of the power grid. However, existing wind-solar-storage system scheduling technologies still have defects and shortcomings in practical applications.

[0003] Most existing technologies make scheduling decisions based on a single time scale. If the focus is on long-term economic optimization, it is often difficult to cope with drastic fluctuations in wind and solar power on a minute or even second scale, resulting in large real-time control deviations. If the focus is on short-term real-time control, it is easy to get stuck in local optima and neglect long-term energy management and equipment lifespan.

[0004] In recent years, although deep reinforcement learning has been attempted to be applied to energy dispatch, its performance is highly dependent on the design of the "reward function." Existing deep reinforcement learning methods typically rely on human experts to manually design the structure and weight parameters of the reward function based on experience. However, wind, solar, and energy storage systems are complex, high-dimensional, nonlinear, and strongly coupled systems. Manually designing the reward function not only incurs extremely high trial-and-error costs but also makes it difficult to cover all extreme operating conditions. Once the reward function design is flawed, the agent often fails to converge to the optimal policy and may even produce dangerous actions that violate physical constraints.

[0005] Once deployed, existing scheduling strategies often have fixed parameters and logic, lacking the ability to iterate and optimize based on actual operational feedback. Furthermore, purely data-driven AI models often lack safety fallback mechanisms in practical engineering applications, leading to scheduling commands potentially exceeding the physical safety boundaries of equipment and posing security risks. Summary of the Invention

[0006] This invention aims to provide a method, system, and computing device for scheduling and control of wind, solar, and energy storage systems. It can effectively solve the problem of rigidity in traditional scheduling strategies, realize the generation of scheduling strategies that take into account both long-term economic efficiency and real-time security, and achieve adaptive evolution and highly reliable closed-loop control of scheduling strategies.

[0007] According to one aspect of the present invention, a method for scheduling and control of a wind-solar-storage system is provided, the method comprising: Acquire real-time operational status data of the wind-solar-storage system and source-load prediction data within a preset time window; The real-time operating status data and the source load prediction data are fused together to construct a comprehensive state vector that reflects the current physical state of the system and its future evolution trend. The integrated state vector is input into a pre-trained hierarchical deep reinforcement learning agent policy model to generate a scheduling reference trajectory that satisfies the goals of long-term economy and power smoothing. Based on the aforementioned scheduling reference trajectory and system safety constraints, action optimization is performed, and scheduling instructions that conform to the physical safety boundaries are output. The system controls the operation of equipment in the wind, solar and energy storage system according to the scheduling instructions, and collects the system response status after the execution of the scheduling instructions in real time. The system response status is used as the status update input for the next scheduling cycle, forming a data-driven closed-loop adaptive control.

[0008] According to some embodiments, the method further includes pre-training the hierarchical deep reinforcement learning agent policy model, wherein the hierarchical deep reinforcement learning agent policy model includes an upper-layer scheduling agent and a lower-layer security control agent, and the pre-training process of the hierarchical deep reinforcement learning agent policy model includes: Construct a digital simulation environment for the wind-solar-storage system and initialize the hierarchical deep reinforcement learning agent policy model and its initial reward function; Using the reward function and hyperparameters of the current round, the hierarchical deep reinforcement learning agent policy model is driven to explore multiple rounds in the digital simulation environment, collecting multiple training trajectory data including state, action, reward and final performance index; Key performance indicators of the training trajectory data are extracted, combined with a preset scheduling target text description, organized into structured natural language prompts, and input into a large language model. The gap between the training trajectory performance and the scheduling target is analyzed by large language model analysis, the shortcomings of the current reward function are inferred, and optimization suggestions containing new reward items or modified reward function structure are generated. The optimization suggestions are input into a preset verification model for security and effectiveness verification; Repeat the joint iterative optimization training until the preset convergence condition is met, and output the finally trained and mature hierarchical deep reinforcement learning agent policy model and its corresponding final reward function.

[0009] According to some embodiments, the optimization suggestions are input into a preset verification model for security and effectiveness verification, including: If the verification fails, either regenerate the optimization suggestion or maintain the current reward function; If the verification is successful, the verified optimization suggestions are analyzed, a new generation reward function is instantiated in the program, and the large language model generates the hyperparameter adjustment strategy required for the next round of training based on the convergence of the training trajectory.

[0010] According to some embodiments, acquiring real-time operating status data of the wind-solar-storage system and source-load prediction data within a preset time window includes: By fusing historical time-series data and external environmental data and inputting them into the prediction model, the time-series evolution characteristics of source-load-storage data are identified. Based on the temporal evolution characteristics and the current state, the predicted source load data within the preset time window is deduced.

[0011] According to some embodiments, the real-time operating status data and the source load prediction data are fused to construct a comprehensive state vector reflecting the current physical state and future evolution trend of the system, including: The real-time operating status data is normalized. Temporal features are extracted from the source load prediction data; The normalized real-time operating status data is concatenated with the source load prediction data after time-series feature extraction to generate the comprehensive state vector containing current physical state features and future evolution trend features.

[0012] According to some embodiments, the comprehensive state vector is input into a pre-trained hierarchical deep reinforcement learning agent policy model to generate a scheduling reference trajectory that satisfies long-term economic efficiency and power smoothing objectives, including: Based on the comprehensive state vector, the upper-level scheduling agent plans the scheduling actions of the wind-solar-storage system on a preset planning time scale with the goals of long-term economic efficiency and power smoothness, and outputs a scheduling reference trajectory that satisfies multi-objective optimization. The scheduling reference trajectory is used as a safety boundary and input benchmark and passed to the lower-level safety control agent.

[0013] According to some embodiments, action optimization is performed based on the scheduling reference trajectory and system safety constraints to output scheduling instructions that conform to physical safety boundaries, including: The lower-level security control agent parses the scheduling reference trajectory into a target benchmark within a preset time scale, and constructs a local security action space by combining the real-time physical state of the current system. The lower-level security control agent, based on the system security constraints, performs high-frequency, refined action optimization on the scheduling reference trajectory, corrects the deviations in the scheduling plan, and finally generates and outputs the scheduling instructions that conform to the physical security boundaries.

[0014] According to some embodiments, a large language model analyzes the gap between the training trajectory performance and the scheduling target, infers the shortcomings of the current reward function, and generates optimization suggestions that include new reward items or modified reward function structures, including: The large language model identifies abnormal training trajectories that cause the performance degradation of the wind-solar-storage system based on the key performance indicators. Based on the state and action sequence corresponding to the abnormal training trajectory, infer the missing penalty term and insufficient incentive term in the current reward function; Generate optimization suggestion text containing new basic reward items, modified weight coefficients of existing reward items, and adjusted reward item calculation logic.

[0015] According to another aspect of the present invention, a system for scheduling and control of a wind-solar-storage system is provided, the system comprising: The data acquisition module is used to acquire real-time operating status data of the wind-solar-storage system and source-load prediction data within a preset time window; The state construction module is used to fuse the real-time operating state data with the source load prediction data to construct a comprehensive state vector that reflects the current physical state of the system and its future evolution trend. The trajectory generation module is used to input the comprehensive state vector into a pre-trained hierarchical deep reinforcement learning agent policy model to generate a scheduling reference trajectory that satisfies the goals of long-term economy and power smoothing. The instruction output module is used to optimize actions based on the scheduling reference trajectory and system safety constraints, and output scheduling instructions that conform to the physical safety boundary. The instruction execution and status acquisition module is used to control the operation of equipment in the wind, solar and energy storage system according to the scheduling instructions, and to collect the system response status after the execution of the scheduling instructions in real time. The system response status is used as the status update input for the next scheduling cycle, forming a data-driven closed-loop adaptive control.

[0016] According to another aspect of the present invention, a computing device is provided, comprising: Processor; and A memory that stores a computer program, which, when executed by the processor, implements the method as described in any of the preceding methods.

[0017] The technical solutions provided by the embodiments of the present invention have the following beneficial effects: This invention constructs a comprehensive state vector by fusing real-time state and source-load prediction data, and utilizes a hierarchical deep reinforcement learning model to generate a scheduling reference trajectory that balances economy and power smoothness over a long time scale. Combined with system safety constraints, it performs refined action optimization on a short time scale to output scheduling instructions that conform to physical safety boundaries. Finally, by controlling equipment operation and collecting response status in real time as update input for the next cycle, a data-driven closed-loop adaptive control mechanism is formed. This invention effectively solves the problem of rigidity in traditional scheduling strategies, achieving scheduling strategy generation that balances long-term economy and real-time safety, and realizing adaptive evolution and highly reliable closed-loop control of the scheduling strategy.

[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0019] Figure 1 A flowchart illustrating a method for scheduling and control of a wind, solar, and energy storage system according to an example embodiment is shown.

[0020] Figure 2 This diagram illustrates the pre-training of a hierarchical deep reinforcement learning agent policy model based on a large language model and deep reinforcement learning, according to an example embodiment.

[0021] Figure 3 A schematic diagram of a wind-solar-storage system according to an example embodiment is shown.

[0022] Figure 4 A schematic diagram of a system for scheduling control of a wind, solar and energy storage system is shown according to an example embodiment.

[0023] Figure 5 A block diagram of a computing device according to an exemplary embodiment is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0025] Figure 1 A flowchart illustrating a method for scheduling and control of a wind, solar, and energy storage system according to an example embodiment is shown.

[0026] According to some embodiments, a wind-solar-storage system includes a photovoltaic power generation unit, a wind power generation unit, at least one controllable power generation unit, an energy storage system, and a load. A digital simulation environment for the wind-solar-storage system is used to simulate the power output of the photovoltaic power generation unit and the wind power generation unit, the operating characteristics of the controllable power generation unit, the charging and discharging process of the energy storage system, and the load demand. The output of the dispatch decision includes start / stop and output commands for the controllable power generation unit, and charging / discharging power commands for the energy storage system. The controllable power generation unit includes at least one of a diesel generator, a gas turbine, or a fuel cell, and the energy storage system is a battery energy storage system.

[0027] See Figure 1 In S101, real-time operating status data of the wind-solar-storage system and source-load prediction data within a preset time window are obtained.

[0028] According to some embodiments, historical time-series data and external environmental data are fused and input into a prediction model to identify the time-series evolution characteristics of source-load-storage data. Based on these time-series evolution characteristics and the current state, the predicted source-load data within a preset time window is extrapolated. The predicted source-load data includes at least one of wind power output prediction data, photovoltaic power generation interval prediction data, and load prediction data.

[0029] According to some embodiments, data from the energy storage system's battery management system (such as cell voltage, temperature, SOC, SOH), wind turbine pitch angle and speed, and photovoltaic inverter operating status are acquired, along with the physical constraint data of the sensing equipment itself. When acquiring load data, calendar information (weekdays / holidays), electricity price signals, and user behavior characteristics are combined to construct a spatiotemporal correlation map of "source-load-storage" to identify the complementary or conflicting characteristics of load and power generation in time.

[0030] In some implementations, raw data often contains noise and missing values, which can affect prediction accuracy if directly input into the model; therefore, preprocessing is necessary. Outlier detection and repair utilizes isolated forests or the statistical 3σ principle to remove abrupt data jumps caused by sensor malfunctions. For missing data, interpolation based on time series characteristics or generative adversarial networks are used for data imputation to ensure temporal continuity. Multi-scale eigenvalue decomposition uses variational mode decomposition or wavelet transform to decompose the non-stationary wind and solar power sequence into eigenmode functions of different frequencies. High-frequency components (random fluctuations) and low-frequency components (trend terms) are separated and input into different prediction branches, reducing the difficulty of model fitting.

[0031] In S103, the real-time operating status data and the source load prediction data are fused together to construct a comprehensive state vector that reflects the current physical state of the system and its future evolution trend.

[0032] According to some embodiments, the real-time operating status data is normalized, and the source load prediction data is subjected to time-series feature extraction. The normalized real-time operating status data and the source load prediction data after time-series feature extraction are fused and concatenated to generate the comprehensive state vector containing current physical state features and future evolution trend features.

[0033] According to some embodiments, multi-scale temporal correlation features of the comprehensive state vector are extracted to capture the fluctuation patterns of wind and solar power output and load. Combined with the physical connection and interaction data of the equipment in the wind-solar-storage system, dynamic correlation features between equipment are learned through graph neural networks. The multi-scale temporal correlation features and dynamic correlation features are concatenated and their dimensions compressed to generate a low-dimensional comprehensive feature representation, which serves as the basic input for the decision-making of the upper-level intelligent agent.

[0034] Specifically, the collected multi-source heterogeneous real-time operating status data is analyzed, noise removal and missing value imputation based on adjacent time points are performed, and aggregation and timestamp binding are performed based on a preset time window granularity to construct a standardized unified feature vector for real-time operating status. Simultaneously, long short-term memory networks or gated recurrent units are used to extract temporal features from the source load prediction data, capturing long-term temporal dependencies. For highly volatile subsequences, empirical wavelet transform combined with sample entropy and singular spectrum analysis is used for secondary decomposition to reduce data complexity and extract the optimal feature set. The information entropy, correlation coefficient, and modal complementarity index between the real-time operating status features and the source load prediction features are calculated to generate a correlation enhancement factor and assign dynamic weights accordingly. Based on these dynamic weights, the normalized unified feature vector for real-time operating status and the extracted source load prediction temporal feature vector are weighted, combined, and concatenated to generate the comprehensive state vector.

[0035] In S105, the integrated state vector is input into the pre-trained hierarchical deep reinforcement learning agent policy model to generate a scheduling reference trajectory that satisfies the objectives of long-term economy and power smoothing.

[0036] According to some embodiments, the scheduling reference trajectory defines the desired path for the macroscopic power allocation target and energy storage state of charge of the wind-solar-storage system over a long period. As a guiding benchmark for lower-level control, the scheduling reference trajectory clarifies the economic operating range for each time period, providing directional guidance for fine-grained optimization of actions at preset time scales, ensuring that the system achieves optimal overall operating costs while satisfying long-term energy balance.

[0037] Figure 2 This diagram illustrates the pre-training of a hierarchical deep reinforcement learning agent policy model based on a large language model and deep reinforcement learning, according to an example embodiment.

[0038] See Figure 2The method of the present invention further includes pre-training the hierarchical deep reinforcement learning agent policy model, wherein the hierarchical deep reinforcement learning agent policy model includes an upper-layer scheduling agent and a lower-layer safety control agent. The pre-training process of the hierarchical deep reinforcement learning agent policy model includes: constructing a digital simulation environment of the wind-solar-storage system, and initializing the hierarchical deep reinforcement learning agent policy model and its initial reward function. Using the reward function and hyperparameters of the current round, the hierarchical deep reinforcement learning agent policy model is driven to explore multiple rounds in the digital simulation environment, collecting multiple training trajectory data including states, actions, rewards, and final performance indicators.

[0039] Key performance indicators of the training trajectory data are extracted and combined with a pre-defined scheduling target text description to form structured natural language prompts, which are then input into a large language model. The large language model analyzes the gap between the training trajectory performance and the scheduling target, infers the shortcomings of the current reward function, and generates optimization suggestions that include new reward items or a modified reward function structure. The optimization suggestions are input into a pre-defined validation model for safety and effectiveness verification. If the verification fails, the optimization suggestions are regenerated or the current reward function is maintained; if the verification passes, the verified optimization suggestions are parsed, a new generation reward function is instantiated in the program, and the large language model generates the hyperparameter adjustment strategy required for the next round of training based on the convergence of the training trajectory. The joint iterative optimization training is repeated until the pre-defined convergence condition is met, and the finally trained and mature hierarchical deep reinforcement learning agent policy model and its corresponding final reward function are output.

[0040] According to some embodiments, the preset scheduling objective text description includes at least one of the following: minimizing the total system operating cost, maximizing the renewable energy absorption rate, maintaining the energy storage system's state of charge within a safe range, and reducing the start-up and shutdown frequency of controllable power generation units.

[0041] According to some embodiments, the method for constructing prompt words includes: extracting training trajectory data. The key performance indicators, along with the scheduling optimization objective text description and the current reward function, are discussed. The definitions are organized into natural language instructions, which guide the large language model to analyze the effect of the current reward function and output optimization suggestions. The optimization result output by the large language model is a structured parameter text, which is parsed to obtain the new reward function. The weight coefficient vector is [w1, w2, w3, w4]. The convergence condition is that the new reward function generated by the large language model in N consecutive iterations... The change in the weight coefficients is lower than a preset threshold, or the performance improvement of the deep reinforcement learning agent in the independent verification environment is lower than a preset threshold.

[0042] According to some embodiments, the verification model is used to ensure the security of the scheduling strategy, including at least one of hard constraint verification and simulation environment backtesting. The verification model verifies the security and effectiveness of the reward function optimization suggestions generated by the large language model, preventing logical errors in the reward function or system operation risks caused by the "illusion" of the large language model.

[0043] Specifically, the hard constraint verification is rule-based. Through the physical safety rule base of the wind, solar and energy storage system and the code syntax checker, fast static verification is performed to check whether the reward function code or parameter format output by the large language model conforms to the programming specifications, ensuring that it can be correctly parsed and executed by the program, and verifying whether the adjusted weight coefficients or newly added penalty terms of the large language model exceed the preset physical safety boundary.

[0044] The simulation environment backtesting is based on lightweight simulation. Using the digital simulation environment of the wind-solar-storage system, a short-term "trial run" is conducted on the agent applying the new reward function to perform dynamic verification. The new generation reward function is loaded into the digital simulation environment, and preset typical scenarios are run. If the agent triggers the system protection mechanism or a non-numerical value appears in the action space during the trial run, the verification is considered failed. If the system runs smoothly and key indicators do not deteriorate, the verification is considered successful.

[0045] According to some embodiments, the large language model identifies abnormal training trajectories that cause a decline in the performance of the wind-solar-storage system based on the key performance indicators; based on the state and action sequences corresponding to the abnormal training trajectories, it infers the missing penalty terms and insufficient incentive terms in the current reward function; and generates optimization suggestion text that includes adding basic reward terms, modifying the weight coefficients of existing reward terms, and adjusting the calculation logic of reward terms.

[0046] See Figure 3 , Figure 3 A schematic diagram of a wind-solar-storage system is shown. This embodiment uses a typical wind-solar-diesel-storage microgrid as an example. The system includes: a 100kW photovoltaic array, a 150kW wind turbine, a 200kW diesel generator, a 500kWh lithium battery energy storage system, and a residential area with a maximum load of 300kW. The dispatch cycle is one day (24 hours, with a time step of 15 minutes). The fuel cost of the diesel generator is calculated using a quadratic function. Calculated at (yuan / 15 minutes).

[0047] According to some embodiments, the digital simulation environment is built based on the Python language and the OpenAI Gym framework. The digital simulation environment is used to simulate the power output of the photovoltaic power generation unit and the wind power generation unit, the operating characteristics of the controllable power generation unit, the charging and discharging process of the energy storage system, and the load demand. The output of the scheduling decision includes start-stop and output commands for the controllable power generation unit and charging and discharging power commands for the energy storage system.

[0048] The deep reinforcement learning agent employs a proximal policy optimization algorithm. Initial reward function. The simple sum of the four basic reward items is set as follows:

[0049] The calculation formulas for each basic reward item include: Incentives related to operational economic efficiency :

[0050] in, The output power of the controllable power generation unit is given by a, b, and c, which are cost coefficients. For its power generation cost, These are the electricity purchase price and the electricity sales price, respectively. Normalized reference value for power purchased and sold Set it to 5000 yuan.

[0051] Awards related to power supply reliability :

[0052] in, This refers to the amount of power supply shortage caused by insufficient power during a given period. This represents the total electricity demand during the specified time period. The average voltage of the critical node, Rated voltage, The coefficients used to balance the power shortage penalty and voltage deviation penalty are all initialized to 1.0.

[0053] Incentives for renewable energy consumption :

[0054] in, This represents the total amount of abandoned solar and wind power. This represents the theoretical maximum power generation of photovoltaic and wind power.

[0055] Awards related to the health of energy storage systems :

[0056] in, This is an indicator variable; it is 1 if the charging / discharging power or depth exceeds the safety threshold, and 0 otherwise. This refers to the state of charge of the energy storage battery. For the optimal state of charge, The penalty coefficient is initialized to 1.0.

[0057] Furthermore, the reward function is iteratively optimized during training. Training includes trajectory collection, reward function optimization, and function update.

[0058] Trajectory collection includes: using the current reward function R k The intelligent agent was trained in multiple scenarios simulating different weather days, and 100 complete daily scheduling trajectories τ were collected. k Each trajectory contains 96 time steps of state (PV / wind power / load power, energy storage charge, diesel engine state), action (diesel engine output command, energy storage charging and discharging power), and corresponding instantaneous reward r. k .

[0059] Reward function optimization includes: analyzing the trajectory τ k The average daily operating cost was calculated to be 1250 yuan, the average power curtailment rate was 15%, and the load shortage rate was 0.5%. Subsequently, the following prompt words were constructed to input the large language model: "The reward function of the current scheduling system is: R = . For operational efficiency, For power supply reliability, For the consumption of renewable energy, For the health of the energy storage system. The latest training results show: daily cost of 1250 yuan, curtailment rate of 15%, and power shortage rate of 0.5%. The scheduling objective is: to significantly reduce operating costs and curtailment while ensuring basic power supply (power shortage rate <1%). Please analyze whether the current reward function weights [w1, w2, w3, w4] = [1.0, 1.0, 1.0, 1.0] are reasonable, and output a set of optimized new weights. Only the JSON format is needed: {"weights": [w1, w2, w3, w4], "reason": "..."}. After large language model analysis, the output structured text suggestion is: "{"weights": [2.5, 0.8, 2.0, 1.2], "reason": "Insufficient cost penalty, excessively high reliability weights lead to overly conservative approach in order to ensure power supply. It is recommended to adjust the weights to strengthen economic efficiency and absorption targets."}". Parsing this text updates the reward function to... This will be used for the next round of training.

[0060] According to some embodiments, after instantiating the next-generation reward function in the program, the large language model simultaneously generates hyperparameter tuning strategies for the next round of training based on the convergence of the training trajectory. These hyperparameter tuning strategies include: when the reward value of the training trajectory oscillates or shows no improvement for a long period, generating a hyperparameter tuning strategy to reduce the neural network learning rate or increase the exploration noise parameter; and when the training trajectory converges rapidly but the key performance indicators fail to meet the targets, generating a hyperparameter tuning strategy to increase the exploration step size and / or adjust the discount factor.

[0061] According to some embodiments, the above iterative process is repeated until a preset convergence condition is met. The preset convergence condition includes: in a preset number of consecutive iterations, the change in the reward function structure and weight coefficients generated by the large language model is lower than a preset threshold; the continuous improvement in the comprehensive performance score of the deep reinforcement learning agent on the independent validation set is lower than a preset threshold; and the natural language analysis report output by the large language model clearly determines that the current reward function has met all preset scheduling objectives. For example, after the fifth iteration, the weight vector generated by the large language model tends to stabilize (the norm of weight change is less than 0.1 for three consecutive iterations), and the agent's average daily cost on the validation scenario set drops to 1050 yuan, the power abandonment rate drops to 8%, the power shortage rate remains at 0.8%, and the performance improvement Δ is lower than a preset threshold (5%). Training terminates, and the final agent model and its final reward function R are output. final (Weights are [2.7, 0.7, 2.3, 1.1]).

[0062] According to some embodiments, the pre-trained hierarchical deep reinforcement learning agent policy model includes an upper-layer scheduling agent and a lower-layer safety control agent. The upper-layer scheduling agent, based on the comprehensive state vector obtained in S103, plans the scheduling actions of the wind-solar-storage system on a preset planning time scale, with long-term economic efficiency, renewable energy consumption, and power smoothing as objectives, and outputs a scheduling reference trajectory that satisfies multi-objective optimization. This scheduling reference trajectory is then used as a safety boundary and input benchmark and passed to the lower-layer safety control agent.

[0063] In S107, action optimization is performed based on the scheduling reference trajectory and system safety constraints, and scheduling instructions that conform to the physical safety boundary are output.

[0064] According to some embodiments, the lower-layer security control agent parses the scheduling reference trajectory into a target benchmark within a preset time scale, and constructs a local security action space by combining it with the real-time physical state of the current system. Based on the system security constraints, the lower-layer security control agent performs high-frequency, refined action optimization on the scheduling reference trajectory, corrects deviations in the scheduling plan, and finally generates and outputs the scheduling instructions that conform to the physical security boundaries.

[0065] Specifically, the scheduling reference trajectory generated by the upper-level planning is parsed into target benchmarks for each time step within a preset time scale (short time scale), and physical state data of the wind-solar-storage system are collected in real time. Based on system safety constraints, a local safety action space is constructed, including the upper and lower limits of controllable power generation unit output, the safe range of the energy storage system's state of charge, and the transmission capacity limitations of the lines. Guided by the target benchmarks, the lower-level safety control agent performs high-frequency, refined action optimization within the local safety action space by solving an optimization problem aimed at minimizing the deviation from the target benchmarks and operating costs. Finally, the optimal action sequence obtained through optimization is output as a scheduling instruction that conforms to the physical safety boundaries to correct deviations in the upper-level scheduling plan and issue it for execution.

[0066] According to some embodiments, system safety constraints cover both the physical limits of equipment and the stability of grid operation. Physical constraints include the safe range of energy storage state of charge, charging and discharging power limits, and ramp rate restrictions for controllable power generation units, while stability constraints involve the range of node voltage deviations and the upper limit of line transmission capacity.

[0067] In S109, the operation of equipment in the wind, solar and energy storage system is controlled according to the scheduling instructions, and the system response status after the execution of the scheduling instructions is collected in real time. The system response status is used as the status update input for the next scheduling cycle, forming a data-driven closed-loop adaptive control.

[0068] According to some embodiments, the operating status of a wind-solar-storage system (including photovoltaic, wind power, controllable power generation units, energy storage, and loads) is collected in real time. The status is then input into a pre-trained deep reinforcement learning agent policy model, which outputs scheduling instructions. The scheduling instructions include the start-up, shutdown, and power output of the controllable power generation units, as well as the charging and discharging power of the energy storage.

[0069] Specifically, the scheduling instructions are issued to the controllable power generation units and energy storage systems within the wind-solar-storage system to control their start-up, shutdown, output, and charging / discharging power. The system response status after executing the scheduling instructions is collected in real time. This system response status includes the actual power of photovoltaic and wind power, the state of charge of the energy storage system, load demand, and the voltage of key nodes. This system response status is used as the state update input for the next scheduling cycle and input into the deployed hierarchical deep reinforcement learning agent strategy model to generate new scheduling instructions, thus forming a data-driven closed-loop adaptive control system of "state acquisition—scheduling decision—instruction execution—response feedback—state update".

[0070] This invention solves the core problem of relying on expert manual design by automatically optimizing the reward function of deep reinforcement learning through a large language model. This method can automatically balance multiple objectives such as economy and safety, guiding the agent to learn better scheduling strategies faster, and significantly improving the scheduling performance and automation level of wind, solar, and energy storage systems.

[0071] This invention effectively addresses the problems of fluctuating renewable energy output and rigid traditional scheduling strategies, achieving global optimization of system operating efficiency and economic benefits. By fusing real-time operating status and source-load prediction data to construct a comprehensive state vector, the method enables the system to accurately identify the periodic characteristics and random fluctuations of source-load-storage data, thereby achieving accurate prediction of future evolution trends and providing a reliable basis for proactive decision-making. Employing a hierarchical deep reinforcement learning architecture, the upper-level scheduling agent plans a reference trajectory that balances long-term economics and power smoothing over a long timescale, while the lower-level safety control agent performs refined action optimization over a short timescale. This mechanism not only breaks the limitations of traditional single-objective scheduling but also automatically corrects deviations under complex operating conditions, ensuring that the output scheduling commands always conform to physical safety boundaries, significantly improving the system's safety and stability.

[0072] Furthermore, this invention innovatively introduces a large language model and a validation model into the pre-training and iterative optimization process of the agent. The large language model can automatically analyze the training trajectory and infer the shortcomings of the reward function, generate targeted optimization suggestions, and automatically update the reward function or adjust hyperparameters after passing safety and effectiveness verification. This data-driven closed-loop adaptive control mechanism completely gets rid of the limitations of manually setting parameters based on experience, enabling the scheduling strategy to continuously evolve with changes in actual operating conditions. Ultimately, under the premise of ensuring power supply safety, it maximizes the absorption rate of new energy and significantly reduces the overall operating cost of the system.

[0073] Figure 4 A schematic diagram of a system for scheduling control of a wind, solar and energy storage system is shown according to an example embodiment.

[0074] See Figure 4The system for scheduling and control of wind, solar and energy storage systems includes a data acquisition module 01, a status construction module 02, a trajectory generation module 03, an instruction output module 04, and an instruction execution and status acquisition module 05.

[0075] According to some embodiments, the data acquisition module 01 is used to acquire real-time operating status data of the wind-solar-storage system and source-load prediction data within a preset time window. The state construction module 02 is used to fuse the real-time operating status data and the source-load prediction data to construct a comprehensive state vector reflecting the current physical state of the system and its future evolution trend. The trajectory generation module 03 is used to input the comprehensive state vector into a pre-trained hierarchical deep reinforcement learning agent policy model to generate a scheduling reference trajectory that satisfies the goals of long-term economic efficiency and power smoothing. The instruction output module 04 is used to optimize actions based on the scheduling reference trajectory and system safety constraints, and output scheduling instructions that conform to the physical safety boundaries. The instruction execution and state acquisition module 05 is used to control the operation of equipment in the wind-solar-storage system according to the scheduling instructions, and to acquire the system response status after executing the scheduling instructions in real time, using the system response status as the state update input for the next scheduling cycle, forming a data-driven closed-loop adaptive control. Figure 4 The system shown is Figure 1 The methods shown correspond to each other, and the same content will not be repeated.

[0076] According to some embodiments, the system controls the operation of equipment in the wind-solar-storage system according to the scheduling instructions, wherein the equipment includes at least one of wind power generation unit, photovoltaic power generation unit, energy storage system, controllable power generation unit, load-side equipment and grid connection interface.

[0077] The system implementation method for scheduling control of wind, solar and energy storage systems proposed in this invention can effectively solve the problem of rigidity in traditional scheduling strategies, realize the generation of scheduling strategies that take into account both long-term economic efficiency and real-time security, and achieve adaptive evolution and highly reliable closed-loop control of scheduling strategies.

[0078] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. The present invention is not limited to the above embodiments; the embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

[0079] Figure 5 A block diagram of a computing device according to an exemplary embodiment is shown.

[0080] like Figure 5As shown, the computing device 30 includes a processor 12 and a memory 14. The computing device 30 may also include a bus 22, a network interface card 16, and an I / O interface 18. The processor 12, memory 14, network interface card 16, and I / O interface 18 can communicate with each other via the bus 22.

[0081] The processor 12 may include one or more general-purpose CPUs (Central Processing Units), microprocessors, or application-specific integrated circuits, for executing relevant program instructions. According to some embodiments, the computing device 30 may also include a high-performance display adapter (GPU) 20 for accelerating the processor 12.

[0082] Memory 14 may include a machine-readable medium in the form of volatile memory, such as random access memory (RAM), read-only memory (ROM), and / or cache memory. Memory 14 is used to store one or more programs containing instructions, as well as data. Processor 12 may read the instructions stored in memory 14 to perform the methods described above according to embodiments of the present invention.

[0083] The computing device 30 can also communicate with one or more networks via the network interface card 16. The network interface card 16 is used for data processing or external communication. The network interface card 16 includes a root system-on-a-chip (SoC) and multiple interfaces. The SoC communicates data through these interfaces. The SoC includes a processor and a memory. The memory stores a computer program. When the processor runs the computer program stored in the memory, it implements the method according to an embodiment of the present invention.

[0084] Bus 22 can include address bus, data bus, control bus, etc. Bus 22 provides a path for exchanging information between components.

[0085] It should be noted that, in specific implementations, the computing device 30 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the device described above may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0086] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), network storage devices, cloud storage devices, or any type of medium or device suitable for storing instructions and / or data.

[0087] This invention also provides a computer program product comprising a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments.

[0088] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for scheduling and control of wind-solar-storage systems, characterized in that, The method includes: Acquire real-time operational status data of the wind-solar-storage system and source-load prediction data within a preset time window; The real-time operating status data and the source load prediction data are fused together to construct a comprehensive state vector that reflects the current physical state of the system and its future evolution trend. The integrated state vector is input into a pre-trained hierarchical deep reinforcement learning agent policy model to generate a scheduling reference trajectory that satisfies the goals of long-term economy and power smoothing. Based on the aforementioned scheduling reference trajectory and system safety constraints, action optimization is performed, and scheduling instructions that conform to the physical safety boundaries are output. The system controls the operation of equipment in the wind, solar and energy storage system according to the scheduling instructions, and collects the system response status after the execution of the scheduling instructions in real time. The system response status is used as the status update input for the next scheduling cycle, forming a data-driven closed-loop adaptive control.

2. The method according to claim 1, characterized in that, The method further includes pre-training the hierarchical deep reinforcement learning agent policy model, wherein the hierarchical deep reinforcement learning agent policy model includes an upper-layer scheduling agent and a lower-layer security control agent, and the pre-training process of the hierarchical deep reinforcement learning agent policy model includes: Construct a digital simulation environment for the wind-solar-storage system and initialize the hierarchical deep reinforcement learning agent policy model and its initial reward function; Using the reward function and hyperparameters of the current round, the hierarchical deep reinforcement learning agent policy model is driven to explore multiple rounds in the digital simulation environment, collecting multiple training trajectory data including state, action, reward and final performance index; Key performance indicators of the training trajectory data are extracted, combined with a preset scheduling target text description, organized into structured natural language prompts, and input into a large language model. The gap between the training trajectory performance and the scheduling target is analyzed by large language model analysis, the shortcomings of the current reward function are inferred, and optimization suggestions containing new reward items or modified reward function structure are generated. The optimization suggestions are input into a preset verification model for security and effectiveness verification; Repeat the joint iterative optimization training until the preset convergence condition is met, and output the finally trained and mature hierarchical deep reinforcement learning agent policy model and its corresponding final reward function.

3. The method according to claim 2, characterized in that, The optimization suggestions are input into a preset verification model for security and effectiveness verification, including: If the verification fails, either regenerate the optimization suggestion or maintain the current reward function; If the verification is successful, the verified optimization suggestions are analyzed, a new generation reward function is instantiated in the program, and the large language model generates the hyperparameter adjustment strategy required for the next round of training based on the convergence of the training trajectory.

4. The method according to claim 1, characterized in that, Acquire real-time operational status data of the wind-solar-storage system and source-load prediction data within a preset time window, including: By fusing historical time-series data and external environmental data and inputting them into the prediction model, the time-series evolution characteristics of source-load-storage data are identified. Based on the temporal evolution characteristics and the current state, the predicted source load data within the preset time window is deduced.

5. The method according to claim 4, characterized in that, The real-time operating status data and the source load prediction data are fused together to construct a comprehensive state vector reflecting the current physical state and future evolution trend of the system, including: The real-time operating status data is normalized. Temporal features are extracted from the source load prediction data; The normalized real-time operating status data is concatenated with the source load prediction data after time-series feature extraction to generate the comprehensive state vector containing current physical state features and future evolution trend features.

6. The method according to claim 2, characterized in that, The integrated state vector is input into a pre-trained hierarchical deep reinforcement learning agent policy model to generate a scheduling reference trajectory that satisfies the long-term economic and power smoothing objectives, including: Based on the comprehensive state vector, the upper-level scheduling agent plans the scheduling actions of the wind-solar-storage system on a preset planning time scale with the goals of long-term economic efficiency and power smoothness, and outputs a scheduling reference trajectory that satisfies multi-objective optimization. The scheduling reference trajectory is used as a safety boundary and input benchmark and passed to the lower-level safety control agent.

7. The method according to claim 6, characterized in that, Based on the aforementioned scheduling reference trajectory and system safety constraints, action optimization is performed to output scheduling instructions that conform to the physical safety boundaries, including: The lower-level security control agent parses the scheduling reference trajectory into a target benchmark within a preset time scale, and constructs a local security action space by combining the real-time physical state of the current system. The lower-level security control agent, based on the system security constraints, performs high-frequency, refined action optimization on the scheduling reference trajectory, corrects the deviations in the scheduling plan, and finally generates and outputs the scheduling instructions that conform to the physical security boundaries.

8. The method according to claim 2, characterized in that, The large language model is used to analyze the gap between the training trajectory performance and the scheduling target, infer the shortcomings of the current reward function, and generate optimization suggestions that include new reward terms or modified reward function structures, including: The large language model identifies abnormal training trajectories that cause the performance degradation of the wind-solar-storage system based on the key performance indicators. Based on the state and action sequence corresponding to the abnormal training trajectory, infer the missing penalty term and insufficient incentive term in the current reward function; Generate optimization suggestion text containing new basic reward items, modified weight coefficients of existing reward items, and adjusted reward item calculation logic.

9. A system for scheduling and control of wind-solar-storage systems, characterized in that, The system includes: The data acquisition module is used to acquire real-time operating status data of the wind-solar-storage system and source-load prediction data within a preset time window; The state construction module is used to fuse the real-time operating state data with the source load prediction data to construct a comprehensive state vector that reflects the current physical state of the system and its future evolution trend. The trajectory generation module is used to input the comprehensive state vector into a pre-trained hierarchical deep reinforcement learning agent policy model to generate a scheduling reference trajectory that satisfies the goals of long-term economy and power smoothing. The instruction output module is used to optimize actions based on the scheduling reference trajectory and system safety constraints, and output scheduling instructions that conform to the physical safety boundary. The instruction execution and status acquisition module is used to control the operation of equipment in the wind, solar and energy storage system according to the scheduling instructions, and to collect the system response status after the execution of the scheduling instructions in real time. The system response status is used as the status update input for the next scheduling cycle, forming a data-driven closed-loop adaptive control.

10. A computing device, characterized in that, include: processor; as well as A memory storing a computer program that, when executed by the processor, implements the method as described in any one of claims 1-8.

Images