A Battery Charge-Discharge Scheduling Method Based on Large Model-Guided Deep Reinforcement Learning

By using a large model-guided deep reinforcement learning approach, combined with multimodal information to generate interpretable scheduling priors and legal action masks, the problem of poor adaptability and low safety in existing battery energy storage charge and discharge scheduling is solved, achieving efficient, safe and interpretable charge and discharge optimization of the battery management system.

CN123121747APending Publication Date: 2026-11-17TIANJIN UNIV +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611242323.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-17
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing battery energy storage charging and discharging scheduling methods are unable to comprehensively consider the coupled effects of multiple factors such as wind and solar forecasting errors, sudden load changes, electricity price changes, battery degradation, safety warnings, and power supply protection tasks, resulting in poor system coordination, delayed control response, and difficulty in timely closed-loop handling of safety risks.

Method used

A method based on large model-guided deep reinforcement learning is adopted to integrate multimodal information such as wind and solar forecasts, real-time load, electricity price signals, energy storage SOC, SOH, temperature, voltage, current and weather text to generate interpretable scheduling priors. The charging and discharging strategies are optimized within the scope of legal actions through deep reinforcement learning, and closed-loop optimization is achieved by combining legal action masks and reward feedback.

Benefits of technology

It enhances the global context awareness capability of battery charging and discharging scheduling, ensures the interpretability and security of complex constraints, and achieves comprehensive optimization of green energy consumption, power supply reliability, economic benefits and battery life protection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123121747A_ABST
    Figure CN123121747A_ABST
Patent Text Reader

Abstract

This invention discloses a method for scheduling the charging and discharging of energy storage batteries based on large-model guided deep reinforcement learning. This method acquires multimodal operational data to construct a digital twin scheduling state; generates interpretable scheduling priors based on contextualized prompts through domain-fine-tuned large-model; and estimates the value of learned actions using a D3QN network. By constraining SOC, SOH, temperature, voltage, current, and power boundaries through a legal action mask, safe and executable charging and discharging actions are selected. The natural language scheduling strategy output by the model is transformed into BMS control commands by a rule engine, and closed-loop learning is performed based on execution feedback through a reward function composed of scheduling benefits and battery safety. This method can achieve comprehensive optimization among green energy consumption, power supply reliability, economic benefits, and battery life protection under conditions of fluctuating renewable energy output, load changes, and differences in battery health status.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the fields of new energy power systems, energy storage battery management systems, intelligent dispatch control, artificial intelligence decision-making and reinforcement learning, and in particular relates to a method for energy storage battery charging and discharging scheduling based on large model-guided deep reinforcement learning. Background Technology

[0002] Qinghai Province, one of my country's richest provinces in clean energy resources, possesses photovoltaic, wind, and hydropower resources with significant development potential. With the accelerated construction of a new power system primarily based on new energy sources, Qinghai has formed a green energy system characterized by the complementarity of photovoltaic, wind, and hydropower. However, new energy power generation exhibits significant intermittency, volatility, and uncertainty. Wind and solar power output fluctuates rapidly with weather, seasons, and time of day, posing challenges to the safe and stable operation of the power system and the high-proportion consumption of clean energy. Energy storage systems can transfer electrical energy over time, smooth power fluctuations, and support grid peak shaving; therefore, their construction, monitoring, and dispatch capabilities have become key factors restricting the further development of clean energy.

[0003] In multi-energy complementary energy storage systems, battery energy storage units are crucial for achieving energy buffering, peak shaving and valley filling, green electricity consumption, and emergency power supply. Battery Management System It is necessary to monitor key operating parameters such as State of Charge (SOC), State of Health (SOH), charging / discharging status, temperature, voltage, and current in real time, and to dynamically formulate charging / discharging strategies based on renewable energy output forecasts, real-time load curves, grid electricity price signals, and safety boundaries. In actual operation, the access methods and operating characteristics of energy storage nodes such as lithium batteries, flow batteries, and flywheel energy storage differ. Data protocols, response speeds, capacity scales, and health statuses also vary between different sites, which can easily lead to poor system coordination, delayed control responses, and difficulties in timely closed-loop handling of safety risks.

[0004] Existing battery energy storage charging and discharging scheduling methods mostly rely on fixed rules, empirical thresholds, or traditional optimization models. For example, while rules such as charging during photovoltaic surplus, discharging during evening peak hours, and limiting discharging during low SOC are easy to implement, they struggle to comprehensively consider the coupled effects of multiple factors, including wind and solar forecasting errors, load fluctuations, electricity price changes, battery degradation, safety warnings, and power supply guarantee tasks. Traditional optimization methods can express some constraints and objectives, but they typically rely on precise mathematical models and offline solution processes, making it difficult to balance real-time performance, accuracy, and generalization ability to new scenarios in dynamic scheduling scenarios with time-to-minute or even time-to-second requirements.

[0005] With the development of large-scale models and deep reinforcement learning techniques, introducing large-scale models into intelligent decision-making and scheduling of new energy power systems has significant value. Large-scale models can simultaneously read and understand multimodal heterogeneous information from digital twin systems, including ultra-short-term photovoltaic forecasts, wind power forecasts, real-time load curves, grid price signals, energy storage SOH health status, and weather text reports, forming a unified understanding of the system's current and future status for the next few hours. Simultaneously, they can receive natural language scenarios and constraints through prompts, generating scheduling suggestions such as "charge at a 0.8C rate from 2:00 PM to 3:00 PM to absorb excess photovoltaic power, and discharge at a 0.5C rate from 7:00 PM to 8:00 PM to participate in evening peak shaving."

[0006] However, relying solely on the natural language output of a large model makes it difficult to guarantee that the actions within each control cycle will meet the upper and lower limits of SOC, temperature, voltage, current, and energy storage converter. Hard constraints such as power output, grid-connected power, and battery life exist; relying solely on reinforcement learning can easily lead to blind exploration and unsafe actions in the early stages of training. Therefore, combining the scene understanding, knowledge generalization, and cue word scheduling capabilities of large models with the online adaptive optimization capabilities, legal action masking mechanisms, and two reward feedback mechanisms of reinforcement learning to form an interpretable, executable, and closed-loop intelligent scheduling method for battery management system charging and discharging has become a key technical problem that clean energy storage platforms urgently need to solve. Summary of the Invention

[0007] In view of this, the present invention aims to overcome the problems of poor adaptability of fixed rules in battery energy storage charging and discharging scheduling, insufficient real-time performance of traditional optimization, low efficiency of pure reinforcement learning in early exploration, and the risk of unsafe actions in the prior art. The present invention proposes a battery charging and discharging scheduling method based on large model-guided deep reinforcement learning. The method takes clean energy storage platform as the application scenario, integrates multimodal information such as wind and solar forecasts, real-time load, electricity price signals, energy storage SOC, SOH, temperature, voltage, current and weather text, uses large model to generate interpretable scheduling priors, and optimizes charging and discharging strategies within the legal action range through deep reinforcement learning.

[0008] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0009] In a first aspect, the present invention provides a method for scheduling the charging and discharging of energy storage batteries based on large-model guided deep reinforcement learning, comprising the following steps:

[0010] Step 1: Obtain multimodal operation data of the new energy storage system, including ultra-short-term photovoltaic forecasts, wind power forecasts, hydropower or external power output, real-time load curves, grid electricity price signals, carbon intensity signals, weather text reports, power supply guarantee task information, and SOC, SOH, charge and discharge status, temperature, voltage, current and alarm status of each energy storage battery cluster;

[0011] Step 2: Perform time synchronization, outlier processing, scale normalization, and digital twin state mapping on the multimodal operation data to construct a unified scheduling state that includes new energy supply, load demand, energy storage capacity, health status, and safety boundaries;

[0012] Step 3: Construct structured prompt words based on scheduling status, future prediction window and scenario constraints, and call the domain fine-tuning large model to generate scheduling priors. The scheduling priors include charging priority, discharging priority, power supply priority, green electricity consumption preference, peak-valley arbitrage preference, lifetime protection preference and their weights.

[0013] Step 4: Construct a set of legal actions for the battery management system's charge and discharge scheduling. Actions include charging, discharging, or putting each energy storage unit into standby mode, with charge / discharge rate or power as the action strength. Generate a legal action mask based on SOC upper and lower limits, SOH threshold, temperature threshold, voltage and current threshold, PCS power constraints, and grid scheduling constraints.

[0014] Step 5: Use a dual-structure dual-depth Q-network to estimate the learned action value of each legal charging and discharging action, and then weight and fuse the learned action value with the prior score of the large model scheduling to obtain the final action score. Finally, select the optimal charging, discharging or standby action from the set of legal actions.

[0015] Step 6: Convert the optimal action into a natural language scheduling strategy and structured control instructions, and after verification by the rule engine, send them to the battery management system or energy storage converter for execution;

[0016] Step 7: Construct a reward function consisting of two parts based on the scheduling benefits and battery safety status after execution, write the state transition samples into the experience replay pool and update the deep reinforcement learning network to achieve closed-loop optimization of the charging and discharging strategy.

[0017] Furthermore, the domain fine-tuning model is based on a privately deployable Chinese general-purpose language model. Before deployment, domain adaptive training is performed using power storage operation procedures, interface documents, battery safety boundary rules, scheduling logs, weather reports, power supply task sheets, and historical wind and solar power output, load, electricity price, and battery status time series data. Then, manual-reviewed scheduling question-and-answer samples, historical optimized charging and discharging strategy samples, and near-optimal samples generated by the optimization solver are used for instruction fine-tuning.

[0018] Furthermore, the domain fine-tuning large model is used to set the upper and lower limits of state of charge, health status threshold, rated capacity, charge and discharge efficiency, maximum charge and discharge power, temperature / voltage / current threshold, reserve capacity threshold, peak and valley electricity price, and carbon intensity formula parameters as prompt word parameter slots; it performs entity recognition, unit conversion, time alignment, and contextual reasoning on weather changes, power supply levels, photovoltaic overcapacity, load surges, and alarm events in natural language scenarios; and outputs scheduling priors in the form of a simplified JS object spectrum.

[0019] Furthermore, the prior score for the large model scheduling is:

[0020]

[0021] Representing the large model's state Next action Scheduling prior scores; Represents the normalization function; This represents the scheduling preference weight vector output by the large model. Representing state With action The corresponding feature vector; express Transpose of;

[0022] Among them, the feature vectors corresponding to states and actions include photovoltaic power output, wind power output, load demand, electricity price signal, weather text encoding, power supply guarantee task level, battery state of charge, health status, temperature, voltage, current, candidate charge and discharge power, candidate action duration, and the predicted change in state of charge after the action is executed within the future prediction window.

[0023] The components of the scheduling preference weight vector correspond to charging priority, discharging priority, power supply priority, green electricity consumption preference, peak-valley arbitrage preference, and lifetime protection preference.

[0024] Furthermore, the process of determining the scheduling preference weights includes:

[0025] The large language model fine-tuned by the energy storage field outputs the original scores for each preference dimension based on structured prompts;

[0026] The rules are adjusted by combining the security priority coefficient and the scenario importance coefficient provided by the rule engine.

[0027] The weight vector is obtained by normalization.

[0028] Furthermore, the final action score is expressed as:

[0029] ,

[0030] ,

[0031] in, Representing state Next action The final action score; This represents the value of the learned actions output by the deep reinforcement learning network; The fusion weights represent the prior scores of the large model; Represents the prior score for scheduling in a large model; Indicates time The optimal action to choose; This represents taking the action that maximizes the objective function; Indicates time The set of legal actions.

[0032] Furthermore, the reward function consists of two items: a scheduling benefit item and a battery safety item.

[0033] ,

[0034] ,

[0035] ,

[0036] In the above reward function, Represents the immediate reward at time t; Indicates the scheduling revenue item; Indicates battery safety features; and These represent the weighting coefficients for the scheduling benefit item and the battery safety item, respectively. Indicates the revenue from green electricity consumption; This indicates that the load meets the revenue requirements; Indicates the cost of purchasing electricity; Indicates penalties for breach of contract; This indicates the cost of battery life degradation; Indicates penalties for security risks; to This represents the weighting coefficient of each sub-indicator.

[0037] Furthermore, the aforementioned and The power supply task level, battery safety status, and dispatch revenue requirements are determined and normalized so that the sum of the two is 1. to The determination is based on the degree of fluctuation of the corresponding sub-indicator in the historical scheduling sample, its contribution to the total reward, and the importance preset by experts.

[0038] Secondly, the present invention provides an energy storage battery charge and discharge scheduling system based on large model-guided deep reinforcement learning, comprising:

[0039] The data acquisition module is used to acquire multimodal operation data of the new energy storage system, including ultra-short-term photovoltaic forecasts, wind power forecasts, hydropower or external power output, real-time load curves, grid electricity price signals, carbon intensity signals, weather text reports, power supply guarantee task information, and SOC, SOH, charge and discharge status, temperature, voltage, current and alarm status of each energy storage battery cluster.

[0040] The state construction module is used to perform time synchronization, outlier processing, scale normalization, and digital twin state mapping on the multimodal operation data, and to construct a unified scheduling state that includes new energy supply, load demand, energy storage capacity, health status, and safety boundaries.

[0041] The large model prior generation module is used to construct structured prompt words based on scheduling status, future prediction window and scenario constraints, and call the domain fine-tuning large model to generate scheduling priors. The scheduling priors include charging priority, discharging priority, power supply priority, green electricity consumption preference, peak-valley arbitrage preference, lifetime protection preference and their weights.

[0042] The legitimate action mask module is used to construct a set of legitimate actions for the battery management system's charge and discharge scheduling. These actions include charging, discharging, or idling each energy storage unit, with charge / discharge rate or power as the action strength. A legitimate action mask is generated based on SOC upper and lower limits, SOH threshold, temperature threshold, voltage and current threshold, PCS power constraints, and grid scheduling constraints.

[0043] The action value learning module is used to estimate the learned action value of each legal charging and discharging action using a dual-structure dual deep Q-network. The learned action value is then weighted and fused with the prior score of the large model scheduling to obtain the final action score, and the optimal charging, discharging or standby action is selected from the set of legal actions.

[0044] The rules engine module is used to convert the optimal action into a natural language scheduling strategy and structured control instructions, which are then sent to the battery management system or energy storage converter for execution after being verified by the rules engine.

[0045] The feedback learning module is used to construct a reward function consisting of two parts based on the scheduling benefits and battery safety status after execution. It writes the state transition samples into the experience replay pool and updates the deep reinforcement learning network to achieve closed-loop optimization of the charging and discharging strategy.

[0046] Thirdly, the present invention provides a computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the above-described method for scheduling the charging and discharging of an energy storage battery based on large model-guided deep reinforcement learning.

[0047] Compared with existing technologies, the energy storage battery charge and discharge scheduling method based on large model-guided deep reinforcement learning described in this invention has the following advantages:

[0048] (1) Unify the understanding of wind and solar forecasting, load curves, electricity price signals, and batteries through a large model. / Combined with multimodal heterogeneous information such as weather text, it enhances the global contextual awareness capability of battery charging and discharging scheduling;

[0049] (2) Complex constraints such as power supply tasks, rainy weather, photovoltaic surplus, peak shaving during the evening rush hour, and battery life protection are transformed into interpretable scheduling priors through scenario-based prompts, which facilitates manual review and strategy traceability;

[0050] (3) Explicit constraints through legal action masking , Temperature, voltage, current, and power boundaries should be controlled to reduce unsafe charging and discharging actions and ensure the safe operation of the battery management system.

[0051] (4) By using two reward functions consisting of scheduling revenue and battery safety, the model achieves comprehensive optimization among green energy consumption, power supply reliability, economic benefits and battery life protection.

[0052] (5) The natural language scheduling strategy is transformed into executable control instructions through the rule engine, realizing the closed-loop implementation from the decision suggestions of the large model to the charging and discharging control of the battery management system. Attached Figure Description

[0053] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0054] Figure 1 This is a schematic diagram of the overall process of intelligent charging and discharging scheduling of the battery management system in an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of the large model-guided charge / discharge decision structure in an embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram of the two reward and charge / discharge closed-loop optimization processes in an embodiment of the present invention. Detailed Implementation

[0057] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0058] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0059] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0060] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0061] Example 1

[0062] This invention provides a method for scheduling the charging and discharging of energy storage batteries based on large model-guided deep reinforcement learning, comprising the following steps:

[0063] Step 1: Acquire multimodal operation data of the new energy storage system, including ultra-short-term photovoltaic forecasts, wind power forecasts, hydropower or external power output, real-time load curves, grid electricity price signals, carbon intensity signals, weather text reports, power supply guarantee task information, and data from each energy storage battery cluster. , Charge / discharge status, temperature, voltage, current, and alarm status;

[0064] Step 2: Perform time synchronization, outlier processing, scale normalization, and digital twin state mapping on the multimodal operation data to construct a unified scheduling state that includes new energy supply, load demand, energy storage capacity, health status, and safety boundaries.

[0065] The unified scheduling state can be represented as:

[0066] ,

[0067] In the aforementioned unified scheduling state, Indicates time The state vector; Indicates the length of the future prediction window; and They represent from time 1 to 2. to Forecasted output sequences of photovoltaic and wind power; Represents the predicted load sequence; This represents a sequence of electricity price or dispatch price signals; , , , and These represent the energy storage units at time [time]. Its state of charge, health status, temperature, voltage, and current; This represents the encoded scene features of weather text, power supply tasks, and alarm descriptions.

[0068] Step 3: Construct structured prompt words based on scheduling status, future prediction window and scenario constraints, and call the domain fine-tuning large model to generate scheduling priors. The scheduling priors include charging priority, discharging priority, power supply priority, green electricity consumption preference, peak-valley arbitrage preference, lifetime protection preference and their weights.

[0069] The domain-specific fine-tuning model is based on a privately deployable general-purpose Chinese language model, preferably employing... Or have equivalent Chinese comprehension and local deployment capabilities Decoder-based large language model; before deployment, first use the power storage operation procedures, The interface documentation, battery safety boundary rules, scheduling logs, weather reports, power supply task sheets, and historical wind and solar power output, load, electricity price, and battery status time series data are used for domain-adaptive training. Then, manual-reviewed scheduling question and answer samples, historical optimized charging and discharging strategy samples, and near-optimal samples generated by the optimization solver are used for instruction fine-tuning.

[0070] The adaptive improvements to the large-scale model in this domain include: setting formula parameters such as upper and lower limits of state of charge, health state threshold, rated capacity, charge and discharge efficiency, maximum charge and discharge power, temperature / voltage / current threshold, reserve capacity threshold, peak and off-peak electricity price, and carbon intensity as prompt word parameter slots; performing entity recognition, unit conversion, time alignment, and contextual reasoning for weather changes, power supply levels, photovoltaic overcapacity, load surges, and alarm events in natural language scenarios; and outputting structured data. The scheduling prior in the form of a priori includes at least the action category, applicable energy storage unit, time window, source of constraint, confidence level, and preference weight.

[0071] Through the above fine-tuning, the large model not only generates numerical values, but also associates unstructured scenario descriptions with energy storage mechanism formulas, equipment safety boundaries, and historical execution feedback to generate interpretable preference weights and constraint rationales. The preference weights are used to compress the ineffective exploration space of deep reinforcement learning in the early stage of training, increase the proportion of legal actions, and provide different scheduling priors in conflicting target scenarios such as power supply during rainy weather, photovoltaic overcapacity, evening peak arbitrage, and battery life protection, thus distinguishing it from the conventional approach of simply superimposing the output of the large model with the reinforcement learning decision.

[0072] Step 4: Construct a set of legal actions for the battery management system's charge and discharge scheduling. These actions include charging, discharging, or putting each energy storage unit into standby mode, with the charge / discharge rate or power as the action intensity. Upper and lower limits Threshold, temperature threshold, voltage and current threshold Power constraints and grid dispatch constraints are used to generate a mask for valid actions.

[0073] The set of legal actions is:

[0074]

[0075] In the above set of legal actions, Indicates time The set of legal actions; Indicates the first The state of charge of each energy storage unit at the next moment; and These represent the upper and lower limits of the charged state, respectively; Indicates the first Each energy storage unit in The charging and discharging action at any given moment, with positive values ​​indicating charging and negative values ​​indicating discharging; Indicates the first The maximum allowable operating amplitude of each energy storage unit; Indicates the first Each energy storage unit at time Temperature; Indicates the first The upper temperature limit of each energy storage unit; This indicates a standby mode.

[0076] The battery state of charge update process is as follows:

[0077] ,

[0078] In the above state of charge update formula, and They represent the first The state of charge of each energy storage unit at the next and current moments; Indicates charging efficiency; Indicates discharge efficiency; Indicates the first Each energy storage unit at time The charging power; Indicates the first Each energy storage unit at time The discharge power; Indicates the scheduling time step; Indicates the first The rated capacity of each energy storage unit.

[0079] Step 5, utilizing dual structure dual depth network Estimate the learning action value of each legal charging and discharging action, and then weight and fuse the learning action value with the prior score of the large model scheduling to obtain the final action score. Finally, select the optimal charging, discharging or standby action from the set of legal actions.

[0080] The prior score for large model scheduling is:

[0081] ,

[0082] Representing the large model's state Next action Scheduling prior scores; Represents the normalization function; This represents the scheduling preference weight vector output by the large model; Representing state With action The corresponding feature vector; express The transpose of .

[0083] Scheduling preference weights The determination process involves three steps: First, a fine-tuned large language model from the energy storage field outputs the original scores for each preference dimension based on structured prompts; second, it is corrected by combining the safety priority coefficient and scenario importance coefficient given by the rule engine; finally, normalization is used to obtain the weight vector. Therefore, the large language model does not simply provide a few fixed parameters, but rather completes constraint identification, target conflict judgment, and preference weight generation under complex contexts such as photovoltaic overcapacity, peak-hour shaving, continuous rain, important power supply tasks, and changes in battery health status.

[0084] State-action feature vectors include photovoltaic power output, wind power output, load demand, electricity price signals, weather text encoding, power supply guarantee task level, and battery status within the future prediction window. , Temperature, voltage, current, candidate charge / discharge power, candidate action duration, and predictions after action execution. Change. The scheduling preference weight vector has components corresponding to charging priority, discharging priority, power supply priority, green electricity consumption preference, peak-valley arbitrage preference, and lifetime protection preference. Used to map the original heuristic score to... Prior scores with the same scale.

[0085] The final action score is expressed as follows:

[0086] ,

[0087] ,

[0088] In the final action score mentioned above, Representing state Next action The final action score; This represents the value of the learned actions output by the deep reinforcement learning network; The fusion weights represent the prior scores of the large model; Represents the prior score for scheduling in a large model; Indicates time The optimal action to choose; This represents taking the action that maximizes the objective function; Indicates time The set of legal actions. The fusion weight can use a fixed default value or be adjusted appropriately according to the scheduling scenario: when the battery safety risk is high or the power preservation task is strong, the weight should be reduced to prioritize meeting safety constraints; when it is necessary to reduce invalid exploration in the early stage of training or in new scenarios, the weight should be appropriately increased to enhance the guiding role of the scheduling prior of the large model.

[0089] Step 6: Convert the optimal action into a natural language scheduling strategy and structured control instructions, and after verification by the rule engine, send them to the battery management system or energy storage converter for execution.

[0090] Step 7: Construct a reward function consisting of two parts based on the scheduling benefits and battery safety status after execution, write the state transition samples into the experience replay pool and update the deep reinforcement learning network to achieve closed-loop optimization of the charging and discharging strategy.

[0091] The reward function consists of two parts: a scheduling benefit term and a battery safety term.

[0092] ,

[0093] ,

[0094] ,

[0095] In the above reward function, Represents the immediate reward at time t; Indicates the scheduling revenue item; Indicates battery safety features; and These represent the weighting coefficients for the scheduling benefit item and the battery safety item, respectively. Indicates the revenue from green electricity consumption; This indicates that the load meets the revenue requirements; Indicates the cost of purchasing electricity; Indicates penalties for breach of contract; This indicates the cost of battery life degradation; Indicates penalties for security risks; to This represents the weighting coefficient of each sub-indicator.

[0096] The weights are determined as follows: and The power supply task level, battery safety status, and dispatch revenue requirements are determined and normalized so that the sum of the two is 1. to The reward is determined based on a comprehensive assessment of the corresponding sub-indicators' volatility in historical scheduling samples, their contribution to the total reward, and the expert-preset importance. When the power supply task level increases or battery safety risks rise, the reward is increased. and with , , corresponding to When the pressure on green energy absorption, the load meets demand, or the demand for optimizing electricity purchase costs increases, the efficiency of green energy consumption should be improved. and with , , corresponding to .

[0097] In determining and When the system first determines whether the current scheduling cycle prioritizes revenue optimization or safety protection, if battery temperature, state of charge, health status, or power preservation tasks indicate that safety protection takes precedence, then the priority is increased. and reduce If the energy storage unit is within a safe range and there is significant room for optimization in terms of green electricity consumption revenue, load satisfaction, or electricity purchase cost, then the efficiency can be increased. and reduce Adjusted and Normalize again to ensure the stable weight distribution of scheduling benefit and battery safety items in the reward function.

[0098] The present invention will be illustrated below with specific examples.

[0099] See Figure 1 This method is designed for provincial or park-level clean energy storage platforms. It takes battery clusters, battery cabinets or energy storage sites as scheduling objects and wind and solar power output forecasts, load demand, electricity price signals and battery health status as inputs. At each scheduling time, it determines the charging, discharging or standby actions of each energy storage unit.

[0100] Step 1, Data Acquisition and Digital Twin Status Construction. This is achieved through data acquisition devices at the energy storage station. , Real-time data is acquired from the energy management system, meteorological interface, and power grid dispatch interface. This real-time data includes data from energy storage batteries. , The system tracks data including temperature, voltage, current, insulation status, fault alarms, charging / discharging status, site available power, photovoltaic and wind power forecasts, regional load curves, electricity price signals, and weather text reports. Data from different sampling frequencies is synchronized in time and missing data is filled in, then mapped to a digital twin system to form a unified scheduling status within the future forecast window. .

[0101] Step 2: Contextualized prompt word construction and large-scale model prior generation. See also... Figure 2 This will include wind and solar forecasts, load curves, electricity price changes, and battery data for the next few hours. / Distribution, safety alerts, weather texts, and power supply task descriptions are encapsulated into structured prompts. For example, a prompt describing "Continued rain tomorrow, solar power output reduced to 30%, regional..." When there is a critical power supply guarantee task and no additional coal-fired units are to be started, the large model output prioritizes power supply reliability, reserves energy storage capacity in advance, and limits low-power consumption. Scheduling preferences for deep battery discharge.

[0102] In this embodiment, the large model prior generation module includes a data adaptation layer, a domain fine-tuning layer, and a prior output layer. The data adaptation layer will... Time-series data, weather text, power supply guarantee task orders, and electricity price curves are converted into a unified template; the domain fine-tuning layer adopts... Alternatively, a small number of parameters can be updated via adapter fine-tuning, allowing the model to learn the meaning of formula parameters, unit conversion relationships, safety boundaries, and scheduling experience in energy storage scenarios; the prior output layer updates according to preset parameters. Output action preferences, weights, constraint sources, and confidence levels. The rule engine only accepts statements that satisfy these criteria. Output of numerical range verification.

[0103] Fine-tuning samples include three categories: the first category consists of text-to-parameter slot annotation samples such as battery safety regulations, energy storage converter power limits, and grid connection scheduling rules; the second category consists of weather, load, electricity price, and other data from historical scheduling logs. , The first class consists of paired samples corresponding to actual charging and discharging actions; the second class consists of reference policy samples obtained under the same boundary conditions through mixed-integer programming, dynamic programming, or human expert review. The fine-tuning objective is to minimize the cross-entropy loss between the structured prior output and the reference policy label, as well as the mean squared error loss of the preference weights. The fine-tuning objective function is as follows: Use a weighted sum of the two losses: ,in The cross-entropy loss is calculated based on the prior categories and action preferences output for structured scheduling. This represents the mean squared error loss between the model output preference weights and the reference weights. The reference weights are obtained by normalizing manually reviewed preferred charging / discharging strategies, historically safe and high-yield strategies, or reference strategies generated by the optimization solver. The values ​​of 0.7 and 0.3 mentioned above are the default weight ratios, which can be slightly adjusted based on the validation set results, but the sum of the two weights remains at 1.

[0104] Step 3: Verification of legitimate action masks and security constraints. At any given scheduling moment, the system verifies the battery... Upper and lower limits Threshold, temperature threshold, voltage and current threshold Rated power, grid-connected power limit, communication status, and fault level generate a valid action mask. If the temperature of a certain battery cluster is too high, If the power level is too low or a fault alarm is detected, high-rate charging or deep discharging will be prohibited. If a future peak load arrives and the power supply guarantee task level is high, premature discharging will be restricted and safe reserve capacity will be maintained.

[0105] Step 4, The network takes the scheduling state as input and outputs the learned action value of each charging and discharging action. The large model prior module outputs... This reflects the heuristic rationale for actions in the current scenario. The system follows... The fusion score is calculated and combined with a legitimate action mask to select the optimal action. This leverages the understanding of complex scenes by a large model while retaining the adaptive optimization capabilities of reinforcement learning based on long-term reward feedback.

[0106] Step 5: Natural Language Policy Generation and Control Command Conversion. The model output can simultaneously include natural language scheduling policies and structured control parameters. For example, the system could output, "From 2:00 PM to 3:00 PM, prioritize scheduling energy storage units..." and by High-rate charging absorbs excess photovoltaic power; choose between 7:00 PM and 8:00 PM. Higher and temperature-safe energy storage units "Discharge at a higher rate to participate in peak shaving during the evening rush hour and obtain electricity price benefits." The rule engine parses this strategy into the target energy storage unit, start and end time, charging and discharging direction, power or rate, Boundary and security interlock conditions, and issued to or implement.

[0107] Step 6, Two Rewards and Closed-Loop Learning. See also Figure 3 After the action is executed, the system will schedule the benefit items. and battery safety items Calculate instant rewards and will Write to the experience replay pool. Where m t This represents the legal action mask corresponding to the state transition sample at time t, used to mask candidate actions that do not meet safety constraints during experience replay and target action calculation. During training, the system periodically samples historical experience for updates. The parameters enable the scheduler to gradually learn charging and discharging strategies under different weather conditions, loads, electricity prices, and battery health states.

[0108] The weights of the two rewards are recalculated at the start of each scheduling cycle: when the power supply task level is increased or Below the reserve capacity threshold, Increase measures to enhance battery safety; when equipment is in a safe range, the risk of photovoltaic curtailment increases, or the peak-valley price difference widens. Increase to enhance scheduling benefits; to The reward function is updated regularly based on the most recent historical dispatch samples and expert-preset importance, so that it reflects both historical operating characteristics and power dispatch safety experience.

[0109] When photovoltaic output significantly exceeds load demand during midday and electricity prices are low, the dispatcher prioritizes... Lower Battery clusters with higher temperatures and within a safe temperature range are charged to improve the green energy utilization capacity and reduce solar curtailment.

[0110] When peak load increases and electricity prices are high during the evening, the dispatcher prioritizes devices with better health status. Sufficient and temperature-safe energy storage unit discharge; simultaneously, for Battery clusters with lower or higher recent cycle counts should have their discharge rate reduced to minimize lifespan degradation.

[0111] When a weather report indicates that there will be continuous rain and that there is a need for power supply in critical load areas, the large model understands the priority of power supply through prompts, generates scheduling priors to increase reserve capacity and limit unnecessary arbitrage discharges, and the reinforcement learning network selects a more conservative discharge strategy under the guidance of these priors to ensure the reliability of power supply to critical loads.

[0112] Example 2

[0113] A battery charge / discharge scheduling system based on large-model guided deep reinforcement learning includes:

[0114] The data acquisition module is used to acquire multimodal operation data of the new energy storage system, including ultra-short-term photovoltaic forecasts, wind power forecasts, hydropower or external power output, real-time load curves, grid electricity price signals, carbon intensity signals, weather text reports, power supply guarantee task information, and SOC, SOH, charge and discharge status, temperature, voltage, current and alarm status of each energy storage battery cluster.

[0115] The state construction module is used to perform time synchronization, outlier processing, scale normalization, and digital twin state mapping on the multimodal operation data, and to construct a unified scheduling state that includes new energy supply, load demand, energy storage capacity, health status, and safety boundaries.

[0116] The large model prior generation module is used to construct structured prompt words based on scheduling status, future prediction window and scenario constraints, and call the domain fine-tuning large model to generate scheduling priors. The scheduling priors include charging priority, discharging priority, power supply priority, green electricity consumption preference, peak-valley arbitrage preference, lifetime protection preference and their weights.

[0117] The legitimate action mask module is used to construct a set of legitimate actions for the battery management system's charge and discharge scheduling. These actions include charging, discharging, or idling each energy storage unit, with charge / discharge rate or power as the action strength. A legitimate action mask is generated based on SOC upper and lower limits, SOH threshold, temperature threshold, voltage and current threshold, PCS power constraints, and grid scheduling constraints.

[0118] The action value learning module is used to estimate the learned action value of each legal charging and discharging action using a dual-structure dual deep Q-network. The learned action value is then weighted and fused with the prior score of the large model scheduling to obtain the final action score, and the optimal charging, discharging or standby action is selected from the set of legal actions.

[0119] The rules engine module is used to convert the optimal action into a natural language scheduling strategy and structured control instructions, which are then sent to the battery management system or energy storage converter for execution after being verified by the rules engine.

[0120] The feedback learning module is used to construct a reward function consisting of two parts based on the scheduling benefits and battery safety status after execution. It writes the state transition samples into the experience replay pool and updates the deep reinforcement learning network to achieve closed-loop optimization of the charging and discharging strategy.

[0121] Example 3

[0122] A computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described energy storage battery charging and discharging scheduling method based on large model-guided deep reinforcement learning.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for energy storage battery charging and discharging scheduling based on large model guided deep reinforcement learning, characterized in that: Includes the following steps: Step 1: Obtain multimodal operation data of the new energy storage system, including ultra-short-term photovoltaic forecasts, wind power forecasts, hydropower or external power output, real-time load curves, grid electricity price signals, carbon intensity signals, weather text reports, power supply guarantee task information, and SOC, SOH, charge and discharge status, temperature, voltage, current and alarm status of each energy storage battery cluster; Step 2: Perform time synchronization, outlier processing, scale normalization, and digital twin state mapping on the multimodal operation data to construct a unified scheduling state that includes new energy supply, load demand, energy storage capacity, health status, and safety boundaries; Step 3: Construct structured prompt words based on scheduling status, future prediction window and scenario constraints, and call the domain fine-tuning large model to generate scheduling priors. The scheduling priors include charging priority, discharging priority, power supply priority, green electricity consumption preference, peak-valley arbitrage preference, lifetime protection preference and their weights. Step 4: Construct a set of legal actions for the battery management system's charge and discharge scheduling. Actions include charging, discharging, or putting each energy storage unit into standby mode, with charge / discharge rate or power as the action strength. Generate a legal action mask based on SOC upper and lower limits, SOH threshold, temperature threshold, voltage and current threshold, PCS power constraints, and grid scheduling constraints. Step 5: Use a dual-structure dual-depth Q-network to estimate the learned action value of each legal charging and discharging action, and then weight and fuse the learned action value with the prior score of the large model scheduling to obtain the final action score. Finally, select the optimal charging, discharging or standby action from the set of legal actions. Step 6: Convert the optimal action into a natural language scheduling strategy and structured control instructions, and after verification by the rule engine, send them to the battery management system or energy storage converter for execution; Step 7: Construct a reward function consisting of two parts based on the scheduling benefits and battery safety status after execution, write the state transition samples into the experience replay pool and update the deep reinforcement learning network to achieve closed-loop optimization of the charging and discharging strategy.

2. The method of claim 1, wherein the method is based on large model guided deep reinforcement learning. The domain fine-tuning large model is based on a privately deployable Chinese general-purpose large language model. Before deployment, it is first trained using power storage operation procedures, interface documents, battery safety boundary rules, scheduling logs, weather reports, power supply task orders, and historical wind and solar power output, load, electricity price, and battery status time series data. Then, it is fine-tuned using manually reviewed scheduling question and answer samples, historical optimized charging and discharging strategy samples, and near-optimal samples generated by the optimization solver.

3. The method of claim 1, wherein the method is characterized by: The fine-tuning model in this domain is used to set the upper and lower limits of state of charge, health state threshold, rated capacity, charge and discharge efficiency, maximum charge and discharge power, temperature / voltage / current threshold, reserve capacity threshold, peak and valley electricity price, and carbon intensity formula parameters as prompt word parameter slots; Perform entity recognition, unit conversion, time alignment, and contextual reasoning on weather changes, power supply levels, photovoltaic overcapacity, load surges, and alarm events in natural language scenarios; It also outputs the scheduling prior in the form of a simplified JS object.

4. The method of claim 1, wherein the method is characterized by: The prior score for the large model scheduling is: ; denotes a state next action a scheduling prior score; denotes a normalization function; denotes a scheduling preference weight vector output by the large model, denotes a state corresponding to an action a feature vector; denotes a transpose of Among them, the feature vectors corresponding to states and actions include photovoltaic power output, wind power output, load demand, electricity price signal, weather text encoding, power supply guarantee task level, battery state of charge, health status, temperature, voltage, current, candidate charge and discharge power, candidate action duration, and the predicted change in state of charge after the action is executed within the future prediction window. The components of the scheduling preference weight vector correspond to charging priority, discharging priority, power supply priority, green electricity consumption preference, peak-valley arbitrage preference, and lifetime protection preference.

5. The method of claim 4, wherein the method is characterized by: The process of determining the scheduling preference weights includes: The large language model fine-tuned by the energy storage field outputs the original scores for each preference dimension based on structured prompts; The rules are adjusted by combining the security priority coefficient and the scenario importance coefficient provided by the rule engine. The weight vector is obtained by normalization.

6. The method of claim 1, wherein the method is based on a large model guided deep reinforcement learning for energy storage battery charging and discharging scheduling. The final action score is expressed as follows: , , wherein, representing a state next action a final action score; representing a learned action value output by a deep reinforcement learning network; representing a fusion weight of a large model prior score; representing a large model scheduled prior score; representing a time optimal action selected; representing an action that maximizes an objective function; representing a set of legal actions at a time .

7. The method of claim 1, wherein the method is based on a large model guided deep reinforcement learning for energy storage battery charging and discharging scheduling. The reward function consists of two components: a scheduling benefit component and a battery safety component. , , , In the above reward function, Represents the immediate reward at time t; Indicates the scheduling revenue item; Indicates battery safety features; and These represent the weighting coefficients for the scheduling benefit item and the battery safety item, respectively. Indicates the revenue from green electricity consumption; This indicates that the load meets the revenue requirements; Indicates the cost of purchasing electricity; Indicates penalties for breach of contract; This indicates the cost of battery life degradation; Indicates penalties for security risks; to This represents the weighting coefficient of each sub-indicator.

8. The energy storage battery charging and discharging scheduling method based on large model-guided deep reinforcement learning according to claim 7, characterized in that: The and The power supply task level, battery safety status, and dispatch revenue requirements are determined and normalized so that the sum of the two is 1. to The determination is based on the degree of fluctuation of the corresponding sub-indicator in the historical scheduling sample, its contribution to the total reward, and the importance preset by experts.

9. A battery charging and discharging scheduling system based on large model-guided deep reinforcement learning, characterized in that: include: The data acquisition module is used to acquire multimodal operation data of the new energy storage system, including ultra-short-term photovoltaic forecasts, wind power forecasts, hydropower or external power output, real-time load curves, grid electricity price signals, carbon intensity signals, weather text reports, power supply guarantee task information, and SOC, SOH, charge and discharge status, temperature, voltage, current and alarm status of each energy storage battery cluster. The state construction module is used to perform time synchronization, outlier processing, scale normalization, and digital twin state mapping on the multimodal operation data, and to construct a unified scheduling state that includes new energy supply, load demand, energy storage capacity, health status, and safety boundaries. The large model prior generation module is used to construct structured prompt words based on scheduling status, future prediction window and scenario constraints, and call the domain fine-tuning large model to generate scheduling priors. The scheduling priors include charging priority, discharging priority, power supply priority, green electricity consumption preference, peak-valley arbitrage preference, lifetime protection preference and their weights. The legitimate action mask module is used to construct a set of legitimate actions for the battery management system's charge and discharge scheduling. These actions include charging, discharging, or idling each energy storage unit, with charge / discharge rate or power as the action strength. A legitimate action mask is generated based on SOC upper and lower limits, SOH threshold, temperature threshold, voltage and current threshold, PCS power constraints, and grid scheduling constraints. The action value learning module is used to estimate the learned action value of each legal charging and discharging action using a dual-structure dual deep Q-network. The learned action value is then weighted and fused with the prior score of the large model scheduling to obtain the final action score, and the optimal charging, discharging or standby action is selected from the set of legal actions. The rules engine module is used to convert the optimal action into a natural language scheduling strategy and structured control instructions, which are then sent to the battery management system or energy storage converter for execution after being verified by the rules engine. The feedback learning module is used to construct a reward function consisting of two parts based on the scheduling benefits and battery safety status after execution. It writes the state transition samples into the experience replay pool and updates the deep reinforcement learning network to achieve closed-loop optimization of the charging and discharging strategy.

10. A computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by the processor, it implements the method according to any one of claims 1 to 8.