A low-carbon energy supply chain optimization method based on machine learning
By constructing an improved robust Markov decision-making framework and a two-layer robust value update mechanism, the problem of insufficient robustness of scheduling strategies in the low-carbon energy supply chain under forecast bias conditions is solved, realizing the integrated optimization of energy scheduling and carbon asset allocation, and improving the stability and low-carbon nature of the low-carbon energy supply chain.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU DONGNENG INFORMATION TECH CO LTD
- Filing Date
- 2026-04-03
- Publication Date
- 2026-07-10
AI Technical Summary
Existing low-carbon energy supply chain optimization methods lack robustness in dispatch strategies when there are deviations in load forecasting, renewable energy output forecasting, and carbon price forecasting. They also struggle to coordinate changes in operating costs, carbon emission costs, and carbon assets within a unified framework and lack a two-tiered decision-making mechanism that prioritizes carbon emission risks.
An improved robust Markov decision framework is constructed, which introduces a sensitivity-based set of transition probability uncertainties and a two-layer robust value update mechanism based on carbon priority to achieve integrated optimization of energy dispatch and carbon asset allocation. The key operating quantities of future dispatch cycles are predicted through machine learning, and state and action matching and cost assessment are performed under the improved RMDP decision framework to generate robust dispatch strategies.
Under conditions of load fluctuations and carbon price disturbances, it improves the robustness and low-carbon nature of the low-carbon energy supply chain, reduces operating costs, mitigates carbon emission risks, and achieves unified optimization of energy dispatch and carbon asset allocation.
Smart Images
Figure CN122366731A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of geographic information technology, and in particular to a method for optimizing a low-carbon energy supply chain based on machine learning. Background Technology
[0002] With the large-scale integration of renewable energy and the gradual improvement of carbon trading mechanisms, the low-carbon energy supply chain is expanding from the traditional source-grid-load power system to a comprehensive system encompassing energy supply, grid interaction, load response, energy storage devices, and carbon asset management. In existing technologies, to reduce operating costs and carbon emissions, economic dispatch models based on day-ahead planning, multi-objective optimization models, or robust optimization models are commonly used. These models employ linear programming, mixed-integer programming, or scenario programming to jointly optimize generator output, energy storage charging and discharging, and electricity purchase and sale behavior. Some schemes incorporate carbon emission constraints or carbon cost terms into the objective function to achieve a trade-off between economic efficiency and low carbon emissions. Newer research is beginning to introduce stochastic optimization or Markov decision processes to model the uncertainties of load fluctuations, renewable energy output, electricity prices, and carbon prices, and to utilize reinforcement learning methods to directly learn dispatch strategies from historical data.
[0003] However, existing technologies typically assume that state transition probabilities are known or approximated through finite scenarios, failing to explicitly characterize the statistical errors and distributional biases inherent in these probabilities. When systematic deviations occur in load forecasting, renewable energy output forecasting, or carbon price forecasting, dispatch strategies derived from these models lack robustness and are prone to cost overruns or carbon emission exceedances in actual operation. Furthermore, most methods merely reflect low-carbon goals by adding carbon emission penalty weights to the objective function, lacking a two-tiered decision-making mechanism that prioritizes carbon emission risk, and thus cannot effectively screen and narrow strategies under the most unfavorable carbon emission scenarios. In addition, existing solutions often treat energy dispatch and carbon asset allocation separately, lacking detailed characterization of the timing of carbon quota purchases, sales, and usage, making it difficult to simultaneously coordinate operating costs, carbon emission costs, and carbon asset changes within a unified decision-making framework.
[0004] Therefore, how to provide a machine learning-based method for optimizing the low-carbon energy supply chain is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] One objective of this invention is to propose a machine learning-based method for optimizing a low-carbon energy supply chain. This invention constructs an improved robust Markov decision framework, introducing a sensitivity-based uncertainty set of transition probabilities and a two-layer robust value update mechanism prioritizing carbon emissions. This ensures that the scheduling strategy maintains stability and low-carbon characteristics even under conditions of load fluctuations, uncertain renewable energy output, and carbon price disturbances. This method achieves integrated optimization of energy scheduling and carbon asset allocation, effectively reducing operating costs, mitigating carbon emission risks, and improving the robustness and effectiveness of the low-carbon energy supply chain in actual operation.
[0006] A low-carbon energy supply chain optimization method based on machine learning according to an embodiment of the present invention includes the following steps:
[0007] Collect multi-source operational data from the low-carbon energy supply chain, and perform cleaning, missing value imputation, and normalization to form a standardized operational dataset;
[0008] Based on the standardized operational dataset, the key operational volume prediction results for future scheduling cycles are calculated in chronological order and merged with the currently monitored operational status information to form a system status description for the current scheduling cycle.
[0009] Based on the system state description, the state set and action set in the improved RMDP are defined, the controllable quantities of each link of source, grid, load, carbon storage are mapped to action options, the correspondence between state and action is established, and a complete decision framework for the improved RMDP is obtained.
[0010] The historical state transition characteristics of each state and action combination are statistically analyzed, and a corresponding uncertainty range of transition probability is set for each combination, forming the uncertainty set of transition probability of the improved RMDP, which can cover the distribution deviation in actual operation;
[0011] Under the improved RMDP decision framework, the stage cost calculation rules are determined based on the combination of stage operating costs and carbon emission costs. The cumulative cost of each state under the most unfavorable situation in the uncertain transition probability set is evaluated according to the two-layer robust value update rule. At the same time, the corresponding action is selected to form a robust scheduling strategy.
[0012] The robust scheduling strategy is adjusted until the scheduling strategy and value assessment results are stable, resulting in a convergent robust low-carbon energy supply chain scheduling strategy.
[0013] During actual operation, the robust scheduling strategy is invoked according to the latest system status description based on the scheduling cycle to generate energy scheduling schemes and carbon asset allocation schemes.
[0014] Optionally, the formation of the standardized runtime dataset specifically includes:
[0015] Based on the integrated source-grid-load-storage-carbon structure of the low-carbon energy supply chain, the collection boundaries of multi-source operation data are determined, the collection cycle, timestamp format and data accuracy requirements are set, a multi-source operation data collection channel is established, and raw multi-source operation data is obtained from the field monitoring system and business system and recorded to form a raw operation data sequence. The integrated source-grid-load-storage-carbon structure refers to a supply chain structure that schedules the energy production side, grid side, load side, energy storage unit and carbon emission and carbon asset management links as a whole.
[0016] The original running data sequence is time-aligned, and running data from different sources are resampled and aggregated according to a preset time granularity to generate a running data sequence with a unified time axis.
[0017] Anomaly detection and marking are performed on the time-axis unified running data sequence. The marked data is removed, and the missing data positions are filled by interpolation. The filled running data is then numerically mapped according to the preset normalization rules to form a standardized running dataset.
[0018] Optionally, the formation of the system state description specifically includes:
[0019] Sort the standardized operational dataset by time to determine the time point corresponding to the current scheduling cycle. Extract time slices continuously forward according to the preset historical window length. Arrange the standardized operational data in each time slice in chronological order to form a historical operational data sequence for prediction.
[0020] The historical operation data sequence is input into a pre-trained machine learning prediction model. According to the set prediction step size, the prediction results of key operation quantities at each prediction moment in the current scheduling cycle are calculated one by one and arranged in chronological order to form a sequence of key operation quantity prediction results covering the entire scheduling cycle. The machine learning prediction model is an LSTM model.
[0021] Obtain real-time operating condition data at the corresponding time point of the current scheduling cycle, align the variable types and order with the key operating quantity prediction result sequence, and concatenate the real-time operating condition data with the key operating quantity prediction result sequence to form a system state vector sequence containing current real-time data and future prediction data, which serves as the system state description for the current scheduling cycle.
[0022] Optionally, the complete improved RMDP decision framework is obtained by specifically including:
[0023] Read the system state description of the current scheduling period, arrange the various state variables used to characterize the operation of the low-carbon energy supply chain in the system state description according to the preset variable order, segment the continuous state variables according to the preset precision, label the discrete state variables according to the value category, confirm the combination of the obtained state variables as a state unit, and establish a state set composed of different state units by traversing the system state descriptions that appear in the historical scheduling period, and use the state set as the state set of the improved robust Markov decision process RMDP.
[0024] Based on the established state set, various controllable quantities in the low-carbon energy supply chain, including the source side, active and reactive power regulation side, grid interaction side, load side, energy storage side, and carbon asset management side, are sorted out. Controllable quantities related to power generation regulation, load response execution, energy storage charging and discharging control, and carbon quota buying and selling are extracted as control variables. Continuous control variables are discretized according to a preset step size, and discrete control variables are enumerated according to selectable levels to form a set of control action units. The set of control action units is used as the action set of the improved RMDP.
[0025] The state set and action set are matched, and the system operation safety constraints, equipment operation constraints, power balance constraints and carbon quota constraints are used as constraints. Under each state unit, control action units that meet the constraints are selected. Feasible state-action combinations are registered as valid state-action pairs, forming a state-action correspondence table. The state set, action set and state-action correspondence are collectively defined as the complete improved RMDP decision framework.
[0026] Optionally, the construction of the set of uncertain transition probabilities specifically includes:
[0027] Based on the state set and action set of the improved RMDP, historical operation records of two consecutive scheduling cycles are extracted from the standardized operation dataset. The operation status of each pair of adjacent scheduling cycles is organized into a triplet form of current state unit, control action unit, and next scheduling cycle state unit. All triplets are classified according to the combination of state and control action to form a historical state transition sample set indexed by state and action combination.
[0028] Read the historical state transition sample set, count all possible next scheduling cycle state units and their occurrence frequency in the historical samples, calculate the change in operating costs and carbon emission costs, set a sensitivity threshold, mark the next scheduling cycle state units with sensitivity indices greater than the preset sensitivity threshold as high-sensitivity state units, and mark the next scheduling cycle state units with sensitivity indices not greater than the preset sensitivity threshold as low-sensitivity state units, and set a wider transition probability value range for high-sensitivity state units and a narrower transition probability value range for low-sensitivity state units, respectively, to obtain the corresponding transition probability uncertainty range, so that each next scheduling cycle state unit corresponds to a transition probability uncertainty range defined by the minimum possible transition probability value and the maximum possible transition probability value under the current state and action combination;
[0029] For each state and action combination, a transition probability uncertainty set is constructed. The transition probability distributions corresponding to each state unit of the next scheduling cycle that are within their preset transition probability uncertainty range and whose sum of transition probabilities corresponding to all state units of the next scheduling cycle is equal to one are included in the set. This results in a structure-aware transition probability uncertainty set with different widths in different directions that varies with node and path sensitivity. The set is then summarized and stored as the transition probability uncertainty set of the improved RMDP.
[0030] Optionally, the formation of the robust scheduling strategy specifically includes:
[0031] Under the improved RMDP decision framework, the uncertainty set of transition probabilities is read, the stage operation cost calculation rules and stage carbon emission cost calculation rules within the current scheduling cycle are determined, the operation cost and carbon emission cost generated when a certain control action unit is executed in the current state unit and transferred to each possible next scheduling cycle state unit are calculated separately, and the operation cost and carbon emission cost are weighted and synthesized according to the preset weight coefficient to obtain the basic data for stage cost evaluation.
[0032] Based on the basic data of stage cost evaluation, for each state and action combination, an evaluation is performed according to the two-layer robust value update rule within the corresponding uncertain set of transition probabilities. All allowed transition probability distributions are traversed, and the single-step carbon emission cost evaluation value corresponding to the current state and control action under each transition probability distribution is calculated. The transition scenario with the largest single-step carbon emission cost evaluation value is selected, and the single-step carbon emission cost evaluation corresponding to the transition scenario is used as the carbon emission risk evaluation of the current state and action combination. At the same time, under the premise of keeping the carbon emission risk evaluation not lower than the preset carbon risk concern level, the transition scenario that makes the sum of stage operation cost, carbon emission cost and scheduling cycle cost evaluation reach the maximum value is selected within the same uncertain set of transition probabilities. The cost evaluation result obtained under the most unfavorable transition scenario is used as the robust action cost of the current state and action combination.
[0033] For each state unit in the improved RMDP, control action units whose corresponding carbon emission risk assessment exceeds the preset carbon emission risk threshold are removed. Among the remaining control action units, the control action unit with the lowest robust action cost is selected as the target control action in this iteration. The correspondence between each state unit and the target control action is recorded as the scheduling strategy for the current iteration. The robust action cost and target control action of each state unit are iteratively updated until the stopping condition is met. The stopping condition is that the target control actions of all state units no longer change or the change magnitude is lower than the preset convergence criterion in two consecutive iterations, thus obtaining the robust scheduling strategy.
[0034] Optionally, obtaining the convergent, robust, low-carbon energy supply chain scheduling strategy specifically includes:
[0035] Read the carbon emission risk assessment, robust action cost, and current corresponding control action of each state unit, and organize the correspondence between each state unit and the current control action to form the initial data of the scheduling strategy;
[0036] For each state unit, under the constraint of a preset carbon emission risk threshold, the control action with the lowest robust action cost is selected from the feasible control actions of the current state unit. The selected control action replaces the control action corresponding to the state unit in the initial data of the scheduling strategy, thus obtaining a set of target control actions covering all state units.
[0037] The scheduling actions of the target control action set are combined in order of state number to form the final scheduling action set covering all state units, which serves as a convergent robust low-carbon energy supply chain scheduling strategy.
[0038] Optionally, the generation of the energy dispatch scheme and carbon asset allocation scheme specifically includes:
[0039] In each scheduling cycle during the actual operation phase, the latest operating condition data of the current cycle is collected and aligned with the key operating quantity prediction results generated in the previous cycle according to the variable order to construct the system state description of the current scheduling cycle.
[0040] The system state description of the current scheduling cycle is input into the convergent robust low-carbon energy supply chain scheduling strategy. The target control action of the current scheduling cycle is determined according to the correspondence between the state unit and the control action unit in the strategy. Control instructions for energy supply, grid interaction, load regulation, energy storage control and carbon asset management are extracted from the target control action.
[0041] Based on the extracted control instructions, operation instructions are issued to the energy supply unit, grid interaction unit, load equipment and energy storage equipment respectively to form the energy dispatch plan for the current dispatch cycle. Carbon quota purchase, sale and use operations are executed according to carbon asset management instructions to form a carbon asset allocation plan. The energy dispatch plan and carbon asset allocation plan are output as the execution results of this cycle.
[0042] The beneficial effects of this invention are:
[0043] This invention constructs a low-carbon energy supply chain optimization framework based on an improved RMDP, and performs unified modeling at three levels: state set, action set, and transition probability uncertainty set. It incorporates the source side, grid side, load side, energy storage side, and carbon asset management into the same decision-making system, realizing integrated optimization of energy flow and carbon asset flow. Compared with existing technologies that separate energy dispatch and carbon trading, this invention can coordinate operating costs and carbon emission costs under the same strategy.
[0044] This invention introduces a sensitivity index consisting of changes in operating costs and carbon emission costs in the transition probability modeling. Differentiated transition probability uncertainty ranges are set for different next states, and a structure-aware transition probability uncertainty set is constructed. This makes the scheduling strategy more robust to the transition of key high-risk states when prediction errors and model biases exist, thereby reducing the risk of cost runaway and carbon emission exceedance in actual operation.
[0045] This invention employs a carbon-priority two-layer robust value update rule in the value update and strategy generation process, and combines a carbon emission risk threshold to screen and narrow control actions. This ensures that carbon risk is controlled under the most unfavorable carbon emission conditions, while also taking into account operational economy in the remaining actions, thereby improving the low-carbon attributes and executability of the strategy.
[0046] This invention directly outputs energy scheduling schemes and carbon asset allocation schemes by rolling out a convergent robust scheduling strategy according to the scheduling cycle during the actual operation phase. This reduces the drawbacks of manual intervention and reliance on static plans, and significantly improves the operational stability and overall optimization level of the low-carbon energy supply chain in complex and uncertain environments. Attached Figure Description
[0047] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0048] Figure 1 This is a flowchart of a low-carbon energy supply chain optimization method based on machine learning proposed in this invention;
[0049] Figure 2 This is a schematic diagram of a low-carbon energy supply chain optimization method based on machine learning proposed in this invention.
[0050] Figure 3 This is a data flow diagram of a machine learning-based low-carbon energy supply chain optimization method proposed in this invention. Detailed Implementation
[0051] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0052] refer to Figure 1-3 A machine learning-based method for optimizing a low-carbon energy supply chain includes the following steps:
[0053] Collect multi-source operational data from the low-carbon energy supply chain, and perform cleaning, missing value imputation, and normalization to form a standardized operational dataset;
[0054] Based on the standardized operational dataset, the key operational volume prediction results for future scheduling cycles are calculated in chronological order and merged with the currently monitored operational status information to form a system status description for the current scheduling cycle.
[0055] Based on the system state description, the state set and action set in the improved RMDP are defined, the controllable quantities of each link of source, grid, load, carbon storage are mapped to action options, the correspondence between state and action is established, and a complete decision framework for the improved RMDP is obtained.
[0056] The historical state transition characteristics of each state and action combination are statistically analyzed, and a corresponding uncertainty range of transition probability is set for each combination, forming the uncertainty set of transition probability of the improved RMDP, which can cover the distribution deviation in actual operation;
[0057] Under the improved RMDP decision framework, the stage cost calculation rules are determined based on the combination of stage operating costs and carbon emission costs. The cumulative cost of each state under the most unfavorable situation in the uncertain transition probability set is evaluated according to the two-layer robust value update rule. At the same time, the corresponding action is selected to form a robust scheduling strategy.
[0058] The robust scheduling strategy is adjusted until the scheduling strategy and value assessment results are stable, resulting in a convergent robust low-carbon energy supply chain scheduling strategy.
[0059] During actual operation, the robust scheduling strategy is invoked according to the latest system status description based on the scheduling cycle to generate energy scheduling schemes and carbon asset allocation schemes.
[0060] In this embodiment, the formation of the standardized operational dataset specifically includes:
[0061] Based on the integrated source-grid-load-storage-carbon structure of the low-carbon energy supply chain, the collection boundaries of multi-source operation data are determined, the collection cycle, timestamp format and data accuracy requirements are set, a multi-source operation data collection channel is established, and raw multi-source operation data is obtained from the field monitoring system and business system and recorded to form a raw operation data sequence. The integrated source-grid-load-storage-carbon structure refers to a supply chain structure that schedules the energy production side, grid side, load side, energy storage unit and carbon emission and carbon asset management links as a whole.
[0062] The original running data sequence is time-aligned, and running data from different sources are resampled and aggregated according to a preset time granularity to generate a running data sequence with a unified time axis.
[0063] Anomaly detection and marking are performed on the time-axis unified running data sequence. The marked data is removed, and the missing data positions are filled by interpolation. The filled running data is then numerically mapped according to the preset normalization rules to form a standardized running dataset.
[0064] In this embodiment, the formation of the system state description specifically includes:
[0065] Sort the standardized operational dataset by time to determine the time point corresponding to the current scheduling cycle. Extract time slices continuously forward according to the preset historical window length. Arrange the standardized operational data in each time slice in chronological order to form a historical operational data sequence for prediction.
[0066] The historical operation data sequence is input into a pre-trained machine learning prediction model. According to the set prediction step size, the prediction results of key operation quantities at each prediction moment in the current scheduling cycle are calculated one by one and arranged in chronological order to form a sequence of key operation quantity prediction results covering the entire scheduling cycle. The machine learning prediction model is an LSTM model.
[0067] Obtain real-time operating condition data at the corresponding time point of the current scheduling cycle, align the variable types and order with the key operating quantity prediction result sequence, and concatenate the real-time operating condition data with the key operating quantity prediction result sequence to form a system state vector sequence containing current real-time data and future prediction data, which serves as the system state description for the current scheduling cycle.
[0068] In this embodiment, the complete improved RMDP decision framework is obtained specifically through:
[0069] Read the system state description of the current scheduling period, arrange the various state variables used to characterize the operation of the low-carbon energy supply chain in the system state description according to the preset variable order, segment the continuous state variables according to the preset precision, label the discrete state variables according to the value category, confirm the combination of the obtained state variables as a state unit, and establish a state set composed of different state units by traversing the system state descriptions that appear in the historical scheduling period, and use the state set as the state set of the improved robust Markov decision process RMDP.
[0070] Based on the established state set, various controllable quantities in the low-carbon energy supply chain, including the source side, active and reactive power regulation side, grid interaction side, load side, energy storage side, and carbon asset management side, are sorted out. Controllable quantities related to power generation regulation, load response execution, energy storage charging and discharging control, and carbon quota buying and selling are extracted as control variables. Continuous control variables are discretized according to a preset step size, and discrete control variables are enumerated according to selectable levels to form a set of control action units. The set of control action units is used as the action set of the improved RMDP.
[0071] The state set and action set are matched, and the system operation safety constraints, equipment operation constraints, power balance constraints and carbon quota constraints are used as constraints. Under each state unit, control action units that meet the constraints are selected. Feasible state-action combinations are registered as valid state-action pairs, forming a state-action correspondence table. The state set, action set and state-action correspondence are collectively defined as the complete improved RMDP decision framework.
[0072] This invention constructs an improved RMDP decision framework composed of a set of states, a set of actions, and effective state-action correspondences. This framework enables a unified expression of the operating state and controllable variables of the low-carbon energy supply chain. It allows for the structured characterization of multi-dimensional operating elements from the source side, grid interaction side, load side, energy storage side, and carbon asset management side within the same model. Under the premise of satisfying operational safety, equipment constraints, power balance, and carbon quota constraints, it forms a state-action space that can be directly used for robust scheduling solutions. This provides a stable, complete, and executable decision foundation for subsequent robust value assessment and strategy generation.
[0073] In this embodiment, the construction of the transition probability uncertainty set specifically includes:
[0074] Based on the state set and action set of the improved RMDP, historical operation records of two consecutive scheduling cycles are extracted from the standardized operation dataset. The operation status of each pair of adjacent scheduling cycles is organized into a triplet form of current state unit, control action unit, and next scheduling cycle state unit. All triplets are classified according to the combination of state and control action to form a historical state transition sample set indexed by state and action combination.
[0075] Read the historical state transition sample set, count all possible next scheduling cycle state units and their occurrence frequency in the historical samples, calculate the change in operating costs and carbon emission costs, set a sensitivity threshold, mark the next scheduling cycle state units with sensitivity indices greater than the preset sensitivity threshold as high-sensitivity state units, and mark the next scheduling cycle state units with sensitivity indices not greater than the preset sensitivity threshold as low-sensitivity state units, and set a wider transition probability value range for high-sensitivity state units and a narrower transition probability value range for low-sensitivity state units, respectively, to obtain the corresponding transition probability uncertainty range, so that each next scheduling cycle state unit corresponds to a transition probability uncertainty range defined by the minimum possible transition probability value and the maximum possible transition probability value under the current state and action combination;
[0076] For each state and action combination, a transition probability uncertainty set is constructed. The transition probability distributions corresponding to each state unit of the next scheduling cycle that are within their preset transition probability uncertainty range and whose sum of transition probabilities corresponding to all state units of the next scheduling cycle is equal to one are included in the set. This results in a structure-aware transition probability uncertainty set with different widths in different directions that varies with node and path sensitivity. The set is then summarized and stored as the transition probability uncertainty set of the improved RMDP.
[0077] This invention constructs a structure-aware transition probability uncertainty set based on historical state transition samples, exhibiting a differentiated range width depending on the sensitivity to operating costs and carbon emissions. This allows for a refined distinction in the uncertainty characterization of different next states based on their impact on costs and carbon emissions. Thus, while ensuring the consistency of the total probability, it increases the emphasis on key high-risk transition directions, effectively enhancing the robustness of robust decision-making under prediction bias and operational disturbances. This provides a more realistic probabilistic uncertainty description for subsequent two-layer robust value assessment.
[0078] In this embodiment, the formation of the robust scheduling strategy specifically includes:
[0079] Under the improved RMDP decision framework, the uncertainty set of transition probabilities is read, the stage operation cost calculation rules and stage carbon emission cost calculation rules within the current scheduling cycle are determined, the operation cost and carbon emission cost generated when a certain control action unit is executed in the current state unit and transferred to each possible next scheduling cycle state unit are calculated separately, and the operation cost and carbon emission cost are weighted and synthesized according to the preset weight coefficient to obtain the basic data for stage cost evaluation.
[0080] Based on the basic data of stage cost evaluation, for each state and action combination, an evaluation is performed according to the two-layer robust value update rule within the corresponding uncertain set of transition probabilities. All allowed transition probability distributions are traversed, and the single-step carbon emission cost evaluation value corresponding to the current state and control action under each transition probability distribution is calculated. The transition scenario with the largest single-step carbon emission cost evaluation value is selected, and the single-step carbon emission cost evaluation corresponding to the transition scenario is used as the carbon emission risk evaluation of the current state and action combination. At the same time, under the premise of keeping the carbon emission risk evaluation not lower than the preset carbon risk concern level, the transition scenario that makes the sum of stage operation cost, carbon emission cost and scheduling cycle cost evaluation reach the maximum value is selected within the same uncertain set of transition probabilities. The cost evaluation result obtained under the most unfavorable transition scenario is used as the robust action cost of the current state and action combination.
[0081] For each state unit in the improved RMDP, control action units whose corresponding carbon emission risk assessment exceeds the preset carbon emission risk threshold are removed. Among the remaining control action units, the control action unit with the lowest robust action cost is selected as the target control action in this iteration. The correspondence between each state unit and the target control action is recorded as the scheduling strategy for the current iteration. The robust action cost and target control action of each state unit are iteratively updated until the stopping condition is met. The stopping condition is that the target control actions of all state units no longer change or the change magnitude is lower than the preset convergence criterion in two consecutive iterations, thus obtaining the robust scheduling strategy.
[0082] This invention employs a two-layer robust value update mechanism prioritizing carbon emission risk within an improved RMDP framework. By combining this mechanism with carbon emission risk thresholds to screen and narrow control actions, the scheduling strategy can maintain controllable risk even under the most unfavorable carbon emission scenarios. Furthermore, it takes into account operating costs, ensuring the strategy remains stable and low-carbon under conditions of high uncertainty. This results in a robust scheduling strategy that accurately reflects operational deviations, carbon cost disturbances, and changes in system constraints, thereby improving the reliability and executability of low-carbon energy supply chain scheduling.
[0083] In this embodiment, obtaining the convergent robust low-carbon energy supply chain scheduling strategy specifically includes:
[0084] Read the carbon emission risk assessment, robust action cost, and current corresponding control action of each state unit, and organize the correspondence between each state unit and the current control action to form the initial data of the scheduling strategy;
[0085] For each state unit, under the constraint of a preset carbon emission risk threshold, the control action with the lowest robust action cost is selected from the feasible control actions of the current state unit. The selected control action replaces the control action corresponding to the state unit in the initial data of the scheduling strategy, thus obtaining a set of target control actions covering all state units.
[0086] The scheduling actions of the target control action set are combined in order of state number to form the final scheduling action set covering all state units, which serves as a convergent robust low-carbon energy supply chain scheduling strategy.
[0087] In this embodiment, the generation of the energy dispatch scheme and carbon asset allocation scheme specifically includes:
[0088] In each scheduling cycle during the actual operation phase, the latest operating condition data of the current cycle is collected and aligned with the key operating quantity prediction results generated in the previous cycle according to the variable order to construct the system state description of the current scheduling cycle.
[0089] The system state description of the current scheduling cycle is input into the convergent robust low-carbon energy supply chain scheduling strategy. The target control action of the current scheduling cycle is determined according to the correspondence between the state unit and the control action unit in the strategy. Control instructions for energy supply, grid interaction, load regulation, energy storage control and carbon asset management are extracted from the target control action.
[0090] Based on the extracted control instructions, operation instructions are issued to the energy supply unit, grid interaction unit, load equipment and energy storage equipment respectively to form the energy dispatch plan for the current dispatch cycle. Carbon quota purchase, sale and use operations are executed according to carbon asset management instructions to form a carbon asset allocation plan. The energy dispatch plan and carbon asset allocation plan are output as the execution results of this cycle.
[0091] Example 1:
[0092] To verify the feasibility of this invention in practice, it was applied to a low-carbon energy supply chain scenario encompassing photovoltaic power plants, wind farms, gas turbine units, lithium battery energy storage devices, industrial park loads, and a carbon quota trading mechanism. In this scenario, power system operators need to minimize overall operating costs while ensuring a balance between power supply and demand and the safe operation of equipment. Simultaneously, they need to control total carbon emissions under carbon quota constraints and carbon trading market rules to avoid overuse of carbon quotas and runaway carbon costs. Traditional economic dispatch methods are often based on the previous day's predicted load and renewable energy output, employing deterministic or simple scenario-based optimization. In actual operation, when wind and solar output, electricity prices, and carbon prices deviate significantly from predictions, operating costs often rise sharply during certain periods, or carbon emissions exceed quotas. Dispatchers need to frequently intervene manually to adjust power generation plans and carbon trading strategies, resulting in significant operational pressure.
[0093] In this embodiment, firstly, multi-source operational data, including photovoltaic (PV) power output, wind power output, gas turbine output, park load, electricity price, carbon price, grid interaction power, energy storage charging and discharging power, and historical carbon quota buying and selling records, are collected through the field monitoring system and the business system. This data is then cleaned, missing value imputation is performed, and normalization is applied to form a standardized operational dataset. Based on this, a pre-trained LSTM prediction model is used to predict the load curve, PV and wind power output, electricity price, and carbon price sequences for the next rolling scheduling cycle, using standardized operational data from the most recent scheduling cycles. These prediction results are then aligned and stitched together with the actual operating conditions monitored at the current moment to obtain a system state description for the current scheduling cycle. This state description not only reflects the current status of the power sources, grid, load, storage, and carbon asset accounts but also includes predictions of key operational quantities for the future, providing an information basis for subsequent decision-making.
[0094] After obtaining the system state description, it is discretized into state units with a preset precision. Simultaneously, various controllable quantities, such as active power regulation of gas turbine units, power regulation under the upper limit constraints of photovoltaic and wind power output, energy storage charging and discharging power levels, grid power purchase and sale levels, and carbon quota purchase, sale, and usage decisions, are discretized into control action units. This constructs the state and action sets of the improved RMDP. Then, using the historical records of two consecutive scheduling cycles from standardized operating data, the operating results of each scheduling cycle are organized into triplets of "current state unit, control action unit, and next scheduling cycle state unit." The historical transitions under different state and action combinations are statistically analyzed, and the corresponding changes in operating costs and carbon emission costs are calculated. Based on this, a sensitivity index is constructed. The next state with a significant impact on costs and carbon emissions is marked as a high-sensitivity state with a wider transition probability uncertainty interval, while the state with a smaller impact is marked as a low-sensitivity state with a relatively narrower interval. This results in a transition probability uncertainty set with structure-aware characteristics.
[0095] After constructing the improved RMDP and its transition probability uncertainty set, the proposed two-layer robust value update rule is adopted. Within the uncertainty set corresponding to each state and action combination, the single-step carbon emission cost under the most unfavorable transition scenario is first evaluated and used as the carbon emission risk assessment. Then, under the premise that the carbon emission risk is not lower than the preset concern level, the transition scenario that maximizes the sum of stage operating cost, carbon emission cost, and future cumulative cost is searched, and the comprehensive cost corresponding to this scenario is used as the robust action cost. On this basis, for each state, control actions with carbon emission risk exceeding the threshold are eliminated, and only the action with the minimum robust action cost is selected as the target control action for that state. Through multiple rounds of updates until the strategy stabilizes, a converged robust low-carbon energy supply chain scheduling strategy is obtained. This strategy is invoked according to the scheduling cycle during the actual operation phase, and directly provides gas turbine output, energy storage charging and discharging instructions, power purchase and sale, and carbon quota purchase, sale, and usage schemes based on the latest system state description, forming an energy scheduling scheme and a carbon asset allocation scheme.
[0096] To verify the advantages of this invention over traditional methods, typical high-fluctuation conditions were selected for multiple consecutive scheduling cycles within the same low-carbon energy supply chain scenario. The traditional deterministic economic scheduling method, the reinforcement learning scheduling method based on ordinary MDP, and the improved RMDP robust scheduling method of this invention were compared. The traditional deterministic economic scheduling method is based on a single or limited number of forecast scenarios, assuming constant parameters, and uses mathematical programming to solve for the optimization of power generation and power purchase / sale plans in a single operation given load and price forecasts. The ordinary MDP reinforcement learning method models the system operation as a Markov decision process, and, given empirical transition probabilities or sampling environments, learns strategies from states to actions through trial and error to optimize long-term returns. Specific experimental data are shown in Table 1.
[0097] Table 1. Comparison of different scheduling methods during the testing period.
[0098] index Traditional deterministic economic scheduling methods Ordinary MDP reinforcement learning method This invention presents an improved robust scheduling method for RMDP. Total operating cost (ten thousand yuan) 102.8 100.9 98.3 Total carbon emissions (tons) 1210 1185 1127 Carbon allowance over-use hours (hours) 14 8 2 Renewable energy curtailment rate (%) 7.9 6.3 3.1 Net income from carbon assets (RMB 10,000) 3.2 4.1 6.0
[0099] As shown in Table 1, compared with traditional deterministic economic dispatch methods, the method of this invention reduces total operating costs by about 4.4%, total carbon emissions by about 6.9%, significantly reduces the number of hours of carbon quota overuse, reduces the renewable energy curtailment rate by about half, and increases the net carbon asset revenue by nearly double. Compared with ordinary MDP reinforcement learning methods, this invention also shows significant improvements in cost and carbon emissions, indicating that by using the structure-aware transition probability uncertainty set and the carbon-priority two-layer robust value update rule, the adverse effects of prediction errors and model biases can be effectively mitigated, making the low-carbon energy supply chain operate more robustly in complex and uncertain environments, and achieving true integrated optimization of energy dispatch and carbon asset allocation.
[0100] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A machine learning-based method for optimizing a low-carbon energy supply chain, characterized in that, Includes the following steps: Collect multi-source operational data from the low-carbon energy supply chain, and perform cleaning, missing value imputation, and normalization to form a standardized operational dataset; Based on the standardized operational dataset, the key operational volume prediction results for future scheduling cycles are calculated in chronological order to form a system state description for the current scheduling cycle. Based on the system state description, the state set and action set in the improved RMDP are defined, the controllable quantities of each link of source, grid, load, carbon storage are mapped to action options, the correspondence between state and action is established, and a complete RMDP decision framework is obtained. The historical state transition characteristics of each state and action combination are statistically analyzed, and a corresponding uncertainty range of transition probability is set for each combination, thus forming the uncertainty set of transition probability of the improved RMDP. Under the improved RMDP decision framework, the stage cost calculation rules are determined, and the cumulative cost of each state in the worst case where the transition probability is in the uncertain set is evaluated according to the two-layer robust value update rule. At the same time, the corresponding action is selected to form a robust scheduling strategy. By adjusting the robust scheduling strategy, a convergent robust low-carbon energy supply chain scheduling strategy is obtained. During actual operation, the robust scheduling strategy is invoked according to the scheduling cycle to generate energy scheduling schemes and carbon asset allocation schemes.
2. The low-carbon energy supply chain optimization method based on machine learning according to claim 1, characterized in that, The formation of the standardized operational dataset specifically includes: Based on the integrated structure of source, grid, load, storage and carbon in the low-carbon energy supply chain, the collection boundaries of multi-source operation data are determined, multi-source operation data collection channels are established, and raw multi-source operation data is obtained from the field monitoring system and business system and recorded to form a raw operation data sequence. The original running data sequence is time-aligned, and running data from different sources are resampled and aggregated according to a preset time granularity to generate a running data sequence with a unified time axis. Anomaly detection and marking are performed on the time-axis unified running data sequence. The marked data is removed, and the missing data positions are filled by interpolation. The filled running data is then numerically mapped according to the preset normalization rules to form a standardized running dataset.
3. The low-carbon energy supply chain optimization method based on machine learning according to claim 1, characterized in that, The formation of the system state description specifically includes: Sort the standardized operation dataset by time to determine the time point corresponding to the current scheduling cycle. Extract time slices continuously forward according to the preset historical window length, and arrange the standardized operation data in each time slice in chronological order to form a historical operation data sequence. The historical operation data sequence is input into the pre-trained machine learning prediction model. Based on the set prediction step size, the prediction results of key operation quantities at each prediction moment in the current scheduling cycle are calculated one by one and arranged in chronological order to form a sequence of key operation quantity prediction results covering the entire scheduling cycle. Obtain real-time operating condition data at the corresponding time point of the current scheduling cycle, align the variable types and order with the key operating quantity prediction result sequence, and concatenate the real-time operating condition data with the key operating quantity prediction result sequence to form a system state vector sequence, which serves as the system state description for the current scheduling cycle.
4. The low-carbon energy supply chain optimization method based on machine learning according to claim 1, characterized in that, The complete improved RMDP decision framework was obtained specifically by including: Read the system state description of the current scheduling period, arrange various state variables according to the preset variable order, segment continuous state variables according to the preset precision, label discrete state variables according to the value category, confirm the combination of the obtained state variables as a state unit, and establish a state set composed of different state units by traversing the system state descriptions that appear in the historical scheduling period. The state set is used as the state set of the improved robust Markov decision process RMDP. Based on the determined set of states, various controllable quantities in the low-carbon energy supply chain are sorted out, continuous control variables are discretized according to a preset step size, and discrete control variables are enumerated according to selectable levels to form a set of control action units. The set of control action units is used as the action set of the improved RMDP. The state set and action set are matched, and control action units that meet the constraints are selected under each state unit. Feasible state-action combinations are registered as valid state-action pairs, forming a state-action correspondence table. The state set, action set, and state-action correspondence are collectively defined as the complete improved RMDP decision framework.
5. The low-carbon energy supply chain optimization method based on machine learning according to claim 1, characterized in that, The construction of the uncertain transition probability set specifically includes: Based on the state set and action set of the improved RMDP, historical operation records of two consecutive scheduling cycles are extracted from the standardized operation dataset. The operation status of each pair of adjacent scheduling cycles is organized into a triplet form of current state unit, control action unit, and next scheduling cycle state unit. All triplets are classified according to the combination of state-control action to form a historical state transition sample set. Read the historical state transition sample set, count all possible next scheduling cycle state units and their occurrence frequency in the historical samples, calculate the change in operating costs and the change in carbon emission costs, set a sensitivity threshold, mark the next scheduling cycle state units with sensitivity indices greater than the preset sensitivity threshold as high-sensitivity state units, and mark the next scheduling cycle state units with sensitivity indices not greater than the preset sensitivity threshold as low-sensitivity state units, and obtain the corresponding uncertainty range of transition probability respectively; For each state and action combination, a transition probability uncertainty set is constructed. The transition probability distributions corresponding to each state unit of the next scheduling cycle that are within its preset transition probability uncertainty range and whose sum of transition probabilities of all state units of the next scheduling cycle is equal to one are included in the set. The structure-aware transition probability uncertainty set is obtained and summarized and stored as the transition probability uncertainty set of the improved RMDP.
6. The method for optimizing a low-carbon energy supply chain based on machine learning according to claim 1, characterized in that, The formation of the robust scheduling strategy specifically includes: Under the improved RMDP decision framework, the uncertainty set of transition probabilities is read, the calculation rules for stage operating costs and stage carbon emission costs within the current scheduling cycle are determined, the operating costs and carbon emission costs are calculated separately, and the operating costs and carbon emission costs are weighted and synthesized according to the preset weight coefficients to obtain the basic data for stage cost evaluation. Based on the basic data of stage cost evaluation, for each state and action combination, an evaluation is performed according to the two-layer robust value update rule within the corresponding uncertain set of transition probabilities. All allowed transition probability distributions are traversed, and the single-step carbon emission cost evaluation value corresponding to the current state and control action under each transition probability distribution is calculated. The transition scenario with the largest single-step carbon emission cost evaluation value is selected, and the single-step carbon emission cost evaluation corresponding to the transition scenario is used as the carbon emission risk evaluation of the current state and action combination. At the same time, under the premise of keeping the carbon emission risk evaluation not lower than the preset carbon risk concern level, the transition scenario that makes the sum of stage operation cost, carbon emission cost and scheduling cycle cost evaluation reach the maximum value is selected within the same uncertain set of transition probabilities. The cost evaluation result obtained under the most unfavorable transition scenario is used as the robust action cost of the current state and action combination. For each state unit in the improved RMDP, control action units whose corresponding carbon emission risk assessment exceeds the preset carbon emission risk threshold are removed. Among the remaining control action units, the control action unit with the lowest robust action cost is selected as the target control action in this iteration. The correspondence between each state unit and the target control action is recorded as the scheduling strategy for the current iteration. The robust action cost and target control action of each state unit are iteratively updated until the stopping condition is met, thus obtaining the robust scheduling strategy.
7. The low-carbon energy supply chain optimization method based on machine learning according to claim 1, characterized in that, The convergence of the robust low-carbon energy supply chain scheduling strategy specifically includes: Read the carbon emission risk assessment, robust action cost, and current corresponding control action of each state unit, and organize the correspondence between each state unit and the current control action to form the initial data of the scheduling strategy; For each state unit, under the constraint of a preset carbon emission risk threshold, the control action with the lowest robust action cost is selected from the feasible control actions of the current state unit. The selected control action replaces the control action corresponding to the state unit in the initial data of the scheduling strategy, thus obtaining a set of target control actions covering all state units. The scheduling actions of the target control action set are combined in order of state number to form the final scheduling action set covering all state units, which serves as a convergent robust low-carbon energy supply chain scheduling strategy.
8. The low-carbon energy supply chain optimization method based on machine learning according to claim 1, characterized in that, The generation of the energy dispatch scheme and carbon asset allocation scheme specifically includes: In each scheduling cycle during the actual operation phase, the latest operating condition data of the current cycle is collected and aligned with the key operating quantity prediction results generated in the previous cycle according to the variable order to construct the system state description of the current scheduling cycle. The system state description of the current scheduling cycle is input into the convergent robust low-carbon energy supply chain scheduling strategy. The target control action of the current scheduling cycle is determined according to the correspondence between the state unit and the control action unit in the strategy, and the control command is extracted from the target control action. Based on the extracted control instructions, operation instructions are issued to form the energy dispatch plan for the current dispatch cycle, and carbon asset allocation plan is formed by executing carbon quota purchase, sale and use operations according to carbon asset management instructions.