Rule language guided multi-target scheduling method and device for integrated energy system

Abstract

The invention provides an integrated energy system multi-target scheduling method and device guided by a rule language. The method comprises the following steps: establishing a mathematical model for multi-target scheduling of an integrated energy system IES; converting a mathematical model of IES multi-target scheduling optimization into a Markov decision process MDP process by combining a large language model LLM, and constructing an LLM agent for multi-target guidance interaction; and solving an MDP process of IES multi-target scheduling optimization by using a multi-target depth deterministic policy gradient algorithm (DDPG) through the LLM intelligent agent to obtain an IES multi-target scheduling decision scheme. According to the method, the LLM and the deep reinforcement learning model are combined, so that multi-target preference guidance of human beings in the IES scheduling optimization process can be well realized, and theoretical and technical support is provided for scheduling departments and personnel to arrange an actual power generation plan of a power grid.

Description

Technical Field

[0001] The present invention relates to the technical field of energy system scheduling, and in particular to a multi-objective scheduling method and device for an integrated energy system guided by rule language. Background Art

[0002] IES (Integrated Energy System) scheduling is usually defined as a complex MO (Multi-Objective) optimization problem. The objectives are tightly coupled through decision variables, and the optimization of one objective often requires the compromise of other objectives. In addition, the unit difference between the objectives makes it difficult to objectively compare the pros and cons of the solutions to multi-objective problems. In existing research, IES scheduling is usually optimized by simplifying multiple objectives such as minimizing cost and minimizing environmental impact into a single objective, or using heuristic algorithms to solve the Pareto frontier.

[0003] IES needs to coordinate the coupled operation between different energy forms, have the ability to manage uncertainty on both the source and load sides, and take into account the flexibility, efficiency and economy of system operation. Therefore, IES energy scheduling management has become a key technology to achieve carbon emission reduction goals. In order to balance the contradiction between carbon emission reduction and economic operation, multi-objective optimization has become a key element in formulating IES scheduling plans.

[0004] The scheduling optimization problem of the integrated energy system is a multi-objective and multi-constrained optimization problem, which can be mathematically described as follows:

[0005]

[0006] In the formula, f n (X) represents the nth objective. Q(X) is the inequality constraint that needs to be satisfied, and P(X) is the equality constraint that needs to be satisfied.

[0007] Generally speaking, the objective function of the comprehensive energy optimization problem can be summarized into three types of objective functions, namely:

[0008] i) Economic target. The economic target is generally the system operating cost. The operating cost covers the equipment operating cost and external energy purchase cost throughout the life cycle.

[0009] ii) Reliability target. For IES containing new energy equipment, the ideal operating state is that the system can achieve self-sufficiency without wind / solar abandonment. However, due to the uncertainty in the system, the actual operating conditions often deviate from the dispatch plan. Therefore, the degree of load imbalance in the system is used to reflect the degree of deviation between the dispatch result and the ideal operating state. The lower the degree of load imbalance in the system, the better the performance of the dispatch scheme.

[0010] iii) Environmental protection objectives. Environmental protection objectives are relatively easy to calculate and can be reflected in many similar model objective functions. For IES optimization scheduling, the system carbon emissions are generally considered.

[0011] Common multi-objective optimization solutions include: linear weighted method, Pareto front method and hierarchical processing method. At present, a solution to the above-mentioned IES dispatch optimization problem in the prior art includes: a method and system for intelligent response of power dispatch based on a large language model. The method steps include: constructing a large language model for power dispatch; selecting the ChatGLM-6B language model for the large language model for power dispatch; constructing a knowledge graph for power dispatch; power dispatch personnel input voice or text questions; the large language model for power dispatch calls the knowledge graph for power dispatch to generate specific commands for execution or answers to questions; and feeding back the specific commands for execution or answers to questions to relevant personnel through a display screen and / or audio. ChatGLM-6B is used as a large language model, and the knowledge graph for power dispatch is added to the ChatGLM-6B large language model to realize intelligent questions, answers and timely responses in the field of power dispatch, which is conducive to improving the work efficiency of power dispatch personnel and filling in the knowledge blind spots of dispatch personnel.

[0012] Another solution to the above IES scheduling optimization problem in the prior art includes: a hybrid energy system multi-objective scheduling optimization method based on deep reinforcement learning. The method steps include: step 1: establishing an electric-thermal integrated energy system model; step 2: establishing an electric integrated energy system model; step 3: establishing an optimization model for an electric-thermal-gas integrated energy system, which has a reasonable structure, using the heating network and the gas supply network as energy storage for the power grid to ensure smooth access of large-scale wind power to the power grid, improve the system's ability to absorb renewable energy, and ensure the stability and economy of the power grid operation.

[0013] The disadvantages of the solutions for solving the above-mentioned IES scheduling optimization problem in the above-mentioned prior art include: Multi-objective conversion to single-objective method: In the process of converting multiple objectives into a single objective, it is usually necessary to determine some weights, coefficients and other parameters to integrate multiple objectives. However, the determination of these parameters is often subjective and uncertain, and it is difficult to find an objective and accurate method to determine them, which may lead to the final single-objective result not accurately reflecting the essential characteristics of the multi-objective. At the same time, there may be complex relationships and interactions between the various objectives in the multi-objective problem, and simply converting it into a single objective will lose a lot of important information. For example, some objectives may be overemphasized or ignored during the conversion process, so that the obtained single-objective solution performs poorly on other objectives that are not fully considered. In practical problems, the importance and priority of multiple objectives may change with time or changes in the environment. However, traditional multi-objective conversion methods are usually based on fixed parameters and weights, which are difficult to flexibly adapt to such dynamic changes, resulting in poor adaptability and robustness of the solution.

[0014] Pareto method: All solutions on the Pareto front are non-inferior solutions, that is, there is no solution that is better than another solution in all objectives. However, this also brings a problem, that is, it is impossible to directly evaluate the solutions on the Pareto front further, and it is difficult to determine which solution is the most suitable for the actual problem. In practical applications, decision makers often need to choose a specific solution on the Pareto front, but there is a lack of clear standards and methods for selection. The above method focuses more on the use of fixed weight control, especially it cannot provide direct adjustment based on human language rule preferences. Summary of the invention

[0015] The embodiments of the present invention provide a method and device for multi-objective scheduling of an integrated energy system guided by rule language, so as to effectively guide IES scheduling by using human multi-objective preferences.

[0016] In order to achieve the above object, the present invention adopts the following technical scheme.

[0017] According to one aspect of the present invention, a multi-objective scheduling method for an integrated energy system guided by rule language is provided, which is characterized by comprising:

[0018] Establish a mathematical model for multi-objective scheduling of the integrated energy system IES;

[0019] Combined with the large language model LLM, the mathematical model of IES multi-objective scheduling optimization is transformed into a Markov decision process MDP process, and an LLM agent for multi-objective guided interaction is constructed;

[0020] The LLM agent uses the multi-objective deep deterministic policy gradient algorithm DDPG to solve the MDP process of IES multi-objective scheduling optimization, and obtains the decision-making plan for IES multi-objective scheduling.

[0021] Preferably, the mathematical model for establishing IES multi-objective scheduling includes:

[0022] Considering the economic, reliability and environmental protection goals, a mathematical model of IES multi-objective scheduling is constructed, and the objective function of the mathematical model is set as shown in formula (1):

[0023]

[0024] In the formula, p HB ,p EB ,p BES is the power of CHP, HB, EB and BES; η BES ,Q BES is BES efficiency and capacity; C buy ,C NG is the unit cost of electricity / natural gas purchased; p buy,t ,M NG,t is the amount of electricity / natural gas purchased; C CHP , C HB , C EB It is the operation and maintenance cost of CHP, HB and EB. grid , Emission NG is the carbon emission per unit of electricity / natural gas. load,e,t , p load,h,t For electrical / heat load demand;

[0025] Considering the IES system balance constraints, various types of unit and line operation constraints, determine the constraints of IES multi-objective scheduling, including:

[0026] IES system balance constraints:

[0027]

[0028] Line constraints:

[0029] |P grid,t |≤P grid,max ,M NG,t ≤M NG ,max (3)

[0030] Operating constraints of various units:

[0031]

[0032] In the formula, p CHP ,pHB ,p EB ,p BES is the power of CHP, HB, EB and BES;

[0033]

[0034] Where η BES ,Q BES For BES efficiency and capacity.

[0035] Preferably, the method of converting the mathematical model of IES multi-objective scheduling optimization into an MDP process by combining LLM includes:

[0036] The mathematical model of IES multi-objective scheduling optimization is transformed into an MDP process, and the state space, action space and reward function of the MDP process are defined, including:

[0037] Define the state space:

[0038]

[0039] Define the action space:

[0040]

[0041] The definition of reward function includes objective function and constraint penalty. The objective function is composed of economic cost, load imbalance degree in the system and carbon emission calculation. Constraint penalty is the penalty for the agent violating the constraint:

[0042] R(n)=R t (n)+R P (n) (9)

[0043] Where: R(n) is the reward function obtained by the nth power generation plan, R t (n) is the objective function reward obtained by the nth power generation plan, R P (n) the constraint penalty obtained for the nth generation plan;

[0044] The objective function part of the MDP process is obtained as follows:

[0045] R t (n)=―λ var ·σ 2 (n) - λ t ·T(n)―λ m ·M(n) (10)

[0046] Where: var is the scaling factor of the economic cost function, λ t is the scaling factor of the load imbalance function within the system, λ mis the scaling factor of the carbon emission function;

[0047] The constraint penalty of the MDP process is:

[0048] R P = -F B (P max +P soc +P load ) (11)

[0049] Where: F B is the scaling factor of the constraint penalty function, P max P is the penalty for each unit and line exceeding the operating constraints. mis is the penalty for violating the energy storage system charging and discharging constraints, P con becomes the penalty for violating the equilibrium constraint.

[0050] Preferably, the construction of an LLM agent for multi-objective guided interaction includes:

[0051] Guide the DRL agent training by talking to the LLM agent, preset the background c and the question q according to the content of the conversation with the LLM agent, the background c includes the rules and regulations for IES multi-objective scheduling, and the question q includes the IES multi-objective scheduling results and the weighted coefficient of the reward function;

[0052] The LLM agent uses the concept of Token x to model the text. The answer of the LLM agent comes from the distribution of a generative model:

[0053]

[0054] Among them, x1,x2,…,x n It is a series of tokens obtained from the vocabulary by applying the probabilistic chain rule;

[0055] Token is sampled by iteration x i+1 , and iteratively convert x i+1 Input the model to sample x i+2 To generate:

[0056]

[0057] Sampling sequence x i+1 ,...,x i+n The generated text is the output of the LLM agent, which is parsed into numerical scores of multiple objective function weights, namely:

[0058] w i (c,q i )=parse W {xi+1 ,…,x i+n |a,s,c,q i} (14)

[0059] a is the action space, formula 8; s is the state space, formula 7; c is the preset context; q is the question; i is the number of dialogue rounds; parse W The function represents the answer generated by the LLM agent under the current action space, state space, background and question;

[0060] Quantify the rewards of the DRL agent and the penalties for violating constraints:

[0061] R target =―∑ i w i (a,s,z,q j )·Q i (s,a) (15)

[0062] Among them, R target Represents the target output containing weighted multiple rewards, quantitative DRL agent rewards.

[0063] Preferably, the LLM agent uses a multi-objective deep deterministic policy gradient algorithm DDPG to solve the MDP process of IES multi-objective scheduling optimization to obtain a decision-making scheme for IES multi-objective scheduling, including:

[0064] The multi-objective DDPG algorithm is used to solve the MDP process of IES multi-objective scheduling optimization. One Actor network, multiple Critic networks, target network, loss function and experience playback are defined to determine the DDPG algorithm.

[0065] Multiple reward functions are introduced into the DDPG algorithm, namely:

[0066] Q(s,a)=[Q1(s,a),Q2(s,a),...,Q m (s,a)] (16)

[0067] The Q-value function of reinforcement learning is expressed as Q(s,a), there are m objective functions and m Q-values;

[0068] The Actor network learns the optimal strategy π(a|s), and the Critic network evaluates the quality of the strategy based on the Q function Q(s,a). Its update rule follows the Bellman equation, and the target network parameter θ target Gradually approach the online network parameter θ in a soft update manner;

[0069] Define experience replay, build a replay storage area, and store the state, action, reward, and next state sampled from the environment each time into the replay buffer. The state space is the set of all states that the agent may be in, and the action space is the set of all actions that the agent can take in each state. The state transition probability describes the probability that the agent will transfer to the next state after taking a certain action in a certain state. The reward function defines the immediate reward obtained by the agent after taking a certain action in a certain state.

[0070] The reinforcement learning reward R is learned through formula 16 and formula 15, and the agent is guided to learn the optimal strategy through language and rules, that is, the optimal action sequence a that should be taken from each state t :

[0071]

[0072] According to another aspect of the present invention, there is provided a multi-objective scheduling device for an integrated energy system guided by a rule language, comprising:

[0073] Mathematical model building module, used to establish the mathematical model of multi-objective scheduling of integrated energy system IES;

[0074] LLM agent building module, which is used to transform the mathematical model of IES multi-objective scheduling optimization into an MDP process by combining the large language model LLM, and to build an LLM agent for multi-objective guided interaction;

[0075] The decision-making scheme acquisition module is used to solve the MDP process of IES multi-objective scheduling optimization through the LLM intelligent agent using the multi-objective deep deterministic policy gradient algorithm DDPG to obtain the decision-making scheme of IES multi-objective scheduling.

[0076] Preferably, the mathematical model building module is specifically used to consider the economic, reliability and environmental protection goals, build a mathematical model of IES multi-objective scheduling, and set the objective function of the mathematical model as shown in formula (1):

[0077]

[0078] In the formula, p HB ,p EB ,p BES is the power of CHP, HB, EB and BES; η BES ,Q BES is BES efficiency and capacity; C buy ,C NG is the unit cost of electricity / natural gas purchased; p buy,t ,M NG,t is the amount of electricity / natural gas purchased; C CHP , CHB , C EB It is the operation and maintenance cost of CHP, HB and EB. grid , Emission NG is the carbon emission per unit of electricity / natural gas. load,e,t , p load,h,t For electrical / heat load demand;

[0079] Considering the IES system balance constraints, various types of unit and line operation constraints, determine the constraints of IES multi-objective scheduling, including:

[0080] IES system balance constraints:

[0081]

[0082] Line constraints:

[0083] |P grid,t |≤P grid,max ,M NG,t ≤M NG ,max (3)

[0084] Operating constraints of various units:

[0085]

[0086] In the formula, p CHP ,p HB ,p EB ,p BES is the power of CHP, HB, EB and BES;

[0087]

[0088] Where η BES ,Q BES For BES efficiency and capacity.

[0089] Preferably, the LLM agent building module is specifically used to convert the mathematical model of IES multi-objective scheduling optimization into an MDP process, define the state space, action space and reward function of the MDP process, including:

[0090] Define the state space:

[0091]

[0092] Define the action space:

[0093]

[0094] The definition of reward function includes objective function and constraint penalty. The objective function is composed of economic cost, load imbalance degree in the system and carbon emission calculation. Constraint penalty is the penalty for the agent violating the constraint:

[0095] R(n)=R t (n)+R P (n) (9)

[0096] Where: R(n) is the reward function obtained by the nth power generation plan, R t (n) is the objective function reward obtained by the nth power generation plan, R P (n) the constraint penalty obtained for the nth generation plan;

[0097] The objective function part of the MDP process is obtained as follows:

[0098] R t (n)=―λ var ·σ 2 (n) - λ t ·T(n)―λ m ·M(n) (10)

[0099] Where: var is the scaling factor of the economic cost function, λ t is the scaling factor of the load imbalance function within the system, λ m is the scaling factor of the carbon emission function;

[0100] The constraint penalty of the MDP process is:

[0101] R P = -F B (P max +P soc +P load ) (11)

[0102] Where: F B is the scaling factor of the constraint penalty function, P max P is the penalty for each unit and line exceeding the operating constraints. mis is the penalty for violating the energy storage system charging and discharging constraints, P con becomes a penalty for violating the equilibrium constraint;

[0103] Guide the DRL agent training by talking to the LLM agent, preset the background c and the question q according to the content of the conversation with the LLM agent, the background c includes the rules and regulations for IES multi-objective scheduling, and the question q includes the IES multi-objective scheduling results and the weighted coefficient of the reward function;

[0104] The LLM agent uses the concept of Token x to model the text. The answer of the LLM agent comes from the distribution of a generative model:

[0105]

[0106] Among them, x1,x2,…,x n It is a series of tokens obtained from the vocabulary by applying the probabilistic chain rule;

[0107] Token is sampled by iteration x i+1 , and iteratively convert x i+1 Input the model to sample x i+2 To generate:

[0108]

[0109] Sampling sequence x i+1 ,...,x i+n The generated text is the output of the LLM agent, which is parsed into numerical scores of multiple objective function weights, namely:

[0110] w i (c,q i )=parse W {x i+1 ,…,x i+n |a,s,c,q i} (14)

[0111] a is the action space, formula 8; s is the state space, formula 7; c is the preset context; q is the question; i is the number of dialogue rounds; parse W The function represents the answer generated by the LLM agent under the current action space, state space, background and question;

[0112] Quantify the rewards of the DRL agent and the penalties for violating constraints:

[0113] R target =―∑ i w i (a,s,z,q j )·Q i (s,a) (15)

[0114] Among them, R target Represents the target output containing weighted multiple rewards, quantitative DRL agent rewards.

[0115] Preferably, the decision solution acquisition module is specifically used to solve the MDP process of IES multi-objective scheduling optimization using a multi-objective DDPG algorithm, and define an Actor network, multiple Critic networks, a target network, a loss function and an experience playback part to determine the DDPG algorithm;

[0116] Multiple reward functions are introduced into the DDPG algorithm, namely:

[0117] Q(s,a)=[Q1(s,a),Q2(s,a),...,Q m (s,a)] (16)

[0118] The Q-value function of reinforcement learning is expressed as Q(s,a), there are m objective functions and m Q-values;

[0119] The Actor network learns the optimal strategy π(a|s), and the Critic network evaluates the quality of the strategy based on the Q function Q(s,a). Its update rule follows the Bellman equation, and the target network parameter θ target Gradually approach the online network parameter θ in a soft update manner;

[0120] Define experience replay, build a replay storage area, and store the state, action, reward, and next state sampled from the environment each time into the replay buffer. The state space is the set of all states that the agent may be in, and the action space is the set of all actions that the agent can take in each state. The state transition probability describes the probability that the agent will transfer to the next state after taking a certain action in a certain state. The reward function defines the immediate reward obtained by the agent after taking a certain action in a certain state.

[0121] The reinforcement learning reward R is learned through formula 16 and formula 15, and the agent is guided to learn the optimal strategy through language and rules, that is, the optimal action sequence a that should be taken from each state t :

[0122]

[0123] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the multi-objective scheduling method of the integrated energy system guided by the rule language is implemented.

[0124] It can be seen from the technical solutions provided by the above-mentioned embodiments of the present invention that the present invention, by combining LLM and deep reinforcement learning models, can better realize the multi-objective preference guidance of humans in the IES scheduling optimization process, and provide theoretical and technical support for scheduling departments and personnel to arrange actual power grid power generation plans.

[0125] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0127] Figure 1 A processing flow chart of a method for implementing a rule-based language-guided integrated energy system multi-objective scheduling method using GPT (Generative Pre-trained Transforme) and DRL (Deep Reinforcement Learning) provided in an embodiment of the present invention;

[0128] Figure 2 A schematic diagram of prompt words for IES multi-objective JSON generation problem provided by an embodiment of the present invention;

[0129] Figure 3 A schematic diagram of weight reasoning results of an intelligent agent provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0130] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0131] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0132] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0133] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0134] The embodiment of the present invention focuses on LLM (Large Language Model) technology, proposes a method for simulating human evaluation of multiple objectives, and combines DRL (Deep Reinforcement Learning) to propose a multi-objective scheduling method for an integrated energy system that considers multiple objective factors such as operating costs, carbon emissions, and reliability.

[0135] The present invention aims to simulate the human evaluation process of multiple objectives by using LLM to train and balance different objectives in IES scheduling in real time. By constructing and training DRL agents, multi-objective IES scheduling guided by knowledge rules is solved in real time, and adjustment strategies with interpretable language output for different objectives are provided to humans.

[0136] The flowchart of the method for implementing a multi-objective scheduling method of an integrated energy system guided by rule language using GPT (Generative Pre-trained Transforme) and DRL provided by the present invention is as follows: Figure 1 As shown, the processing steps include the following:

[0137] Step S101, establishing a mathematical model of IES multi-objective scheduling.

[0138] Step S102, combining LLM to perform MDP (Markov Decision Process) modeling on the mathematical model of IES multi-objective scheduling optimization, and converting the mathematical model of IES multi-objective scheduling optimization into an MDP process.

[0139] Step S103, using the multi-objective DDPG (Deep Deterministic Policy Gradient) algorithm to solve the MDP process of IES multi-objective scheduling optimization, and obtain a decision plan for IES multi-objective scheduling.

[0140] Step S104, using a certain electric heating IES system to verify the effectiveness of the IES multi-objective scheduling decision-making scheme obtained by the proposed method.

[0141] Specifically, the above S101 includes the following sub-steps:

[0142] S1011, considering the economic, reliability and environmental protection goals, determine the objective function of IES multi-objective scheduling as shown in formula (1):

[0143]

[0144] In the formula, p HB ,p EB ,p BES is the power of CHP, HB, EB and BES; η BES ,Q BES for BES efficiency and capacity;

[0145] C buy ,C NG is the unit cost of electricity / natural gas purchased; p buy,t ,M NG,t is the amount of electricity / natural gas purchased; C CHP ,

[0146] C HB , C EB The operation and maintenance costs of CHP, HB and EB; Emission grid , Emission NG is the carbon emission per unit of electricity / natural gas; p load,e,t , p load,h,t For electrical / heat load demand;

[0147] S1012: Considering the IES system balance constraints, various types of unit and line operation constraints, determine the constraints of IES multi-objective scheduling, including:

[0148] IES system balance constraints:

[0149]

[0150] Line constraints:

[0151] |P grid,t |≤P grid,max ,M NG,t ≤M NG ,max (3)

[0152] Operating constraints of various units:

[0153]

[0154] In the formula, p CHP ,p HB ,p EB ,p BES For CHP, HB, EB and BES power.

[0155]

[0156] Where η BES ,Q BES For BES efficiency and capacity.

[0157] Specifically, the above S102 includes the following sub-steps:

[0158] S1021, transform the mathematical model of IES multi-objective scheduling optimization into a Markov decision process, and define the state space, action space and reward function. Including:

[0159] Define the state space:

[0160]

[0161] Define the action space:

[0162]

[0163] Define the reward function. The reward function for the agent should be divided into two parts. One is the objective function, which is composed of the calculation of economic cost, load imbalance in the system and carbon emissions. The other part is the constraint penalty, which is mainly the penalty for the agent violating the constraint. Add the reward function part and the constraint penalty part to get the reward function:

[0164] R(n)=R t (n)+RP (n) (9)

[0165] Where: R(n) is the reward function obtained by the nth power generation plan, R t (n) is the objective function reward obtained by the nth power generation plan, R P (n) is the constraint penalty obtained for the nth generation plan.

[0166] On this basis, scaling by a certain proportion can obtain the objective function part as follows:

[0167] R t (n)=―λ var ·σ 2 (n) - λ t ·T(n)―λ m ·M(n) (10)

[0168] Where: var is the scaling factor of the economic cost function, λ t is the scaling factor of the load imbalance function within the system, λ m is the scaling factor of the carbon emission function.

[0169] The constraint penalty is:

[0170] R P = -F B (P max +P soc +P load ) (11)

[0171] Where: F B is the scaling factor of the constraint penalty function, P max P is the penalty for each unit and line exceeding the operating constraints. mis is the penalty for violating the energy storage system charging and discharging constraints, P con becomes the penalty for violating the equilibrium constraint.

[0172] S1022, construct an LLM agent for multi-objective guided interaction.

[0173] After the mathematical model of IES multi-objective scheduling optimization is converted into an MDP process, the LLM agent can be used to derive the MDP process. Through dialogue between humans and LLM agents, the DRL agent training can be guided by "using human multi-objective preferences and using rule language to guide".

[0174] The LLM agent uses the concept of Token x to model text, that is, the preset background c and the question q. The answer of the LLM agent comes from the distribution of a generative model:

[0175]

[0176] where x1,x2,...,x n It is a series of Tokens obtained from the vocabulary by applying the probabilistic chain rule.

[0177] Token is sampled by iteration x i+1 And iteratively change x i+1 Input the model to sample x i+2 To generate:

[0178]

[0179] Sampling sequence x i+1 ,...,x i+n The generated text is the output of the LLM agent and can be formatted in JavaScript Object Notation (JSON).

[0180] Given a preset background c and a question q, the input token is formulated. Both background c and question q are the content of the conversation between humans and LLM agents. Background c mainly refers to some rules and regulations for IES multi-objective scheduling. Question q mainly refers to the current IES multi-objective scheduling results and the weighted coefficients of the reward function.

[0181] The output JSON can be parsed into numerical scores of multiple objective function weights, namely:

[0182] w i (c,q i )=parse W {x i+1 ,…,x i+n |a,s,c,q i} (14)

[0183] a is the action space, formula 8; s is the state space, formula 7; c is the preset context; q is the question; i is the number of dialogue rounds; parse W The function represents the answer generated by the LLM agent under the current action space, state space, background and problem.

[0184] The parsing function converts the answer of LLM into a numerical value with a string explanation. By integrating the parsed explanation into the reward and constraint, the answer of LLM outputs two parts: one part is the weight of each objective under the multi-objective optimization problem. The weights given by LLM in text form are converted into the numerical value of each objective weight using Json language. The weights of each objective are used to calculate the reward and penalty values ​​in the DRL agent later (i.e., Formula 15). The other part is LLM's explanation of the multi-objective problem and weights. That is, why the LLM agent gives this weight.

[0185] The rewards of the DRL agent and the penalties for violating constraints are quantified as follows:

[0186] R target =―∑ i w i (a,s,z,q j )·Q i (s,a) (15)

[0187] Among them, R target Represents the target output containing weighted multiple rewards, quantitative DRL agent rewards.

[0188] Specifically, the above step S103 includes the following sub-steps:

[0189] The multi-objective DDPG algorithm is used to solve the MDP process of IES multi-objective scheduling optimization, and the action space at that moment is obtained:

[0190]

[0191] include:

[0192] Define 1 Actor network, multiple Critic networks, target network, loss function and experience replay to determine the DDPG algorithm.

[0193] Multiple reward functions are introduced into the DDPG algorithm, namely:

[0194] Q(s,a)=[Q1(s,a),Q2(s,a),...,Q m (s,a)] (16)

[0195] The Q-value function of reinforcement learning is expressed as Q(s,a). Since it is a multi-objective problem, there are m objective functions, so there are m Q values.

[0196] The Actor network learns the optimal strategy π(a|s), and the Critic network evaluates the quality of the strategy based on the Q function Q(s,a), and its update rule follows the Bellman equation.

[0197] target network parameters θ target By using the soft update method, the online network parameter θ is gradually approached, which improves the ability of DDPG to find the approximate optimal strategy in the complex IES scheduling environment.

[0198] Define experience replay, build a replay storage area, and store the four-tuple data (state, action, reward, next state) sampled from the environment each time into the replay buffer.

[0199] MDP is a mathematical framework for describing sequential decision problems. It contains four main elements: state space (S), action space (A), state transition probability (P), and reward function (R). The state space is the set of all possible states that the agent may be in; the action space is the set of all actions that the agent can take in each state; the state transition probability describes the probability that the agent will transfer to the next state after taking an action in a certain state; and the reward function defines the immediate reward obtained by the agent after taking an action in a certain state.

[0200] Formula 16 and Formula 15 implement the reinforcement learning reward R. The reward function is a key factor in guiding the behavior of the agent. It provides a goal-oriented signal to the agent, telling the agent which behaviors are "good" and which are "bad". In MDP, the goal of the agent is to maximize the long-term cumulative reward. Through the reinforcement learning reward R learned by Formula 16 and Formula 15, the present invention can guide the agent to learn the optimal strategy through language and rules, that is, the optimal sequence of actions to be taken from each state.

[0201] Specifically, the above S104 includes the following steps.

[0202] The LLM agent is based on ChatGLM-4-9B and is programmed in Python and tested on a computer with an NVIDIA RTX A5000 graphics card and an Intel(R) Xeon(R) W-2295CPU@3.00GHz 3.00GHz processor.

[0203] The Chain of Thought Prompt and Few-shot Prompt methods are used to ask questions q to the LLM agent, that is, a small group of examples is provided to the LLM model and it is required to reason step by step to improve its performance on weight generation problems.

[0204] To verify the effectiveness of the proposed method, an example is given for the IES multi-objective JSON generation problem. The specific prompt words are as follows: Figure 2As shown in Figure 2. It is known that the weights of each target in the previous round are 1 / 3. After LLM analysis of the weights of each target, the current round state, and the preset multi-target requirement background c, the agent obtains Figure 3 The weight inference results are shown, with human-understandable explanations.

[0205] The present invention also provides a multi-objective scheduling device for an integrated energy system guided by rule language, comprising:

[0206] Mathematical model building module, used to establish the mathematical model of multi-objective scheduling of integrated energy system IES;

[0207] LLM agent building module, which is used to transform the mathematical model of IES multi-objective scheduling optimization into an MDP process by combining the large language model LLM, and to build an LLM agent for multi-objective guided interaction;

[0208] The decision-making scheme acquisition module is used to solve the MDP process of IES multi-objective scheduling optimization through the LLM intelligent agent using the multi-objective deep deterministic policy gradient algorithm DDPG to obtain the decision-making scheme of IES multi-objective scheduling.

[0209] The mathematical model building module is specifically used to consider the economic, reliability and environmental protection goals, build a mathematical model for IES multi-objective scheduling, and set the objective function of the mathematical model as shown in formula (1):

[0210]

[0211] In the formula, p HB ,p EB ,p BES is the power of CHP, HB, EB and BES; η BES ,Q BES is BES efficiency and capacity; C buy ,C NG is the unit cost of electricity / natural gas purchased; p buy,t ,M NG,t is the amount of electricity / natural gas purchased; C CHP , C HB , C EB It is the operation and maintenance cost of CHP, HB and EB. grid , Emission NG is the carbon emission per unit of electricity / natural gas. load,e,t , p load,h,t For electrical / heat load demand;

[0212] Considering the IES system balance constraints, various types of unit and line operation constraints, determine the constraints of IES multi-objective scheduling, including:

[0213] IES system balance constraints:

[0214]

[0215] Line constraints:

[0216] |P grid,t |≤P grid,max ,M NG,t ≤M NG,max (3)

[0217] Operating constraints of various units:

[0218]

[0219] In the formula, p CHP ,p HB ,p EB ,p BES is the power of CHP, HB, EB and BES;

[0220]

[0221] Where η BES ,Q BES For BES efficiency and capacity.

[0222] The LLM agent building module is specifically used to transform the mathematical model of IES multi-objective scheduling optimization into an MDP process, define the state space, action space and reward function of the MDP process, including:

[0223] Define the state space:

[0224]

[0225] Define the action space:

[0226]

[0227] The definition of reward function includes objective function and constraint penalty. The objective function is composed of economic cost, load imbalance degree in the system and carbon emission calculation. Constraint penalty is the penalty for the agent violating the constraint:

[0228] R(n)=R t (n)+R P (n) (9)

[0229] Where: R(n) is the reward function obtained by the nth power generation plan, R t (n) is the objective function reward obtained by the nth power generation plan, R P (n) the constraint penalty obtained for the nth generation plan;

[0230] The objective function part of the MDP process is obtained as follows:

[0231] R t (n)=―λ var ·σ 2 (n) - λ t ·T(n)―λ m ·M(n) (10)

[0232] Where: var is the scaling factor of the economic cost function, λ t is the scaling factor of the load imbalance function within the system, λ m is the scaling factor of the carbon emission function;

[0233] The constraint penalty of the MDP process is:

[0234] R P = -F B (P max +P soc +P load ) (11)

[0235] Where: F B is the scaling factor of the constraint penalty function, P max P is the penalty for each unit and line exceeding the operating constraints. mis is the penalty for violating the energy storage system charging and discharging constraints, P con becomes a penalty for violating the equilibrium constraint;

[0236] Guide the DRL agent training by talking to the LLM agent, preset the background c and the question q according to the content of the conversation with the LLM agent, the background c includes the rules and regulations for IES multi-objective scheduling, and the question q includes the IES multi-objective scheduling results and the weighted coefficient of the reward function;

[0237] The LLM agent uses the concept of Token x to model the text. The answer of the LLM agent comes from the distribution of a generative model:

[0238]

[0239] Among them, x1,x2,…,x n It is a series of tokens obtained from the vocabulary by applying the probabilistic chain rule;

[0240] Token is sampled by iteration x i+1 , and iteratively convert x i+1 Input the model to sample x i+2 To generate:

[0241]

[0242] Sampling sequence x i+1 ,…,x i+n The generated text is the output of the LLM agent, which is parsed into numerical scores of multiple objective function weights, namely:

[0243] w i (c,q i )=parse W {x i+1 ,…,x i+n |a,s,c,q i} (14)

[0244] a is the action space, formula 8; s is the state space, formula 7; c is the preset context; q is the question; i is the number of dialogue rounds; parse W The function represents the answer generated by the LLM agent under the current action space, state space, background and question;

[0245] Quantify the rewards of the DRL agent and the penalties for violating constraints:

[0246] R target =―∑ i w i (a,s,z,q j )·Q i (s,a) (15)

[0247] Among them, R target Represents the target output containing weighted multiple rewards, quantitative DRL agent rewards.

[0248] The decision-making solution acquisition module is specifically used to solve the MDP process of IES multi-objective scheduling optimization using the multi-objective DDPG algorithm, and define an Actor network, multiple Critic networks, a target network, a loss function and an experience playback part to determine the DDPG algorithm;

[0249] Multiple reward functions are introduced into the DDPG algorithm, namely:

[0250] Q(s,a)=[Q1(s,a),Q2(s,a),...,Q m (s,a)] (16)

[0251] The Q-value function of reinforcement learning is expressed as Q(s,a), there are m objective functions and m Q-values;

[0252] The Actor network learns the optimal strategy π(a|s), and the Critic network evaluates the quality of the strategy based on the Q function Q(s,a). Its update rule follows the Bellman equation, and the target network parameter θ target Gradually approach the online network parameter θ in a soft update manner;

[0253] Define experience replay, build a replay storage area, and store the state, action, reward, and next state sampled from the environment each time into the replay buffer. The state space is the set of all states that the agent may be in, and the action space is the set of all actions that the agent can take in each state. The state transition probability describes the probability that the agent will transfer to the next state after taking a certain action in a certain state. The reward function defines the immediate reward obtained by the agent after taking a certain action in a certain state.

[0254] The reinforcement learning reward R is learned through formula 16 and formula 15, and the agent is guided to learn the optimal strategy through language and rules, that is, the optimal action sequence a that should be taken from each state t :

[0255]

[0256] A non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the multi-objective scheduling method of an integrated energy system guided by a rule language is implemented.

[0257] In summary, the present invention provides a method for multi-objective scheduling of an integrated energy system guided by rule language using GPT and DRL, which effectively solves the problem that quantitative modeling research on multi-objective scheduling is difficult to effectively simulate the comprehensive thinking process of people on multiple objectives in the loop. The model and method proposed in the present invention can better realize the multi-objective preference guidance of human beings, and provide theoretical and technical support for scheduling departments and personnel to arrange actual power grid power generation plans.

[0258] This method can quickly adjust the weights and priorities of multiple objectives based on real-time data and feedback. In practical applications, when the environment or conditions of the problem change, this method can timely perceive and adjust the multi-objective optimization strategy accordingly to adapt to the new situation. This dynamic adaptability can improve the effect and practicality of multi-objective optimization.

[0259] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0260] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0261] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0262] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-objective scheduling method for an integrated energy system guided by rule language, characterized in that: include: Establish a mathematical model for multi-objective scheduling of the integrated energy system IES; Combined with the large language model LLM, the mathematical model of IES multi-objective scheduling optimization is transformed into a Markov decision process MDP process, and an LLM agent for multi-objective guided interaction is constructed; The LLM agent uses the multi-objective deep deterministic policy gradient algorithm DDPG to solve the MDP process of IES multi-objective scheduling optimization, and obtains the decision-making plan for IES multi-objective scheduling.

2. The method according to claim 1, characterized in that The mathematical model for establishing IES multi-objective scheduling includes: Considering the economic, reliability and environmental protection goals, a mathematical model of IES multi-objective scheduling is constructed, and the objective function of the mathematical model is set as shown in formula (1): In the formula, p HB ,p EB ,p BES is the power of CHP, HB, EB and BES; η BES ,Q BES is BES efficiency and capacity; C buy ,C NG is the unit cost of electricity / natural gas purchased; p buy,t ,M NG,t is the amount of electricity / natural gas purchased; C CHP , C HB , C EB It is the operation and maintenance cost of CHP, HB and EB. grid ,Emission NG Carbon emissions per unit of electricity / natural gas. p load,e,t , p load,h,t For electrical / heat load demand; Considering the IES system balance constraints, various types of unit and line operation constraints, determine the constraints of IES multi-objective scheduling, including: IES system balance constraints: Line constraints: |P grid,t |≤P grid,max ,M NG,t ≤M NG,max (3) Operating constraints of various units: In the formula, p CHP ,p HB ,p EB ,p BES is the power of CHP, HB, EB and BES; Where η BES ,Q BES For BES efficiency and capacity.

3. The method according to claim 2, characterized in that The method of converting the mathematical model of IES multi-objective scheduling optimization into an MDP process by combining LLM includes: The mathematical model of IES multi-objective scheduling optimization is transformed into an MDP process, and the state space, action space and reward function of the MDP process are defined, including: Define the state space: Define the action space: The definition of reward function includes objective function and constraint penalty. The objective function is composed of economic cost, load imbalance degree in the system and carbon emission calculation. Constraint penalty is the penalty for the agent violating the constraint: R(n)=R t (n)+R P (n) (9) Where: R(n) is the reward function obtained by the nth power generation plan, R t (n) is the objective function reward obtained by the nth power generation plan, R P (n) the constraint penalty obtained for the nth generation plan; The objective function part of the MDP process is obtained as follows: R t (n)=―λ var ·s 2 (n)―λ t ·T(n)―λ m ·M(n) (10) Where: var is the scaling factor of the economic cost function, λ t is the scaling factor of the load imbalance function within the system, λ m is the scaling factor of the carbon emission function; The constraint penalty of the MDP process is: R P =―F B (P max +P soc +P load ) (11) Where: F B is the scaling factor of the constraint penalty function, P max P is the penalty for each unit and line exceeding the operating constraints. mis is the penalty for violating the energy storage system charging and discharging constraints, P con becomes the penalty for violating the equilibrium constraint.

4. The method according to claim 3, characterized in that The LLM agent constructed for multi-objective guided interaction includes: Guide the DRL agent training by talking to the LLM agent, preset the background c and the question q according to the content of the conversation with the LLM agent, the background c includes the rules and regulations for IES multi-objective scheduling, and the question q includes the IES multi-objective scheduling results and the weighted coefficient of the reward function; The LLM agent uses the concept of Token x to model the text. The answer of the LLM agent comes from the distribution of a generative model: Among them, x1,x2,…,x n It is a series of tokens obtained from the vocabulary by applying the probabilistic chain rule; Token is sampled by iteration x i+1 , and iteratively convert x i+1 Input the model to sample x i+2 To generate: Sampling sequence x i+1 ,...,x i+n The generated text is the output of the LLM agent, which is parsed into numerical scores of multiple objective function weights, namely: w i (c,q i )=parse W {x i+1 ,…,x i+n ∣a,s,c,q i } (14) a is the action space, formula 8; s is the state space, formula 7; c is the preset context; q is the question; i is the number of dialogue rounds; parse W The function represents the answer generated by the LLM agent under the current action space, state space, background and question; Quantify the rewards of the DRL agent and the penalties for violating constraints: R target =―Σ i w i (a,s,z,q j )·Q i (s,a) (15) Among them, R target Represents the target output containing weighted multiple rewards, quantitative DRL agent rewards.

5. The method according to claim 4, characterized in that The LLM agent uses the multi-objective deep deterministic policy gradient algorithm DDPG to solve the MDP process of IES multi-objective scheduling optimization to obtain the decision-making plan of IES multi-objective scheduling, including: The multi-objective DDPG algorithm is used to solve the MDP process of IES multi-objective scheduling optimization. One Actor network, multiple Critic networks, target network, loss function and experience playback are defined to determine the DDPG algorithm. Multiple reward functions are introduced into the DDPG algorithm, namely: Q(s,a)=[Q1(s,a),Q2(s,a),...,Q m (s,a)] (16) The Q-value function of reinforcement learning is expressed as Q(s,a), there are m objective functions and m Q-values; The Actor network learns the optimal strategy π(a|s), and the Critic network evaluates the quality of the strategy based on the Q function Q(s,a). Its update rule follows the Bellman equation, and the target network parameter θ target Gradually approach the online network parameter θ in a soft update manner; Define experience replay, build a replay storage area, and store the state, action, reward, and next state sampled from the environment each time into the replay buffer. The state space is the set of all states that the agent may be in, and the action space is the set of all actions that the agent can take in each state. The state transition probability describes the probability that the agent will transfer to the next state after taking a certain action in a certain state. The reward function defines the immediate reward obtained by the agent after taking a certain action in a certain state. The reinforcement learning reward R is learned through formula 16 and formula 15, and the agent is guided to learn the optimal strategy through language and rules, that is, the optimal action sequence a that should be taken from each state t :

6. A multi-objective scheduling device for an integrated energy system guided by rule language, characterized in that: include: Mathematical model building module, used to establish the mathematical model of multi-objective scheduling of integrated energy system IES; LLM agent building module, which is used to transform the mathematical model of IES multi-objective scheduling optimization into an MDP process by combining the large language model LLM, and to build an LLM agent for multi-objective guided interaction; The decision-making scheme acquisition module is used to solve the MDP process of IES multi-objective scheduling optimization by using the multi-objective deep deterministic policy gradient algorithm DDPG through the LLM intelligent agent to obtain the decision-making scheme of IES multi-objective scheduling.

7. The device according to claim 6, characterized in that The mathematical model building module is specifically used to consider the economic, reliability and environmental protection goals, build a mathematical model for IES multi-objective scheduling, and set the objective function of the mathematical model as shown in formula (1): In the formula, p HB ,p EB ,p BES is the power of CHP, HB, EB and BES; η BES ,Q BES is BES efficiency and capacity; C buy ,C NG is the unit cost of electricity / natural gas purchased; p buy,t ,M NG,t is the amount of electricity / natural gas purchased; C CHP , C HB , C EB It is the operation and maintenance cost of CHP, HB and EB. grid ,Emission NG Carbon emissions per unit of electricity / natural gas. p load,e,t , p load,h,t For electrical / heat load demand; Considering the IES system balance constraints, various types of unit and line operation constraints, determine the constraints of IES multi-objective scheduling, including: IES system balance constraints: Line constraints: |P grid,t |≤P grid,max ,M NG,t ≤M NG ,max (3) Operating constraints of various units: In the formula, p CHP ,p HB ,p EB ,p BES is the power of CHP, HB, EB and BES; Where η BES ,Q BES For BES efficiency and capacity.

8. The device according to claim 7, characterized in that The LLM agent building module is specifically used to transform the mathematical model of IES multi-objective scheduling optimization into an MDP process, define the state space, action space and reward function of the MDP process, including: Define the state space: Define the action space: The definition of reward function includes objective function and constraint penalty. The objective function is composed of economic cost, load imbalance degree in the system and carbon emission calculation. Constraint penalty is the penalty for the agent violating the constraint: R(n)=R t (n)+R P (n) (9) Where: R(n) is the reward function obtained by the nth power generation plan, R t (n) is the objective function reward obtained by the nth power generation plan, R P (n) the constraint penalty obtained for the nth generation plan; The objective function part of the MDP process is obtained as follows: R t (n)=―λ var ·s 2 (n)―λ t ·T(n)―λ m ·M(n) (10) Where: var is the scaling factor of the economic cost function, λ t is the scaling factor of the load imbalance function within the system, λ m is the scaling factor of the carbon emission function; The constraint penalty of the MDP process is: R P =―F B (P max +P soc +P load ) (11) Where: F B is the scaling factor of the constraint penalty function, P max P is the penalty for each unit and line exceeding the operating constraints. mis is the penalty for violating the energy storage system charging and discharging constraints, P con becomes a penalty for violating the equilibrium constraint; Guide the DRL agent training by talking to the LLM agent, preset the background c and the question q according to the content of the conversation with the LLM agent, the background c includes the rules and regulations for IES multi-objective scheduling, and the question q includes the IES multi-objective scheduling results and the weighted coefficient of the reward function; The LLM agent uses the concept of Token x to model the text. The answer of the LLM agent comes from the distribution of a generative model: Among them, x1,x2,…,x n It is a series of tokens obtained from the vocabulary by applying the probabilistic chain rule; Token is sampled by iteration x i+1 , and iteratively convert x i+1 Input the model to sample x i+2 To generate: Sampling sequence x i+1 ,...,x i+n The generated text is the output of the LLM agent, which is parsed into numerical scores of multiple objective function weights, namely: w i (c,q i )=parse W {x i+1 ,…,x i+n ∣a,s,c,q i } (14) a is the action space, formula 8; s is the state space, formula 7; c is the preset context; q is the question; i is the number of dialogue rounds; parse W The function represents the answer generated by the LLM agent under the current action space, state space, background and question; Quantify the rewards of the DRL agent and the penalties for violating constraints: R target =―∑ i w i (a,s,z,q j )·Q i (s,a) (15) Among them, R target Represents the target output containing weighted multiple rewards, quantitative DRL agent rewards.

9. The device according to claim 8, characterized in that The decision solution acquisition module is specifically used to solve the MDP process of IES multi-objective scheduling optimization using the multi-objective DDPG algorithm, and define an Actor network, multiple Critic networks, a target network, a loss function and an experience playback part to determine the DDPG algorithm; Multiple reward functions are introduced into the DDPG algorithm, namely: Q(s,a)=[Q1(s,a),Q2(s,a),...,Q m (s,a)] (16) The Q-value function of reinforcement learning is expressed as Q(s,a), there are m objective functions and m Q-values; The Actor network learns the optimal strategy π(a|s), and the Critic network evaluates the quality of the strategy based on the Q function Q(s,a). Its update rule follows the Bellman equation, and the target network parameter θ target Gradually approach the online network parameter θ in a soft update manner; Define experience replay, build a replay storage area, and store the state, action, reward, and next state sampled from the environment each time into the replay buffer. The state space is the set of all states that the agent may be in, and the action space is the set of all actions that the agent can take in each state. The state transition probability describes the probability that the agent will transfer to the next state after taking a certain action in a certain state. The reward function defines the immediate reward obtained by the agent after taking a certain action in a certain state. The reinforcement learning reward R is learned through formula 16 and formula 15, and the agent is guided to learn the optimal strategy through language and rules, that is, the optimal action sequence a that should be taken from each state t :

10. A non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the multi-objective scheduling method of an integrated energy system guided by a rule language as described in any one of claims 1 to 5 is implemented.

Images