Real-time optimal power flow method based on generative pre-training model (GPT) and deep reinforcement learning

Abstract

The invention discloses a real-time optimal power flow method based on a generative pre-training model (GPT) and deep reinforcement learning, relates to the technical field of intelligent regulation and control of a power grid, and utilizes technical breakthrough in the field of large language models to directly model non-quantifiable natural language stipulations as targets and constraints in an OPF problem. The method comprises the steps of establishing comprehensive performance indexes, setting contexts and problems, converting GPT agent feedback into reward signals, carrying out iterative sampling, outputting GPT agent results in a JSON format and importing the GPT agent results into a DRL agent, carrying out offline training and online application and the like. Three deep neural networks are used in the DRL process, a cost function is optimized through a primitive-dual method, a series of constraint conditions including language specification constraints are set, and the method can be applied online to solve the OPF problem in real time.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of power grids, and in particular to a real-time optimal power flow method based on a generative pre-training model (GPT) and deep reinforcement learning. Background Art

[0002] In the operation and management of power systems, the power flow method (OPF) has always played a vital role. Usually, the power flow method (OPF) is modeled as a complex mathematical optimization problem. Its objective function is often set to minimize the total power generation cost, and the power generation cost is optimized by continuously adjusting various parameters in the power system. At the same time, the power flow method (OPF) is also subject to a series of strict restrictions on the operation of the power system. These constraints cover voltage, current, power and other aspects to ensure that the power system operates within a safe and stable range. However, in the actual operation of the power system, grid regulations must be strictly observed. Most of these grid regulations are presented in the form of language regulations, which have certain ambiguity and complexity. These language regulations are difficult to directly convert into specific constraints in the mathematical model, making it difficult to accurately quantify them in the traditional power flow method (OPF) model. For example, some grid regulations may stipulate that under certain circumstances, the voltage fluctuation range of the power system shall not exceed a certain value, but this provision is difficult to express directly in mathematical formulas.

[0003] Many requirements in grid regulations and operation manuals follow different safety standards or policy regulations. These regulations come from a wide range of sources, which may come from national laws and regulations, industry standards, or internal management requirements of enterprises. Traditionally, the verification of these language regulations is mainly carried out through manual evaluation. This requires professional engineers and system operators to spend a lot of time and energy to analyze the operating status of the power system and determine whether it complies with the requirements of grid regulations. When regulations are found to be violated, system operators usually work with engineers to take a series of corrective measures to ensure that the power system can be restored to a compliant state as soon as possible.

[0004] Since system operators may give priority to these language provisions rather than pure operating costs in actual operations, after all, violating grid regulations may bring serious consequences, such as safety accidents and fines. Therefore, it is very valuable to include these qualitative language provisions in the OPF model. This can make the OPF model more in line with the actual power system operation requirements and improve the safety and reliability of the power system. Summary of the invention

[0005] The purpose of the present invention is to provide a real-time optimal power flow method based on a generative pre-trained model (GPT) and deep reinforcement learning, based on the recent technical breakthroughs in the field of large language models (LLMs), to achieve the possibility of directly modeling traditionally unquantifiable natural language regulations as objectives and constraints in power flow method (OPF) problems. And by considering the language regulations from power grid regulations or operating manuals, it is possible to directly model traditionally unquantifiable natural language regulations as objectives and constraints in power flow method (OPF) problems.

[0006] To achieve the above object, the present invention provides the following technical solution: a real-time optimal power flow method based on a generative pre-training model (GPT) and deep reinforcement learning, characterized in that the method comprises the following steps:

[0007] S1: Establish the comprehensive performance index of the optimization system for the power flow method (OPF) problem, including the total power generation cost and language regulations The degree of satisfaction is:

[0008]

[0009] in, The output of the ith generator is The cost function of It is a cost item based on language provisions, used to explain the degree of compliance with grid regulations or operation manuals under specific operating conditions, ω j is the weighted value of the language-based cost item;

[0010] S2: Set the context Z as the background, set the question as Q, and the answer of the large language model (LLM) as A. Specifically, the jth question input to the large language model (LLM) is recorded as Q j , the answer of the large language model (LLM) is recorded as A j , the number of power generation units is N G , the number of language questions is N Q ;

[0011] S3: Use the GPT agent feedback and convert it into a reward signal, and use the GPT agent to model the text (context Z and question Q) using the concept of tokenx; the answer of the GPT agent comes from the distribution generation model:

[0012]

[0013] Among them, x1,x2,…,x nIt is a series of tokens obtained from the vocabulary by applying the probabilistic chain rule. The GPT series of models is based on the transformer model and uses reinforcement learning through human feedback to maximize the probability of the best response in the training dataset given a prefix token.

[0014] S4: Iterative sampling, the GPT agent is required to generate new text (token) about rewards and trend explanations given the language context Z, that is:

[0015]

[0016] The language specification can be set in the context Z; the formula is expressed as sampling x by iteration i+1 , and x i+1 Gradually feed back to the model f LLM Take sample x i+2 generated in the manner of;

[0017] S5: In the sampling sequence x i+1 …,x i+n The generated text is the output of the GPT agent. These outputs can be set in JavaScript Object Notation (JSON) format. Given the context Z and question Q to construct the input token, the output JSON can be parsed into an explanation of the cost and constraints and a numerical score. The formula is:

[0018]

[0019] The parsing function converts the answer of GPT into a numerical value with a string explanation, which is then imported into the deep reinforcement learning (DRL) agent by integrating the parsed language explanation into the rewards and constraints. The reward quantization formula of the deep reinforcement learning (DRL) agent is as follows:

[0020]

[0021] The constraint violation penalty quantization formula of the deep reinforcement learning (DRL) agent is as follows:

[0022]

[0023] Among them, R target represents the target output of the reward critic network, quantifying the reward of the deep reinforcement learning (DRL) agent, C target represents the target output of the cost-critic network, quantifying the constraint violations of the deep reinforcement learning (DRL) agent;

[0024] S6: Perform offline training. In each round, when the deep reinforcement learning (DRL) agent interacts with the environment, i.e., the power system, the power flow results are packaged into text and questions Q for the GPT agent to evaluate the performance of the power flow method (OPF) solution in context Z.

[0025] S7: The reward of the DRL agent is calculated by integrating the power generation cost with the qualitative evaluation results in JSON format output by the GPT agent; after sufficient training, the DRL agent is applied online to solve the OPF problem.

[0026] In the deep reinforcement learning (DRL) process, three deep neural networks (DNNs) are used, including an actor network for determining the optimal scheduling action, a critic network R for approximating the expected reward, and a critic network C for approximating the constraint violation value.

[0027] In the learning phase, based on the security of the primal-dual deep deterministic policy gradient, deep reinforcement learning (DRL) is used for the reward quantization formula, while enforcing the constraint violation penalty quantization formula during the learning process; in the power flow method (OPF) problem, the action-value function of deep reinforcement learning (DRL) is modeled in the form of penalty as follows:

[0028]

[0029] L=-R(s i ,a i ,θ,z,q i )+λC(s i ,a i ,θ,z,q i )

[0030] Among them, L(a t ,θ,λ,μ) is the enhanced action-value function, and R is obtained by R target The reward critic output of the training, C, is obtained by C target Output of the trained cost critic; the mean squared error loss (MSE) is used to train the reward critic and the cost critic, θ represents the parameters of the deep neural network (DNN), λ represents the Lagrange multiplier for constraint satisfaction, and λ≥0.

[0031] Since the gradient of the multimodal reward shaped by the generative pre-trained model (GPT) agent with respect to the power flow method (OPF) action cannot be derived analytically, a pure primal-dual DRL method is adopted, and the critic neural network is used to deal with non-differentiable rewards. By approximating the non-differentiable language target reward as a continuous differentiable function, the gradient descent algorithm can be used to optimize the RT-OPF containing language regulations;

[0032] To optimize the deep neural network (DNN) parameters θ to minimize the enhanced cost function L, its update equation is based on stochastic gradient descent as follows:

[0033]

[0034] Where η is the learning rate, is the gradient of the enhancement cost function with respect to the deep neural network (DNN) parameter θ. Through the primal-dual method, the expected reward in the reward quantization formula of the deep reinforcement learning (DRL) agent can be maximized while optimizing the total generation cost C G and qualitative performance C Q , and reduce the constraint violation situations in the constraint violation penalty quantification formula.

[0035] The setting constraints include node power balance, generation limit, bus voltage limit, line flow limit and language specification A k (Answer by GPT agent) Constraints:

[0036]

[0037]

[0038] A k,i (a,s,z,q i )≤A k,i,max (s,z,q i )

[0039] in, Respectively represent the active load, reactive load, active generation and reactive generation of the i-th node, δ ij is the phase angle difference between nodes i and j, V j is the voltage amplitude of the jth node, |S ij | is the apparent power flow between nodes i and j, Y ij =G ij +jB ij is the (i,j) element of the node admittance matrix, and the qualitative constraints specified by the k-th language based on the language are given by A k Represented and interpreted by a Large Language Model (LLM).

[0040] The question Q is the prompt or string input provided to the Large Language Model (LLM).

[0041] The GPT agent is a probabilistic model and the knowledge sources it refers to may not be completely reliable, so it is necessary to specify them explicitly in the context Z.

[0042] The qualitative constraints considered by the method are that at least one language stipulates that the voltage limit or the backup safety is always met, and other language provisions can be considered similarly.

[0043] The state of the DRL agent is the net node power load of the system, and the action is the generation scheduling instruction. The goal of the DRL agent is to minimize the total generation cost and maximize the satisfaction of the language provisions while satisfying the operational constraints.

[0044] By integrating GPT agents into a deep reinforcement learning (DRL) training loop, the DRL agent can be optimized taking into account human-interpretable reasoning, criticism, and language-specified rewards.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The traditional power flow method (OPF) model is difficult to accurately quantify the power grid regulations based on language regulations. However, based on the generative pre-training model (GPT) and deep reinforcement learning, the present invention uses the technical breakthrough in the field of large language model (LLM) to directly model the traditionally unquantifiable natural language regulations as the goals and constraints in the power flow method (OPF) problem. This enables the requirements in the power grid regulations and operation manuals to be more effectively incorporated into the optimization model, improving the practicality and accuracy of the model.

[0047] Traditionally, verification of language regulations is mainly done through manual evaluation, which requires professional engineers and system operators to spend a lot of time and effort. The present invention, through the combination of GPT agents and deep reinforcement learning (DRL) agents, can automatically evaluate the operating status of the power system and generate corresponding reward and constraint information. This not only improves the efficiency of the evaluation, but also reduces the errors caused by human factors and improves the accuracy of the evaluation.

[0048] In actual operation, system operators may prioritize language regulations over pure operating costs, because violating grid regulations may have serious consequences. The present invention incorporates language regulations into the power flow method (OPF) model, allowing the deep reinforcement learning (DRL) agent to consider both the total power generation cost and the satisfaction of language regulations during the optimization process, thereby making more reasonable decisions. By integrating the GPT agent, the deep reinforcement learning (DRL) agent can optimize while considering human-interpretable reasoning, criticism, and language regulation rewards, further improving the quality of decision making.

[0049] By setting a series of constraints including node power balance, generation limit, bus voltage limit, line flow limit and language-specified constraints, it is ensured that the power system meets the actual physical and operational requirements during the optimization process. In particular, the qualitative constraints specified by the language are considered, such as at least one language-specified voltage limit or backup security is always met, which improves the safety and reliability of the power system.

[0050] After sufficient training, the deep reinforcement learning (DRL) agent can be applied online to solve the OPF problem in real time. This enables the method of the present invention to respond to changes in the operation of the power system in a timely manner, improving the response speed and adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 The present invention provides a real-time optimal power flow method framework based on a generative pre-training model (GPT) and deep reinforcement learning; DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be fully described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0053] like Figure 1 As shown, a real-time optimal power flow method based on a generative pre-training model (GPT) and deep reinforcement learning is characterized in that the method comprises the following steps:

[0054] S1: Establish the comprehensive performance index of the optimization system for the power flow method (OPF) problem, including the total power generation cost and language regulations The degree of satisfaction is:

[0055]

[0056] in, The output of the ith generator is The cost function of It is a cost item based on language provisions, used to explain the degree of compliance with grid regulations or operation manuals under specific operating conditions, ω j is the weighted value of the language-based cost item;

[0057] S2: Set the context Z as the background, set the question as Q, and the answer of the large language model (LLM) as A. Specifically, the jth question input to the large language model (LLM) is recorded as Q j , the answer of the large language model (LLM) is recorded as A j , the number of power generation units is N G , the number of language questions is N Q ;

[0058] S3: Use the GPT agent feedback and convert it into a reward signal, and use the GPT agent to model the text (context Z and question Q) using the concept of tokenx; the answer of the GPT agent comes from the distribution generation model:

[0059]

[0060] Among them, x1,x2,…,x n It is a series of tokens obtained from the vocabulary by applying the probabilistic chain rule. The GPT series of models is based on the transformer model and uses reinforcement learning through human feedback to maximize the probability of the best response in the training dataset given a prefix token.

[0061] S4: Iterative sampling, the GPT agent is required to generate new text (token) about rewards and trend explanations given the language context Z, that is:

[0062]

[0063] The language specification can be set in the context Z; the formula is expressed as sampling x by iteration i+1 , and x i+1 Gradually feed back to the model f LLM Take sample x i+2 generated in the manner of;

[0064] S5: In the sampling sequence x i+1 …,x i+n The generated text is the output of the GPT agent. These outputs can be set in JavaScript Object Notation (JSON) format. Given the context Z and question Q to construct the input token, the output JSON can be parsed into an explanation of the cost and constraints and a numerical score. The formula is:

[0065]

[0066] The parsing function converts the answer of GPT into a numerical value with a string explanation, which is then imported into the deep reinforcement learning (DRL) agent by integrating the parsed language explanation into the rewards and constraints. The reward quantization formula of the deep reinforcement learning (DRL) agent is as follows:

[0067]

[0068] The constraint violation penalty quantization formula of the deep reinforcement learning (DRL) agent is as follows:

[0069]

[0070] Among them, R target represents the target output of the reward critic network, quantifying the reward of the deep reinforcement learning (DRL) agent, C target represents the target output of the cost-critic network, quantifying the constraint violations of the deep reinforcement learning (DRL) agent;

[0071] S6: Perform offline training. In each round, when the deep reinforcement learning (DRL) agent interacts with the environment, i.e., the power system, the power flow results are packaged into text and questions Q for the GPT agent to evaluate the performance of the power flow method (OPF) solution in context Z.

[0072] S7: The reward of the DRL agent is calculated by integrating the power generation cost with the qualitative evaluation results in JSON format output by the GPT agent; after sufficient training, the DRL agent is applied online to solve the OPF problem.

[0073] In the deep reinforcement learning (DRL) process, three deep neural networks (DNNs) are used, including an actor network for determining the optimal scheduling action, a critic network R for approximating the expected reward, and a critic network C for approximating the constraint violation value.

[0074] In the learning phase, based on the security of the primal-dual deep deterministic policy gradient, deep reinforcement learning (DRL) is used for the reward quantization formula, while enforcing the constraint violation penalty quantization formula during the learning process; in the power flow method (OPF) problem, the action-value function of deep reinforcement learning (DRL) is modeled in the form of penalty as follows:

[0075]

[0076] L=-R(s i ,a i ,θ,z,qi )+λC(s i ,a i ,θ,z,q i )

[0077] Among them, L(a t ,θ,λ,μ) is the enhanced action-value function, and R is obtained by R target The reward critic output of the training, C, is obtained by C target Output of the trained cost critic; the mean squared error loss (MSE) is used to train the reward critic and the cost critic, θ represents the parameters of the deep neural network (DNN), λ represents the Lagrange multiplier for constraint satisfaction, and λ≥0.

[0078] Since the gradient of the multimodal reward shaped by the generative pre-trained model (GPT) agent with respect to the power flow method (OPF) action cannot be derived analytically, a pure primal-dual DRL method is adopted, and the critic neural network is used to deal with non-differentiable rewards. By approximating the non-differentiable language target reward as a continuous differentiable function, the gradient descent algorithm can be used to optimize the RT-OPF containing language regulations;

[0079] To optimize the deep neural network (DNN) parameters θ to minimize the enhanced cost function L, its update equation is based on stochastic gradient descent as follows:

[0080]

[0081] Where η is the learning rate, is the gradient of the enhancement cost function with respect to the deep neural network (DNN) parameter θ. Through the primal-dual method, the expected reward in the reward quantization formula of the deep reinforcement learning (DRL) agent can be maximized while optimizing the total generation cost C G and qualitative performance C Q , and reduce the constraint violation situations in the constraint violation penalty quantification formula.

[0082] The setting constraints include node power balance, generation limit, bus voltage limit, line flow limit and language specification A k (Answer by GPT agent) Constraints:

[0083]

[0084] A k,i (a,s,z,q i )≤A k,i,max (s,z,q i )

[0085] in, Respectively represent the active load, reactive load, active generation and reactive generation of the i-th node, δ ij is the phase angle difference between nodes i and j, V j is the voltage amplitude of the jth node, |S ij | is the apparent power flow between nodes i and j, Y ij =G ij +jB ij is the (i,j) element of the node admittance matrix, and the qualitative constraints specified by the k-th language based on the language are given by A k Represented and interpreted by a Large Language Model (LLM).

[0086] The question Q is the prompt or string input provided to the Large Language Model (LLM).

[0087] The GPT agent is a probabilistic model and the knowledge sources it refers to may not be completely reliable, so it is necessary to specify them explicitly in the context Z.

[0088] The qualitative constraints considered by the method are that at least one language stipulates that the voltage limit or the backup safety is always met, and other language provisions can be considered similarly.

[0089] The state of the DRL agent is the net node power load of the system, and the action is the generation scheduling instruction. The goal of the DRL agent is to minimize the total generation cost and maximize the satisfaction of the language provisions while satisfying the operational constraints.

[0090] By integrating GPT agents into a deep reinforcement learning (DRL) training loop, the DRL agent can be optimized taking into account human-interpretable reasoning, criticism, and language-specified rewards.

[0091] First, by establishing a comprehensive performance index of the optimized system including the total power generation cost and the degree of satisfaction of the language regulations. For the total power generation cost, it reflects the economic cost of the power system in the power generation process, which is usually related to the output power of each generator. The introduction of the degree of satisfaction of the language regulations is the key point. By setting the cost items based on the language regulations and the weighted values, the language regulations in the grid regulations and operation manuals that were originally difficult to quantify are converted into cost factors that can be considered in the optimization process. In this way, both the economic cost is taken into account and the system can be ensured to meet the qualitative language regulations.

[0092] The context Z and question Q are set to provide input information for the GPT agent and the deep reinforcement learning (DRL) agent. The scale and complexity of the problem are determined by specifying the number of power generation units and the number of language questions. The context Z describes the operating status of the power system and related background information, and the question Q guides the large language model (LLM) to generate specific answers. For example, question Q can be a query about whether the system currently meets a specific language requirement. The large language model generates the corresponding answer A based on the input context and question.

[0093] The GPT agent uses the concept of tokens to model text (context Z and question Q). Its answers come from a distribution generation model that generates a series of tokens from a vocabulary by applying the probabilistic chain rule. The GPT series of models is based on the transformer architecture and uses reinforcement learning with human feedback to maximize the probability of the optimal response in the training dataset given a prefix token. In this way, the GPT agent can generate answers with a certain probability distribution based on the input text information, providing a basis for subsequent conversion into reward signals. Specifically, by analyzing the generated text, information related to rewards and trend interpretation can be extracted, which can be converted into reward signals to guide the optimization process of the deep reinforcement learning (DRL) agent.

[0094] Iterative sampling is performed, requiring the GPT agent to generate new text (token) about the explanation of rewards and trends under the given language context Z. New tokens are continuously extracted from the distribution generation model until a complete text is generated. In this process, the language regulations can be set as needed, and the text generated by the GPT agent can be controlled to be related to specific language regulations by adjusting the context Z. After processing, the generated text can be converted into an explanation of the cost and constraints and a numerical score, and output in JSON format.

[0095] The output JSON formatted text is parsed into explanations and numerical scores of costs and constraints, and then imported into the deep reinforcement learning (DRL) agent. The reward quantization formula of the deep reinforcement learning (DRL) agent combines the output of the reward critic network and the output of the cost critic network to balance the importance of rewards and constraint violations. During the learning phase, deep reinforcement learning (DRL) is used for the reward quantization formula based on the safety of primal-dual deep deterministic policy gradients, and the constraint violation penalty quantization formula is enforced during the learning process. By using three deep neural networks (actor network, reward critic network R, and cost critic network C), the deep reinforcement learning (DRL) agent can be optimized while considering human-interpretable reasoning, criticism, and language-specified rewards. The actor network selects the optimal scheduling action based on the current system state, the critic network R approximates the expected reward to help the actor network evaluate the quality of different actions, and the critic network C approximates the constraint violation value to ensure that the system is optimized while satisfying the constraints.

[0096] A series of constraints are set, including node power balance, generation limit, bus voltage limit, line flow limit and language specification constraints. These constraints ensure that the power system meets the actual physical and operational requirements during the optimization process. The node power balance constraint ensures the balance between the supply and demand of active power and reactive power; the generation limit constraint limits the output power range of the generator; the bus voltage limit constraint ensures that the bus voltage is within a safe range; the line flow limit constraint prevents line overload; and the language specification constraint reflects the compliance with grid regulations or operating manuals. By enforcing these constraints during the optimization process, the safe and stable operation of the system can be guaranteed.

[0097] During the offline training phase, the DRL agent interacts with the power system, and the power flow results are packaged as text and questions Q for the GPT agent to evaluate the performance of the OPF solution in context Z. Through continuous iterative training, the DRL agent gradually learns the optimal strategy. After sufficient training, the DRL agent is applied online to solve the OPF problem in real time, achieving the goals of minimizing the total power generation cost, maximizing the satisfaction of language regulations, and satisfying operational constraints.

Claims

1. A real-time optimal power flow method based on a generative pre-training model (GPT) and deep reinforcement learning, characterized in that: The method comprises the following steps: S1: Establish the comprehensive performance index of the optimization system for the power flow method (OPF) problem, including the total power generation cost and language regulations The degree of satisfaction is: in, The output of the ith generator is The cost function of It is a cost item based on language provisions, used to explain the degree of compliance with grid regulations or operation manuals under specific operating conditions, ω j is the weighted value of the language-based cost item; S2: Set the context Z as the background, set the question as Q, and the answer of the large language model (LLM) as A. Specifically, the jth question input to the large language model (LLM) is recorded as Q j , the answer of the large language model (LLM) is recorded as A j , the number of power generation units is N G , the number of language questions is N Q ; S3: Use the GPT agent feedback and convert it into a reward signal, and use the GPT agent to model the text (context Z and question Q) using the concept of tokenx; the answer of the GPT agent comes from the distribution generation model: Among them, x1,x2,…,x n It is a series of tokens obtained from the vocabulary by applying the probabilistic chain rule. The GPT series of models is based on the transformer model and uses reinforcement learning through human feedback to maximize the probability of the best response in the training dataset given a prefix token. S4: Iterative sampling, the GPT agent is required to generate new text (token) about rewards and trend explanations given the language context Z, that is: The language specification can be set in the context Z; the formula is expressed as sampling x by iteration i+1 , and x i+1 Gradually feed back to the model f LLM Take sample x i+2 generated in the following way; S5: In the sampling sequence x i+1 …,x i+n The generated text is the output of the GPT agent. These outputs can be set in JavaScript Object Notation (JSON) format. Given the context Z and question Q to construct the input token, the output JSON can be parsed into an explanation of the cost and constraints and a numerical score. The formula is: The parsing function converts the answer of GPT into a numerical value with a string explanation, which is then imported into the deep reinforcement learning (DRL) agent by integrating the parsed language explanation into the rewards and constraints. The reward quantization formula of the deep reinforcement learning (DRL) agent is as follows: The constraint violation penalty quantization formula of the deep reinforcement learning (DRL) agent is as follows: Among them, R target represents the target output of the reward critic network, quantifying the reward of the deep reinforcement learning (DRL) agent, C target represents the target output of the cost-critic network, quantifying the constraint violations of the deep reinforcement learning (DRL) agent; S6: Perform offline training. In each round, when the deep reinforcement learning (DRL) agent interacts with the environment, i.e., the power system, the power flow results are packaged into text and questions Q for the GPT agent to evaluate the performance of the power flow method (OPF) solution in context Z. S7: The reward of the DRL agent is calculated by integrating the power generation cost with the qualitative evaluation results in JSON format output by the GPT agent; after sufficient training, the DRL agent is applied online to solve the OPF problem.

2. A real-time optimal power flow method based on a generative pre-training model (GPT) and deep reinforcement learning according to claim 1, characterized in that: In the deep reinforcement learning (DRL) process, three deep neural networks (DNNs) are used, including an actor network for determining the optimal scheduling action, a critic network R for approximating the expected reward, and a critic network C for approximating the constraint violation value.

3. The real-time optimal power flow method based on generative pre-training model (GPT) and deep reinforcement learning according to claim 1, characterized in that: In the learning phase, based on the security of the primal-dual deep deterministic policy gradient, deep reinforcement learning (DRL) is used for the reward quantization formula, while enforcing the constraint violation penalty quantization formula during the learning process; in the power flow method (OPF) problem, the action-value function of deep reinforcement learning (DRL) is modeled in the form of penalty as follows: L=-R(s i ,a i ,θ,z,q i )+λC(s i ,a i ,θ,z,q i ) Among them, L(a t ,θ,λ,μ) is the enhanced action-value function, and R is obtained by R target The reward critic output of the training, C, is obtained by C target Output of the trained cost critic; the mean squared error loss (MSE) is used to train the reward critic and the cost critic, θ represents the parameters of the deep neural network (DNN), λ represents the Lagrange multiplier for constraint satisfaction, and λ≥0.

4. The real-time optimal power flow method based on generative pre-training model (GPT) and deep reinforcement learning according to claim 1, characterized in that: Since the gradient of the multimodal reward shaped by the generative pre-trained model (GPT) agent with respect to the power flow method (OPF) action cannot be derived analytically, a pure primal-dual DRL method is adopted, and the critic neural network is used to deal with non-differentiable rewards. By approximating the non-differentiable language target reward as a continuous differentiable function, the gradient descent algorithm can be used to optimize the RT-OPF containing language regulations; To optimize the deep neural network (DNN) parameters θ to minimize the enhanced cost function L, its update equation is based on stochastic gradient descent as follows: Where η is the learning rate, is the gradient of the enhancement cost function with respect to the deep neural network (DNN) parameter θ. Through the primal-dual method, the expected reward in the reward quantization formula of the deep reinforcement learning (DRL) agent can be maximized while optimizing the total generation cost C G and qualitative performance C Q , and reduce the constraint violation situations in the constraint violation penalty quantification formula.

5. The real-time optimal power flow method based on generative pre-training model (GPT) and deep reinforcement learning according to claim 1, characterized in that: The setting constraints include node power balance, generation limit, bus voltage limit, line flow limit and language specification A k (Answer by GPT agent) Constraints: A k,i (a,s,z,q i )≤A k,i,max (s,z,q i ) in, Respectively represent the active load, reactive load, active generation and reactive generation of the i-th node, δ ij is the phase angle difference between nodes i and j, V j is the voltage amplitude of the jth node, |S ij | is the apparent power flow between nodes i and j, Y ij =G ij +jB ij is the (i,j) element of the node admittance matrix, and the qualitative constraints specified by the k-th language based on the language are given by A k Represented and interpreted by a Large Language Model (LLM).

6. A real-time optimal power flow method based on a generative pre-training model (GPT) and deep reinforcement learning according to claim 1, characterized in that: The question Q is the prompt or string input provided to the Large Language Model (LLM).

7. The real-time optimal power flow method based on generative pre-training model (GPT) and deep reinforcement learning according to claim 1, characterized in that: The GPT agent is a probabilistic model and the knowledge sources it refers to may not be completely reliable, so it is necessary to specify them explicitly in the context Z.

8. The real-time optimal power flow method based on generative pre-training model (GPT) and deep reinforcement learning according to claim 1, characterized in that: The qualitative constraints considered by the method are that at least one language stipulates that the voltage limit or the backup safety is always met, and other language provisions can be considered similarly.

9. The real-time optimal power flow method based on generative pre-training model (GPT) and deep reinforcement learning according to claim 1, characterized in that: The state of the DRL agent is the net node power load of the system, and the action is the generation scheduling instruction. The goal of the DRL agent is to minimize the total generation cost and maximize the satisfaction of the language provisions while satisfying the operational constraints.

10. The real-time optimal power flow method based on generative pre-training model (GPT) and deep reinforcement learning according to claim 1, characterized in that: By integrating GPT agents into a deep reinforcement learning (DRL) training loop, the DRL agent can be optimized taking into account human-interpretable reasoning, criticism, and language-specified rewards.

Images