Decision generation method and system for intra-day prospective scheduling of power grid based on SAC

By introducing the SAC algorithm, the training efficiency and stability of power grid dispatching are improved, the problems of large data requirements and slow response speed in existing methods are solved, and dynamic adjustment and stability of power grid operation are realized.

CN118982161APending Publication Date: 2024-11-19STATE GRID JIANGSU ELECTRIC POWER CO LTD +3
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410820765.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-24
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

Existing machine learning-based power grid dispatching methods require a large amount of training data and computing resources, have slow response speeds, and are prone to overfitting and training instability when dealing with high-dimensional, nonlinear, and time-varying data, leading to a decrease in the reliability and accuracy of dispatching results.

Method used

A method for generating intraday forward-looking scheduling decisions for power grids based on the SoftActor-Critic (SAC) algorithm is adopted. By collecting future state prediction section data of power grid operation, conducting convergence tests, initializing the SAC deep learning model, generating adjustment actions, and storing experience to update model weights, the training efficiency and stability of the model are improved.

Benefits of technology

It enables dynamic monitoring and adjustment of the power grid status, ensuring the stability and security of power grid operation. The model can adapt to new operating environments and requirements, and has strong scalability and application prospects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982161A_ABST
    Figure CN118982161A_ABST
Patent Text Reader

Abstract

The invention discloses a power grid intra-day prospective scheduling decision generation method and system based on SAC, and the method comprises the steps: collecting power grid operation future state prediction section data, and taking the data as a training sample; and carrying out convergence test on the sample, and judging and selecting a key section. And initializing a training state of the SAC deep learning model, and generating an adjustment action. And storing experience, and updating the model weight. According to the SAC-based power grid intra-day prospective scheduling decision generation method and system provided by the invention, the convergence test is performed on the sample, the validity and representativeness of the training data are ensured, and the training quality of the model is improved. By initializing the SAC model and generating the adjustment action, the dynamic monitoring and adjustment of the power grid state are realized, and the stability and safety of the operation of the power grid are ensured. The experience is stored and the model weight is updated, so that the model can continuously adapt to new operating environments and requirements, and the method has relatively strong expansibility and application prospect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of decision-making for intraday forward-looking dispatching of power grids, and in particular to a decision-making generation method and system for intraday forward-looking dispatching of power grids based on SAC. Background Art

[0002] With the continuous increase in electricity demand and the increasingly complex power grid operation environment, power grid dispatching faces increasing challenges. Traditional dispatching methods are often unable to effectively cope with the dynamic changes and various uncertainties in power grid operation, resulting in frequent over-limit of key transmission lines, affecting the stable operation of the power grid. Therefore, there is an urgent need for an advanced method that can dynamically optimize the power grid dispatching strategy based on multiple factors. Summary of the invention

[0003] In view of the above-mentioned problems, the present invention is proposed.

[0004] Therefore, the technical problem solved by the present invention is that the existing power grid dispatching methods based on machine learning still have some shortcomings. First, these methods usually require a large amount of training data and computing resources, and have limited response speed to changes in power grid status and environment. Secondly, traditional machine learning models are prone to overfitting and training instability when processing high-dimensional, nonlinear and time-varying data, resulting in reduced reliability and accuracy of dispatching results. In response to these problems, the present invention proposes a method for generating intraday forward-looking dispatching decisions for power grids based on a soft actor-critic (SAC) algorithm. By introducing the SAC algorithm in reinforcement learning, the training efficiency and stability of the model are improved, thereby achieving more efficient and reliable power grid dispatching.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: a decision-making method for intraday forward-looking scheduling of power grids based on SAC, comprising: collecting future state prediction section data of power grid operation as training samples.

[0006] Conduct convergence test on samples and judge and select key sections.

[0007] Initialize the SAC deep learning model training state and generate adjustment actions.

[0008] Store experience and update model weights.

[0009] As a preferred solution of the decision-making method for intraday forward-looking scheduling of power grids based on SAC described in the present invention, the collection of future state prediction section data of power grid operation as training samples includes, in the power dispatching system, deriving the future state prediction section of power grid operation every 5 minutes, obtaining 16 sections every 5 minutes, and forming these 16 sections into a group of training samples in the SAC model training process.

[0010] As a preferred solution of the decision-making method for intraday forward-looking scheduling of power grid based on SAC described in the present invention, wherein: the convergence test of the sample includes setting the initial voltage vector V for all computing nodes (0) , choose a small positive number ∈ as the convergence criterion.

[0011] Calculate the power imbalance and for each computing node i, calculate the actual value S of the injected power i and estimated values The difference is expressed as:

[0012]

[0013] in, represents the specified active power of computing node i, represents the specified reactive power of computing node i, and Represents the active and reactive power calculated from the current voltage estimate, expressed as:

[0014]

[0015] Among them, V i represents the voltage at node i, V j represents the voltage at node j, G ij Represents the real part of the admittance matrix between node i and node j, called conductance. ij represents the voltage phase difference between node i and node j, B ij Represents the imaginary part of the admittance matrix between node i and node j, called susceptance.

[0016] Construct the Jacobian matrix J, expressed as:

[0017]

[0018] Among them, ΔP represents the active power imbalance of the node, ΔQ represents the reactive power imbalance of the node, θ represents the phase angle of the node voltage, and |V| represents the amplitude of the node voltage.

[0019] Each element is expressed in the form of:

[0020]

[0021]

[0022] Solving the linear equation system JΔx = -f(x), we get the voltage increment, expressed as:

[0023]

[0024] Among them, Δθ represents the incremental vector of the voltage phase angle of each node, and Δ|V| represents the incremental vector of the voltage amplitude of each node.

[0025] Update voltage estimate:

[0026] θ (k+1) =θ (k) +Δθ

[0027] |V| (k+1) =|V| (k) +Δ|V|

[0028] Among them, θ (k+1) represents the node voltage phase angle vector at the k+1th iteration, θ (k) represents the node voltage phase angle vector at the kth iteration, |V| (k+1) represents the node voltage amplitude vector at the k+1th iteration, |V| (k) represents the node voltage amplitude vector at the kth iteration, and Δθ represents the voltage phase angle increment vector.

[0029] Check whether the maximum values ​​of ΔP and ΔQ are less than the predetermined convergence standard ∈. If so, the artificial power flow calculation converges, otherwise continue to calculate the power imbalance.

[0030] As a preferred solution of the decision-making method for intraday forward-looking dispatch of power grid based on SAC of the present invention, wherein: the judging and selecting of key sections includes calculating the total over-limit quantity of key transmission lines in each section. i , expressed as:

[0031]

[0032] Among them, P i represents the active power of the i-th key transmission line, P imax represents the active power exceeding value of the ith key transmission line, and m represents the total number of key transmission lines.

[0033] Find the section D closest to the current time where the critical transmission line exceeds the limit tmin , D tmin = arg min i t i And O i >0, where t i is the time difference of section i.

[0034] Find the section D with the largest total line limit vmax :D vmax = arg max i (Q i ).

[0035] As a preferred solution of the decision-making method for intraday forward-looking scheduling of power grid based on SAC described in the present invention, wherein: the initialization of the SAC deep learning agent training state includes, the initialization state S includes the section D tmin And section D vmax , expressed as:

[0036] S={P line ,Q line ,V bus ,P gen}

[0037] The initial state includes the active power P of the line in the section. line 、Line reactive power Q line , bus voltage V bus and the active power P of the unit gen .

[0038] For D tmin The cross section, state S is represented by the following vector:

[0039] S={p line1 ,...,p linen ,q line1 ,...,q linen ,v bus2 ,...,v busn ,p gen1 ,...,p linen}

[0040] Initializing S means setting all variables in it to 0.

[0041] Initialize the experience replay buffer It is expressed as:

[0042]

[0043] Among them, S represents the current state of the environment, A represents the action selected according to the strategy in the current environment, that is, the adjustment amount of each unit in the section. R represents the reward given after the action is selected, S' represents the next power grid environment after the action is executed, and d represents the flag of whether this round is over.

[0044] For the cross-section state S after adjustment t times t , select action a t :a t =π θ (S t ), action a t Expressed as a vector containing the recommended adjustments for all units in the section, π θ (S t) represents the strategy function, according to the current grid state S t Get the corresponding action.

[0045] During the adjustment process, the adjustment rate of the unit is considered to ensure effective adjustment within a limited time. The maximum adjustment rate of the unit is R max , the active power adjustment of the unit is ΔP gen , then the maximum adjustment amount of the unit within time Δt is expressed as:

[0046] ΔPgen,max=Rmax·Δt

[0047] Among them, Δt represents two sections D tmin With D vmax When making adjustments, it is necessary to ensure that each adjustment ΔPgen does not exceed the maximum adjustment calculated above:

[0048] ΔP gen =min(ΔP gen,desire ,ΔP gen,max )

[0049] Where ΔP gen,desire Represents the ideal adjustment calculated by the algorithm based on the current state and reward function.

[0050] As a preferred solution of the decision-making method for intraday forward-looking scheduling of power grid based on SAC described in the present invention, wherein: the generating adjustment action includes, for the obtained action a t After executing this action, the power flow calculation is performed to obtain the state of the section in the next stage, and the next state s is obtained through the reward function t+1 and instant rewards t , expressed as:

[0051] S t+1 ,r t =Environment(S t ,a t )

[0052] Among them, S t+1 = {P t+1,line ,Q t+1,line ,V t+1,bus ,P t+1,gen}.

[0053] When the over-limit of the key transmission line in the next section becomes smaller, a positive reward is given to the model. If it does not become smaller, a negative reward is given.

[0054] Calculate the reward value of two key sections, expressed as:

[0055]

[0056]

[0057] in, Indicates D tmin The total line crossing limit of the section, Indicates D vmax The total line exceeding limit of the section, α represents the weight coefficient, reflecting the size of the difference in exceeding limit, and β represents the control system, which is input by the interface.

[0058] The reward function rt is expressed as:

[0059]

[0060] in, At the end of the round, the value is 300, and at the end of the round, the value is The value is 300 at the end of the round, and -0 before the end of the round Dvmax .

[0061] As a preferred solution of the decision-making method for intraday forward-looking scheduling of power grid based on SAC described in the present invention, wherein: the storing experience and updating the model weights include replaying the experience buffer from the experience A small batch of samples (S t ,a t ,r t ,s'), calculate the target value y:

[0062]

[0063] Among them, y represents the target Q value, which refers to the expected cumulative reward after the model selects action a in state s, and is used to update the Q function. r represents the immediate reward, which means the reward obtained after taking action a in the current state s. γ represents the discount factor, which is used to weigh the relative importance of future rewards and current rewards. represents the expected value of action a' generated according to strategy π, Q φ' (s',a') represents the target Q function, which represents the Q value obtained by taking action a' in the new state s'. φ' represents the parameters of the Q function, α represents the temperature parameter, which controls the randomness of the strategy, and logπ(a'|s') is the logarithm of the probability that strategy π takes action a' in state s'.

[0064] The loss function of minimizing the mean square error is expressed as:

[0065]

[0066] Among them, Q φ(s,a) represents the expected cumulative reward after taking action a in state s under the Q function parameter φ.

[0067] Update strategy network π θ To maximize the objective function:

[0068]

[0069] Among them, π θ represents a parameterized strategy, represents the sum of the cumulative rewards, γ t Represents the discount factor, which is used to weigh the relative importance of future rewards and current rewards. t represents the time step, which means that the reward after each step is decayed by γ times. t ,a t ) represents the immediate reward function, which is in the state s at time step t t Take action a t Instant rewards received.

[0070] Update the value function:

[0071]

[0072] Among them, Q φ (s,a) represents the Q function, which represents the expected cumulative reward of taking action a in state s. φ' (s',a') represents the expected cumulative reward under the next state s' and action a', logπ(a'|s') represents the logarithm of the probability of selecting action a' in state s' under strategy π, and represents the entropy of the strategy.

[0073] A decision-making system for intraday forward-looking dispatch of power grid based on SAC, characterized by: comprising:

[0074] The sample collection module collects the predicted section data of the future state of power grid operation as training samples.

[0075] Select the critical section module, perform convergence test on the samples, and judge and select the critical section.

[0076] The model training module initializes the SAC deep learning model training state and generates adjustment actions.

[0077] Update model modules, store experience, and update model weights.

[0078] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0079] A computer-readable storage medium stores a computer program, which implements the steps of the method described above when executed by a processor.

[0080] Beneficial effects of the present invention: By collecting the cross-sectional data of the future state prediction of the power grid operation, the model can obtain the latest state of the power grid operation in a timely manner, avoiding the problem of inaccurate scheduling due to data lag. The convergence test of the samples ensures the validity and representativeness of the training data and improves the training quality of the model. By initializing the SAC model and generating adjustment actions, dynamic monitoring and adjustment of the power grid state are achieved, ensuring the stability and safety of the power grid operation. Storing experience and updating model weights enable the model to continuously adapt to new operating environments and requirements, and has strong scalability and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0082] Figure 1 An overall flow chart of a decision-making method for intraday forward-looking scheduling of a power grid based on SAC provided in the first embodiment of the present invention;

[0083] Figure 2 A model training parameter selection diagram for a decision-making method for intraday forward-looking scheduling of a power grid based on SAC provided in the first embodiment of the present invention;

[0084] Figure 3 A control situation diagram during a model training process of a decision-making method for intraday forward-looking scheduling of a power grid based on SAC provided in the first embodiment of the present invention;

[0085] Figure 4 A model training result success rate diagram of a decision-making method for intraday forward-looking scheduling of a power grid based on SAC provided in a second embodiment of the present invention;

[0086] Figure 5 A diagram showing the effect of model training for a decision-making method for intraday forward-looking scheduling of a power grid based on SAC provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0087] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0088] Example 1

[0089] Reference Figure 1 to Figure 3 , which is an embodiment of the present invention, provides a decision-making method for intraday forward-looking scheduling of a power grid based on SAC, comprising:

[0090] S1: Collect the predicted section data of the future state of power grid operation as training samples.

[0091] In the power dispatching system, the future state prediction section of the power grid operation is derived every 5 minutes, including the operation sections after 15 minutes, 30 minutes and 4 hours, for a total of 16 future state prediction sections. That is, 16 sections can be obtained every 5 minutes. These 16 sections form a set of training samples in the SAC model training process.

[0092] It should be noted that the company's internal section export tool was used to export the equipment information in the database at that time (equipment such as line table, line endpoint table, main transformer table, bus table, unit table, load table, switch table, circuit breaker table, etc.) (information such as voltage, current, active power, equipment location, topological nodes, etc.) in the form of a file.

[0093] S2: Conduct convergence test on samples, determine and select key sections.

[0094] Set the initial voltage vector V for all computing nodes (0) , choose a small positive number ∈ as the convergence criterion.

[0095] Calculate the power imbalance and for each computing node i, calculate the actual value S of the injected power i and estimated values The difference is expressed as:

[0096]

[0097] in, represents the specified active power of computing node i, represents the specified reactive power of computing node i, and Represents the active and reactive power calculated from the current voltage estimate, expressed as:

[0098]

[0099] Among them, V i represents the voltage at node i, V j represents the voltage at node j, G ij Represents the real part of the admittance matrix between node i and node j, called conductance. ij represents the voltage phase difference between node i and node j, B ij Represents the imaginary part of the admittance matrix between node i and node j, called susceptance.

[0100] Construct the Jacobian matrix J, expressed as:

[0101]

[0102] Among them, ΔP represents the active power imbalance of the node, ΔQ represents the reactive power imbalance of the node, θ represents the phase angle of the node voltage, and |V| represents the amplitude of the node voltage.

[0103] Each element is expressed in the form of:

[0104]

[0105] Solving the linear equation system JΔx = -f(x), we get the voltage increment, expressed as:

[0106]

[0107] Among them, Δθ represents the incremental vector of the voltage phase angle of each node, and Δ|V| represents the incremental vector of the voltage amplitude of each node.

[0108] Update voltage estimate:

[0109] θ (k+1) =θ (k) +Δθ

[0110] |V| (k+1) =|V| (k) +Δ|V|

[0111] Among them, θ (k+1) represents the node voltage phase angle vector at the k+1th iteration, θ (k) represents the node voltage phase angle vector at the kth iteration, |V| (k+1) represents the node voltage amplitude vector at the k+1th iteration, |V| (k) represents the node voltage amplitude vector at the kth iteration, and Δθ represents the voltage phase angle increment vector.

[0112] Check whether the maximum values ​​of ΔP and ΔQ are less than the predetermined convergence standard ∈. If so, the artificial power flow calculation converges, otherwise continue to calculate the power imbalance.

[0113] Calculate the total over-limit value O of the key transmission lines in each section i , expressed as:

[0114]

[0115] Among them, P i represents the active power of the i-th key transmission line, P imax represents the active power exceeding value of the ith key transmission line, and m represents the total number of key transmission lines.

[0116] Find the section D closest to the current time where the critical transmission line exceeds the limit tmin , D tmin = arg min i t i And O i >0, where t i is the time difference of section i.

[0117] Find the section D with the largest total line limit vmax :D vmax = arg max i (Q i ).

[0118] S3: Initialize the SAC deep learning model training state and generate adjustment actions.

[0119] The initialization state S includes the section D tmin And section D vmax , expressed as:

[0120] S={P line ,Q line ,V bus ,P gen}

[0121] The initial state includes the active power P of the line in the section. line 、Line reactive power Q line , bus voltage V bus and the active power P of the unit gen .

[0122] For D tmin The cross section, state S is represented by the following vector:

[0123] S={p line1 ,...,p linen ,q line1 ,...,q linen ,v bus2 ,...,v busn ,p gen1 ,...,plinen}

[0124] Initializing S means setting all variables in it to 0.

[0125] Initialize the experience replay buffer It is expressed as:

[0126]

[0127] Among them, S represents the current state of the environment, A represents the action selected according to the strategy in the current environment, that is, the adjustment amount of each unit in the section. R represents the reward given after the action is selected, S' represents the next power grid environment after the action is executed, and d represents the flag of whether this round is over.

[0128] It should be noted that the state is part of the environment in the deep learning training process (the purpose of deep reinforcement learning is simply to train an intelligent agent that can give an action (the action in this article refers to the active value of each unit in the section) according to the current section. The strategy reduces the over-limit of the key transmission lines in the section. The simple training process: First, the agent will be given an initial strategy (such as randomly adjusting the active output of the adjustable unit). This strategy can be regarded as a function. The input of the function is a vector composed of a set of states S, and the output is the action to be performed. After using the initial state as input, the agent gives an action, and then executes this action in the simulation environment, and the environment changes. When the change reduces the over-limit of the key transmission line, a positive reward is given to the model. If it does not reduce, a negative reward is given. The initial strategy of the agent is updated through this reward, and the training is continuously iterated so that the agent finds a strategy that is applicable to all sections), which contains a description of the section at the current moment. The agent obtains information about the current environment through the state, and uses this information to obtain the action to be performed next.

[0129] Furthermore, when optimizing a strategy, it is necessary to refer to past experience for improvement. The experience replay buffer stores the process that the agent experiences when exploring the environment. Through this buffer, the agent can improve its own strategy (since a deep learning neural network is used to represent the strategy, improving the agent's strategy means improving the neural network weight parameters).

[0130] For the cross-section state S after adjustment t times t , select action a t :a t =π θ (S t ), action a t Expressed as a vector containing the recommended adjustments for all units in the section, π θ (St ) represents the strategy function, according to the current grid state S t Get the corresponding action.

[0131] During the adjustment process, the adjustment rate of the unit is considered to ensure effective adjustment within a limited time. The maximum adjustment rate of the unit is R max , the active power adjustment of the unit is ΔP gen , then the maximum adjustment amount of the unit within time Δt is expressed as:

[0132] ΔPgen,max=Rmax·Δt

[0133] Among them, Δt represents two sections D tmin With D vmax When making adjustments, it is necessary to ensure that each adjustment ΔPgen does not exceed the maximum adjustment calculated above:

[0134] ΔP gen =min(ΔP gen,desire ,ΔP gen,max )

[0135] Where ΔP gen,desire Represents the ideal adjustment calculated by the algorithm based on the current state and reward function.

[0136] For the obtained action a t After executing this action, the power flow calculation is performed to obtain the state of the section in the next stage, and the next state s is obtained through the reward function t+1 and instant rewards t , expressed as:

[0137] S t+1 ,r t =Environment(S t ,a t )

[0138] Among them, S t+1 = {P t+1,line ,Q t+1,line ,V t+1,bus ,P t+1,gen}.

[0139] When the over-limit of the key transmission line in the next section becomes smaller, a positive reward is given to the model. If it does not become smaller, a negative reward is given.

[0140] Calculate the reward value of two key sections, expressed as:

[0141]

[0142]

[0143] in, Indicates D tmin The total line crossing limit of the section, Indicates D vmax The total line exceeding limit of the section, α represents the weight coefficient, reflecting the size of the difference in exceeding limit, and β represents the control system, which is input by the interface.

[0144] The reward function rt is expressed as:

[0145]

[0146] in, At the end of the round, the value is 300, and at the end of the round, the value is The value is 300 at the end of the round, and -0 before the end of the round Dvmax .

[0147] S4: Store experience and update model weights.

[0148] From the Experience Replay Buffer A small batch of samples (S t ,a t ,r t ,s'), calculate the target value y:

[0149]

[0150] Among them, y represents the target Q value, which refers to the expected cumulative reward after the model selects action a in state s, and is used to update the Q function. r represents the immediate reward, which means the reward obtained after taking action a in the current state s. γ represents the discount factor, which is used to weigh the relative importance of future rewards and current rewards. represents the expected value of action a' generated according to strategy π, Q φ' (s',a') represents the target Q function, which represents the Q value obtained by taking action a' in the new state s'. φ' represents the parameters of the Q function, α represents the temperature parameter, which controls the randomness of the strategy, and logπ(a'|s') is the logarithm of the probability that strategy π takes action a' in state s'.

[0151] It should be noted that in reinforcement learning, the Q function is a state-action value function, which is used to estimate the expected cumulative reward of taking a specific action in a specific state. Specifically, the Q function represents the expected value of the total reward obtained in the future after taking action a from state s under strategy π. In the above formula, Q φ' (s',a') is the Q value obtained by taking action a' in state s'.

[0152] Furthermore, the discount factor γ is used to balance the relative importance of current rewards and future rewards, and its range is between 0-1. When γ is 0, the above formula is y=r, that is, y is the immediate reward, which is different from the future reward. When γ=1, the agent attaches equal importance to future rewards and current rewards. This formula is actually the expected value of the calculation formula for future rewards.

[0153] Minimizing the mean square error loss function (this formula is used to measure the mean square error between the predicted value of the Q function and the target value, thereby optimizing the parameter φ in the Q function) is expressed as:

[0154]

[0155] Among them, Q φ (s,a) represents the expected cumulative reward after taking action a in state s under the Q function parameter φ.

[0156] Update strategy network π θ To maximize the objective function:

[0157]

[0158] Among them, π θ represents a parameterized strategy, represents the sum of the cumulative rewards, γ t Represents the discount factor, which is used to weigh the relative importance of future rewards and current rewards. t represents the time step, which means that the reward after each step is decayed by γ times. t ,a t ) represents the immediate reward function, which is in the state s at time step t t Take action a t Instant rewards received.

[0159] Update the value function:

[0160]

[0161] Among them, Q φ (s,a) represents the Q function, which represents the expected cumulative reward of taking action a in state s. φ' (s',a') represents the expected cumulative reward under the next state s' and action a', logπ(a'|s') represents the logarithm of the probability of selecting action a' in state s' under strategy π, and represents the entropy of the strategy.

[0162] If the adjustment step exceeds 50 steps, or any section does not converge during the adjustment process, the adjustment will be stopped.

[0163] If the critical lines in the two critical sections do not exceed the limit after adjustment, the SAC model continues to be trained.

[0164] Through the above steps and formulas, the SAC algorithm adjusts two key sections simultaneously in one round, and ensures the stable operation of the power grid and the effective control of the key section status through the set stop conditions. The introduction of weight coefficients enables the algorithm to dynamically adjust the priority of regulation according to the difference in the section limit.

[0165] It should be noted that all kinds of weight parameters and adjustment parameters in the present invention are obtained through experiments and can be modified manually in actual tests.

[0166] The above embodiments also include a decision-making system for intraday forward-looking scheduling of power grids based on SAC, specifically:

[0167] The sample collection module collects the predicted section data of the future state of power grid operation as training samples.

[0168] Select the critical section module, perform convergence test on the samples, and judge and select the critical section.

[0169] The model training module initializes the SAC deep learning model training state and generates adjustment actions.

[0170] Update model modules, store experience, and update model weights.

[0171] The computer device may be a server. The computer device includes a processor, a memory, an input / output interface (I / O for short) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data cluster data of a power monitoring system. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method is implemented.

[0172] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0173] Example 2

[0174] Reference Figure 4 , Figure 5 , which is an embodiment of the present invention, provides a decision-making method and system for intraday forward-looking scheduling of power grids based on SAC. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through simulation experiments.

[0175] This embodiment demonstrates the configuration and operation process of the decision-making system for intraday forward-looking dispatch of power grids based on the SAC (Soft Actor-Critic) algorithm in practical applications. The system configuration includes setting the task mode to historical mode, the number of sample expansions to 2000 times, the training start date to May 1, 2024, the training end date to May 7, 2024, the number of future state sections to 16, and the distance between sections to 15 minutes. The intelligent control algorithm selects the SAC-based intelligent control algorithm, the evaluation result status is applicable, and the reward function verification status is verification.

[0176] In terms of sample data acquisition, sample data for May 1, 2024 was collected, including detailed information of multiple sections. For example, the JS_20240501_1000_0015.qs section was generated at 10:00, the status time was 10:15, the section type was D_tmin, and the total amount of critical line crossings was 134.68. Similarly, the generation time, status time, section type, and total amount of crossings of other sections are also included.

[0177] The results of the intelligent agent control effect evaluation show that the control effect evaluation success rate is 96%, the comprehensive score is 95, and the detailed score is 91. The evaluation details include 3 occasions, 20 failures, 15 successes, and 20 successes, and the convergence status is yes, no, and yes.

[0178] The system operation process is as follows: First, find the section D with the largest over-limit from the future transmission sections within the day. vmax and the section D at the nearest time when the line exceeds the limit tmin Then, the state s and the experience replay buffer are initialized. Then the SAC model is trained, including strategy selection and execution, interaction with the environment (according to D vma and D tmin ) and model updates (drawing samples from the experience replay buffer, minimizing the mean square error loss function and updating the policy network). Then check D vmax or D tmin Whether the critical line in the section is eliminated, if the conditions are met, it is determined whether all rounds are completed, if not, it returns to the strategy selection and execution step. Finally, if all rounds are completed, the process ends, otherwise, continue to the next round of training.

[0179] The results are as follows Figure 4 , Figure 5 As shown, Figure 4 The horizontal axis is the number of training times, and the vertical axis is the control success rate. Figure 5 The horizontal axis is the number of sections, and the vertical axis is the reward value.

[0180] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A decision-making method for intraday forward-looking scheduling of power grid based on SAC, characterized in that: include: Collect the predicted section data of the future state of power grid operation as training samples; Conduct convergence test on samples, determine and select key sections; Initialize the SAC deep learning model training state and generate adjustment actions; Store experience and update model weights.

2. The decision-making method for intraday forward-looking scheduling of power grid based on SAC according to claim 1, characterized in that: The collected future state prediction section data of the power grid operation as training samples include, in the power dispatching system, deriving the future state prediction section of the power grid operation every 5 minutes, obtaining 16 sections every 5 minutes, and forming these 16 sections into a group of training samples in the SAC model training process.

3. The decision-making method for intraday forward-looking scheduling of power grid based on SAC as claimed in claim 2, characterized in that: The convergence test of the sample includes setting the initial voltage vector V for all computing nodes. (0) , choose a small positive number ∈ as the convergence criterion; Calculate the power imbalance and for each computing node i, calculate the actual value S of the injected power i and estimated values The difference is expressed as: in, represents the specified active power of computing node i, represents the specified reactive power of computing node i, and Represents the active and reactive power calculated from the current voltage estimate, expressed as: Among them, V i represents the voltage at node i, V j represents the voltage at node j, G ij represents the real part of the admittance matrix between node i and node j, called conductance; θ ij represents the voltage phase difference between node i and node j, B ij Represents the imaginary part of the admittance matrix between node i and node j, called susceptance; Construct the Jacobian matrix J, expressed as: Where ΔP represents the active power imbalance of the node, ΔQ represents the reactive power imbalance of the node, θ represents the phase angle of the node voltage, and |V| represents the amplitude of the node voltage; Each element is expressed in the form of: Solving the linear equation system JΔx = -f(x), we get the voltage increment, expressed as: Among them, Δθ represents the incremental vector of the voltage phase angle of each node, and Δ|V| represents the incremental vector of the voltage amplitude of each node; Update voltage estimate: i (k+1) =θ (k) +Δθ |V| (k+1) =|V| (k) +Δ|V| Among them, θ (k+1) represents the node voltage phase angle vector at the k+1th iteration, θ (k) represents the node voltage phase angle vector at the kth iteration, |V| (k+1) represents the node voltage amplitude vector at the k+1th iteration, |V| (k) represents the node voltage amplitude vector at the kth iteration, Δθ represents the voltage phase angle increment vector; Check whether the maximum values ​​of ΔP and ΔQ are less than the predetermined convergence standard ∈. If so, the artificial power flow calculation converges, otherwise continue to calculate the power imbalance.

4. The decision-making method for intraday forward-looking scheduling of power grid based on SAC as claimed in claim 3, characterized in that: The determination and selection of the key sections includes calculating the total over-limit value of the key transmission lines in each section. i , expressed as: Among them, P i represents the active power of the i-th key transmission line, P imax represents the active power over-value of the i-th key transmission line, and m represents the total number of key transmission lines; Find the section D closest to the current time where the critical transmission line exceeds the limit tmin , D tmin = arg min i t i And O i >0, where t i is the time difference of section i; Find the section D with the largest total line limit vmax :D vmax = arg max i (Q i ).

5. The decision-making method for intraday forward-looking scheduling of power grid based on SAC as claimed in claim 4, characterized in that: The initialization SAC deep learning agent training state includes: the initialization state S includes the section D tmin And section D vmax , expressed as: S={P line ,Q line ,V bus ,P gen } The initial state includes the active power P of the line in the section. line 、Line reactive power Q line , bus voltage V bus and the active power P of the unit gen ; For D tmin The cross section, state S is represented by the following vector: S={p line1 ,...,p linen ,q line1 ,...,q linen ,v bus2 ,...,v busn ,p gen1 ,...,p linen } Initialize S to set all variables in it to 0; Initialize the experience replay buffer It is expressed as: Among them, S represents the state of the current environment, A represents the action selected according to the strategy in the current environment, that is, the adjustment amount of each unit in the section; R represents the reward given after the action is selected, S' represents the next power grid environment after the action is executed, and d represents the flag of whether this round is over; For the cross-section state S after adjustment t times t , select action a t :a t =π θ (S t ), action a t Expressed as a vector containing the recommended adjustments for all units in the section, π θ (S t ) represents the strategy function, according to the current grid state S t Get the corresponding action; During the adjustment process, the adjustment rate of the unit is considered to ensure effective adjustment within a limited time; the maximum adjustment rate of the unit is R max , the active power adjustment of the unit is ΔP gen , then the maximum adjustment amount of the unit within time Δt is expressed as: ΔPgen,max=Rmax·Δt Among them, Δt represents two sections D tmin With D vmax When making adjustments, it is necessary to ensure that each adjustment ΔPgen does not exceed the maximum adjustment calculated above: ΔP gen =min(ΔP gen,desire ,ΔP gen,max ) Among them, ΔP gen,desire Represents the ideal adjustment calculated by the algorithm based on the current state and reward function.

6. The decision-making method for intraday forward-looking scheduling of power grid based on SAC as claimed in claim 5, characterized in that: The generating adjustment action includes: for the obtained action a t After executing this action, the power flow calculation is performed to obtain the state of the section in the next stage, and the next state s is obtained through the reward function t+1 and instant rewards t , expressed as: S t+1 ,r t =Environment(S t ,a t ) Among them, S t+1 = {P t+1,line ,Q t+1,line ,V t+1,bus ,P t+1,gen }; When the over-limit of the key transmission line in the next section decreases, the model is given a positive reward, and if it does not decrease, a negative reward is given; Calculate the reward value of two key sections, expressed as: ΔO=|O Dtmin -Oh Dvmax | in, Indicates D tmin The total line crossing limit of the section, Indicates D vmax The total line crossing amount of the section, α represents the weight coefficient, reflecting the size of the crossing amount difference, β represents the control system, which is input by the interface; The reward function rt is expressed as: in, The value is 300 at the end of the round, and -0 before the end of the round Dtmin ; The value is 300 at the end of the round, and -0 before the end of the round Dvmax .

7. The decision-making method for intraday forward-looking scheduling of power grid based on SAC as claimed in claim 6, characterized in that: The storing of experience and updating of model weights includes replaying the experience buffer from the experience A small batch of samples (S t ,a t ,r t ,s'), calculate the target value y: Among them, y represents the target Q value, which refers to the expected cumulative reward after the model selects action a in state s, and is used to update the Q function; r represents the immediate reward, which means the reward obtained after taking action a in the current state s; γ represents the discount factor, which is used to weigh the relative importance of future rewards and current rewards; represents the expected value of action a' generated according to strategy π, Q φ' (s',a') represents the target Q function, which indicates the Q value obtained by taking action a' in the new state s'; φ' represents the parameter of the Q function, α represents the temperature parameter, which controls the randomness of the strategy, and logπ(a'|s') is the logarithm of the probability that strategy π takes action a' in state s'; The loss function of minimizing the mean square error is expressed as: Among them, Q φ (s,a) represents the expected cumulative reward after taking action a in state s under the Q function parameter φ; Update strategy network π θ To maximize the objective function: Among them, π θ represents a parameterized strategy, represents the sum of the cumulative rewards, γ t represents the discount factor, which is used to weigh the relative importance of future rewards and current rewards; t represents the time step, which means that the reward after each step has a γ-fold decay; R(s t ,a t ) represents the immediate reward function, which is in the state s at time step t t Take action a t The immediate rewards received; Update the value function: Among them, Q φ (s,a) represents the Q function, which represents the expected cumulative reward of taking action a in state s; Q φ' (s',a') represents the expected cumulative reward under the next state s' and action a', logπ(a'|s') represents the logarithm of the probability of selecting action a' in state s' under strategy π, and represents the entropy of the strategy.

8. A decision-making system for intraday forward-looking scheduling of a power grid based on SAC using the method according to any one of claims 1 to 7, characterized in that: The sample collection module collects the predicted section data of the future state of the power grid operation as training samples; Select the key section module, conduct convergence test on the samples, and judge and select the key sections; Model training module, initializes the SAC deep learning model training state and generates adjustment actions; Update model modules, store experience, and update model weights.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.