Optimization method and device of power system, computer program product and optimization system

Abstract

The invention provides an optimization method and device of an electric power system, a computer program product and an optimization system.The method comprises the steps that off-line related data of the electric power system are obtained, and the off-line related data at least comprise one or more of current, voltage, power and phase difference; a Deep Q-network algorithm is adopted, and data features of the off-line related data are extracted; a supervised learning algorithm is adopted, and optimization control is carried out based on the data features of the off-line related data. According to the scheme, the problem that in the prior art, a traditional reactive power optimization method is poor in applicability in a complex actual power system is solved.

Description

Technical Field

[0001] The present application relates to the field of deep learning, and more specifically, to an optimization method, device, computer program product, and optimization system for a power system. Background Art

[0002] Reactive power optimization can alleviate the voltage drop problem of the receiving power grid by controlling the reactive compensation equipment in the power grid and correcting the power flow distribution of the power system, thereby achieving economic and safe operation of the power system. Reactive power optimization can be modeled as a constrained nonlinear multi-objective optimization problem. Traditional reactive power optimization methods based on operations research and optimization theory include gradient method, Newton algorithm, quadratic programming algorithm and linear programming algorithm. These traditional algorithms have strong convergence guarantees and relatively complete theoretical support. However, with the increasing complexity of the power system structure, the operating state of the power system has become more changeable, which hinders the application of traditional algorithms in complex actual power systems. Summary of the invention

[0003] The main purpose of the present application is to provide an optimization method, device, computer program product and optimization system for an electric power system, so as to at least solve the problem that the traditional reactive power optimization method in the prior art has poor applicability in complex actual electric power systems.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, there is provided a method for optimizing an electric power system, comprising: obtaining offline related data of the electric power system, wherein the offline related data includes at least one or more of current, voltage, power, and phase difference; using a Deep Q-network algorithm to extract data features of the offline related data; and using a supervised learning algorithm to perform optimization control based on the data features of the offline related data.

[0005] Optionally, a Deep Q-network algorithm is used to extract data features of the offline related data, including: constructing a feature recognition model, wherein the feature recognition model is trained by a Deep Q-network algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes historical offline related data acquired within a historical time period and historical data features corresponding to the historical offline related data; inputting the offline related data into the feature recognition model to obtain the data features corresponding to the offline related data.

[0006] Optionally, before constructing the feature recognition model, the method also includes: constructing a first network layer, wherein the first network layer is a Transformer layer, and the first network layer is used to extract state data features, wherein the state data features are electrical characteristic quantities during the steady-state operation of the power system, and the state data features include one or more of node active power, reactive power injection, and node voltage amplitude; constructing a second network layer, wherein the second network layer is a first fully connected layer, and the second network layer is used to extract action data features, wherein the action data features are reactive compensation amounts of reactive control devices that have been feature-encoded; constructing a third network layer, wherein the third network layer is a second fully connected layer, and the third network layer is used to extract a first value of the state data features and a second value of the action data features.

[0007] Optionally, before constructing the second network layer, the method further includes: obtaining initial action data, wherein the initial action data is data with a change amount in the offline related data; and preprocessing the initial action data using a one-hot encoding method to obtain action data.

[0008] Optionally, before constructing the feature recognition model, the method also includes: constructing a main network layer, wherein the main network layer is constructed based on the Deep Q-network algorithm, and the main network layer is used to calculate the expected return value corresponding to the input state data and action data. In each training step, the parameters of the main network layer will be updated according to the samples in the experience playback; constructing a target network layer, wherein the target network layer is constructed based on the Deep Q-network algorithm, and the target network layer is used to calculate the state-action value of the target.

[0009] Optionally, a supervised learning algorithm is used to perform optimal control based on the data features of the offline related data, including: constructing a gain identification model, wherein the gain identification model is trained by a supervised learning algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes historical data features acquired within a historical time period, and historical gains corresponding to the historical data features; inputting the data features into the gain identification model to obtain the gains corresponding to the data features; and using the gains and reward functions to perform optimal control on the power system.

[0010] Optionally, the reward function is:

[0011]

[0012] r represents the reward function, P loss represents the total active power loss of the power system, ∑Δv 2represents the overall voltage deviation of the system bus, ∑|q input | represents the total injected reactive power, α represents the first weight coefficient, β represents the second weight coefficient, and λ represents the third weight coefficient.

[0013] According to another aspect of the present application, there is provided an optimization device for an electric power system, comprising: a first acquisition unit, for acquiring offline related data of the electric power system, wherein the offline related data comprises at least one or more of current, voltage, power, and phase difference; an extraction unit, for extracting data features of the offline related data by using a Deep Q-network algorithm; and an optimization unit, for performing optimization control based on the data features of the offline related data by using a supervised learning algorithm.

[0014] According to another aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of any one of the power system optimization methods.

[0015] According to another aspect of the present application, an optimization system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the optimization methods of the power system described.

[0016] By applying the technical solution of the present application, the data obtained are all samples collected offline and do not interact with the real online power system, so security can be guaranteed. The Deep Q-network algorithm is a deep learning and reinforcement learning algorithm that can learn the characteristics of samples and has efficient feature extraction capabilities. The supervised learning algorithm can predict future power demand and supply based on historical data and real-time input, thereby optimizing the operation of the power system, reducing the operating cost of the power system, improving the stability of the system, achieving efficient decision-making and control, and better adapting to the power system, and can better improve the applicability of reactive power optimization of the power system under various operating conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings constituting part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for optimizing a power system provided in an embodiment of the present application is shown;

[0019] Figure 2A schematic diagram of a flow chart of a method for optimizing a power system provided according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of a reactive power optimization framework based on transfer reinforcement learning is shown;

[0021] Figure 4 A schematic diagram showing the state visualization of the input;

[0022] Figure 5 A structural block diagram of an optimization device for a power system provided according to an embodiment of the present application is shown.

[0023] The above drawings include the following reference numerals:

[0024] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0025] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] For the convenience of description, some nouns or terms involved in the embodiments of the present application are explained below:

[0029] Deep Q-Network (DQN) is a method that combines deep learning and reinforcement learning. It aims to solve reinforcement learning problems with large action spaces. It uses deep neural networks to approximate value functions, so that it can handle complex decision-making problems with high-dimensional state spaces. The core idea of ​​the DQN algorithm is to use the Q learning framework and improve the stability and efficiency of learning through experience replay and target network technology.

[0030] Reinforcement Learning (RL) is a machine learning method whose core idea is to learn how to take actions to maximize cumulative rewards through the interaction between the agent and the environment. In reinforcement learning, the agent selects actions based on the state of the environment and updates its strategy based on the feedback (reward or punishment) from the environment. This learning method emphasizes the trial and error process, and the agent needs to find a balance between exploring the unknown environment and using known information to achieve the learning goal.

[0031] As introduced in the background technology, the traditional reactive power optimization method in the prior art has poor applicability in complex actual power systems. To solve the above problems, the embodiments of the present application provide a power system optimization method, device, computer program product and optimization system.

[0032] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0033] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a power system optimization method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0034] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0035] In this embodiment, a method for optimizing an electric power system running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] Figure 2 FIG. 1 is a flow chart of a method for optimizing a power system according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0037] Step S201, acquiring offline related data of the power system, wherein the offline related data at least includes one or more of current, voltage, power, and phase difference;

[0038] Specifically, offline relevant data of the power system are obtained, which include at least one or more of current, voltage, power, phase difference, etc. Offline relevant data refers to data recorded offline during the operation of the power system, which is used to analyze, monitor and optimize the operation of the power system.

[0039] Current is the amount of charge flowing through an electrical circuit, usually expressed in amperes (A); voltage is the potential difference between the two ends of a circuit, usually expressed in volts (V); power is the rate of energy conversion in a circuit, usually expressed in watts (W); phase difference is the time difference between waveforms in an AC circuit, usually expressed in degrees or radians.

[0040] Step S202, using the Deep Q-network algorithm to extract data features of the above offline related data;

[0041] Specifically, the Deep Q-network (DQN) algorithm is used to extract data features from previously collected offline related data. The DQN algorithm is a deep reinforcement learning algorithm that is usually used to train agents to learn optimal strategies in an environment. In this case, the DQN algorithm will be used to analyze and extract features from offline related data, which can contain information about the environment, actions, and rewards.

[0042] Step S203: using a supervised learning algorithm to perform optimization control based on the above data features of the above offline related data.

[0043] Specifically, a supervised learning algorithm is used to optimize control using previously collected offline related data and data features. A supervised learning algorithm is a method of learning a prediction model through labeled training data, while offline related data refers to task-related data acquired in advance. Through these data features, the system can be optimized and controlled to improve the performance and efficiency of the system.

[0044] Through this embodiment, the data obtained are all samples collected offline and do not interact with the real online power system, so safety can be guaranteed. The Deep Q-network algorithm is a deep learning and reinforcement learning algorithm that can learn the characteristics of samples and has efficient feature extraction capabilities. The supervised learning algorithm can predict future power demand and supply based on historical data and real-time input, thereby optimizing the operation of the power system, reducing the operating cost of the power system, improving the stability of the system, achieving efficient decision-making and control, and better adapting to the power system, and can better improve the applicability of reactive power optimization of the power system under various operating conditions.

[0045] As the operation modes of power systems become increasingly complex and changeable, the rapid solution of reactive power optimization and the migration and application of reactive power optimization strategies under different conditions have become new challenges. Transfer reinforcement learning is a branch of reinforcement learning that aims to improve the performance of reinforcement learning agents by transferring knowledge from other sources (such as previous tasks, expert demonstrations, or auxiliary information). Transfer reinforcement learning can help agents learn knowledge of new environments faster, better, and more robustly in new and complex environments. One of the problems faced by transfer reinforcement learning is sample efficiency, which is particularly important for practical applications such as power systems and robotics because the number of samples available for training is limited.

[0046] Unlike traditional model-based methods, reinforcement learning is an end-to-end method that does not require a large number of optimization iteration steps in online applications. Considering the flexibility and excellent performance in complex decision-making scenarios, reinforcement learning and its variant deep reinforcement learning have been widely studied in reactive power optimization. However, with the high proportion of new energy and electronic equipment fed into the power system, the operating state of the power system has become more complex and changeable, and the adaptability of reinforcement learning agents based on limited sample training is poor. Transfer reinforcement learning can alleviate the mismatch between different operating scenarios, and there are also some related studies on this. However, existing transfer learning research still assumes that the agent can interact with the target environment, which reduces the safety of the proposed method. At the same time, the existing transfer learning sample utilization efficiency is low, which also limits the efficiency of knowledge transfer. To solve the above problems, this scheme proposes a reactive power optimization method based on deep transfer reinforcement learning that integrates efficient state feature extraction.

[0047] This scheme proposes a reactive power optimization method based on deep transfer reinforcement learning with efficient state feature extraction. Figure 3 As shown in the figure, specifically: First, a deep Q-network (DQN) with an efficient state feature network structure is used on the source system to train a reinforcement learning (RL) agent, which can extract general power system state features from the state variables of the source system. This knowledge can be easily transferred to the learning of other scenarios. Secondly, a limited data set is used on the target system to train a reward predictor through supervised learning. The predictor uses the state features learned from the source system and can well learn the dynamic characteristics of the target system. Finally, the migrated reward predictor is used to guide the RL agent to perform reactive power optimization control on the target system. This scheme can effectively transfer and utilize knowledge in reactive power voltage optimization, and further improve the adaptability of reactive power optimization of power systems under various operating conditions.

[0048] Specifically, first, a general power system state representation is learned on the source system, and the learned features can effectively reflect the reactive power loss characteristics of the power system; secondly, in the second stage, considering the limited number of samples of the target system, in order to improve the sample efficiency, combined with the state representation learned in the first stage, supervised learning is used to train the reward predictor in this stage. Both models can be trained based on the pytorch environment. Through actual case verification, it can be found that the training of the two processes can converge.

[0049] In the specific implementation process, the Deep Q-network algorithm is used to extract the data features of the above-mentioned offline related data, which can be achieved through the following steps: constructing a feature recognition model, wherein the above-mentioned feature recognition model is trained by the Deep Q-network algorithm using multiple sets of training data, and each set of training data in the above-mentioned multiple sets of training data includes historical offline related data obtained within a historical time period and historical data features corresponding to the above-mentioned historical offline related data; the above-mentioned offline related data is input into the above-mentioned feature recognition model to obtain the above-mentioned data features corresponding to the above-mentioned offline related data.

[0050] Specifically, this scheme performs Markov modeling of reactive optimization. In deep reinforcement learning, the agent is replaced by a neural network, and the term environment is used to refer to the part that interacts with the agent. The interaction between the agent and the environment can be described by three elements: state s, action a, and reward r obtained at time t. t Now define s, a and r for reactive power optimization t .

[0051] Specifically, the states and actions are defined as follows: Considering the form of reactive power optimization, the node voltage amplitude V, the active and reactive power injection amounts P, Q, and the transmission line active power and reactive power P are selected. l , Q l As the input variable of the agent. Since both reactive power shortage and corresponding reactive power support have strong spatial characteristics, the above input information is converted into a two-dimensional tensor. Finally, the agent input for reactive power optimization can be quantified into the following three-dimensional tensor:

[0052] O(t)=[V(t),P(t),Q(t),P l (t),Q l (t)]

[0053] V(t)=diag(v(t))

[0054] P(t)=diag(p(t))

[0055] Q(t)=diag(q(t))

[0056] P l (t)=(p ij (t))

[0057] Q l (t)=(q ij (t)),

[0058] Where v(t), p(t) and q(t) represent the bus voltage amplitude, active power and reactive power vector at time t respectively. The active power and reactive power from bus i to bus j are represented by p and q respectively. ij (t) and q ij (t) represents. In particular, the input is dimensionally normalized to ensure consistency. An example of the input visualization is shown in Figure 4 shown.

[0059] Specifically, the action space A refers to all possible control options in reactive power optimization, including switching capacitors or reactors, setting transformer ratios, setting generator voltages, etc. To simplify the analysis, only the reactive power injection amount ΔQ is adjusted by adjusting the capacitors or reactors connected to certain buses. input (t),

[0060] Among them, ΔQ input (t) = [ΔQ input,i (t),...,Δq input,na ], ΔQ input,i (t) represents the reactive injection change of the ith reactive injection device at time t, and na represents the number of reactive control devices. Although a simple control device is used as an example, the proposed framework can be extended to more complex situations.

[0061] In this solution, the Deep Q-network algorithm is used to train the model during the process of building a feature recognition model. By inputting offline related data into the feature recognition model, the features corresponding to these data can be obtained. By building a feature recognition model, the features of the data can be identified quickly and accurately, further improving the efficiency of data processing.

[0062] In some embodiments, before constructing the feature recognition model, the above method also includes the following steps: constructing a first network layer, wherein the first network layer is a Transformer layer, and the first network layer is used to extract state data features, and the state data features are electrical characteristic quantities during the steady-state operation of the power system, and the state data features include one or more of node active power, reactive power injection, and node voltage amplitude; constructing a second network layer, wherein the second network layer is a first fully connected layer, and the second network layer is used to extract action data features, and the action data features are reactive compensation amounts of reactive control devices that have been feature-encoded; constructing a third network layer, wherein the third network layer is a second fully connected layer, and the third network layer is used to extract a first value of the state data features and a second value of the action data features.

[0063] In this solution, a neural network structure is provided, which includes two network layers: the first layer is the Transformer layer, which is used to extract the features of the state data; the second layer is the first fully connected layer, which is used to extract the features of the action data. The Transformer layer performs well in processing sequence data and can better capture the relationship between data. The fully connected layer can combine and fuse different features, which helps to improve the expressiveness and generalization ability of the model. Through such a deep network structure, the features of the state data and action data can be effectively extracted, which is conducive to the network learning more effective representations.

[0064] Specifically, the following introduces the network structure that integrates universal knowledge feature extraction. Considering the low voltage problem and the local characteristics of reactive power support, a Q network based on Transformer neural network is designed to cope with the spatial characteristics of the power system. Different from the traditional Q network structure, the solution of this application proposes a novel deep Q network structure to independently extract state and action features. Assuming that reactive power ΔQ is injected at bus i i Considering that the topology of the power system remains unchanged, reactive power optimization is mostly near the system operating point (P, Q) represented by the injected power space. After relaxing the constraints, the voltage change has an approximate linear relationship with the reactive power injection amount. However, when the topology of the power system changes, the reactive power injection amount is increased by ΔQ. i The voltage change caused by is no longer similar to the above. Therefore, it is reasonable to extract the features of action and state separately. The designed network structure is as follows Figure 3The proposed network structure consists of three different neural network modules: the state feature extraction module consists of a Transformer neural network for extracting the local characteristics of reactive voltage in the power system, the action feature extraction module consists of a fully connected neural network for mining action information, and the value module consists of a fully connected neural network for combining the processed state and action information to obtain the action value function estimate, whose parameters are denoted as α, β and θ respectively.

[0065] Specifically, the proposed network can extract state and action features independently. Since reactive power shortage has obvious local characteristics and has similarities at different operating levels, the Transformer neural network with two-dimensional power system operation information as input can effectively extract reactive power shortage under different operating states. In terms of action, the same action will have different influences and effects in systems with different operating modes, but it has similar performance at different operating points under the same operating mode, which is also consistent with the cognition of reactive power optimization problems. Extracting state features and action features separately is conducive to allowing the agent to obtain more comprehensive reactive optimization knowledge during the training process. Extracting state features and action features separately can enhance the generalization ability under different reactive optimization states. In addition, this scheme solves the correlation problem in the estimation of advantage functions of different actions by NN-based estimators. Note that the backpropagation process of a sample changes all parameters of the network, which in turn affects the estimation results of the advantage function for other actions. Therefore, in knowledge transfer, the accuracy of the estimation results will decrease with the limited number of samples. In order to overcome the above problems, this scheme uses a one-hot encoding module to preprocess the input actions. This approach reduces the correlation between different actions and weakens the impact of backpropagation of a single sample on other evaluation results during training. Specifically, this move makes the action space more sparse, which in turn makes the number of parameters affected by backpropagation of a single sample during training less. This is more obvious when the activation function is ReLU. Therefore, this scheme achieves more efficient training of the advantage function with limited samples.

[0066] In the specific implementation process, before constructing the second network layer, the above method also includes the following steps: obtaining initial action data, wherein the above initial action data is data with a change amount in the above offline related data; using one-Hot encoding to preprocess the above initial action data to obtain action data.

[0067] In this scheme, in knowledge transfer, as the number of samples is limited, the accuracy of the estimation results will decrease, and the input actions are preprocessed using one-hot encoding. This method reduces the correlation between different actions and weakens the impact of the back propagation of a single sample on other evaluation results during training.

[0068] Specifically, in order to achieve fast and efficient migration and update of reactive power optimization strategies between power systems under different operating modes, this scheme proposes a reactive power optimization framework based on transfer reinforcement learning, which has high feature mining and knowledge transfer efficiency. The overview of the proposed framework is as follows: Figure 3 As shown. The knowledge transfer proposed in the scheme of the present application is based on the power system state representation learned by the Transformer neural network, and can be divided into two stages. Specifically, in the first stage, a general power system state representation is learned on the source system, and the learned features can effectively reflect the reactive power loss characteristics of the power system. In the second stage, considering the limited number of samples of the target system, in order to improve the sample efficiency, combined with the state representation learned in the first stage, supervised learning is used to train the reward predictor in this stage. Since the training process is only based on samples collected offline and does not interact with the real power system, the proposed framework can ensure the safety of the target system. At the same time, supervised learning improves the utilization efficiency of samples, and further improves the efficiency of knowledge transfer and utilization.

[0069] In some embodiments, before constructing the feature recognition model, the above method also includes the following steps: constructing a main network layer, wherein the main network layer is constructed based on the Deep Q-network algorithm, and the main network layer is used to calculate the expected return value corresponding to the input state data and action data. In each training step, the parameters of the main network layer will be updated according to the samples in the experience playback; constructing a target network layer, wherein the target network layer is constructed based on the Deep Q-network algorithm, and the target network layer is used to calculate the state-action value of the target.

[0070] In this scheme, the training process can be stabilized through the network layer constructed above. The parameters of the target network layer are not updated at every step, but the parameters of the main network layer are copied to the target network at a fixed number of steps. Since the parameters of the target network remain unchanged for a period of time, it provides a relatively stable target value and avoids the instability of the training process.

[0071] Specifically, this scheme implements knowledge transfer based on state representation. First, the agent learns general reactive state characteristics from the source system through state representation. At this stage, the agent interacts with the source system to learn the general characteristics of the reactive power optimization state variables of the power system. Considering the feasibility and stability of the training process, we adopt the DQN method in the training process of state feature extraction. The algorithm uses two Q networks to stabilize the learning process. These two networks are called the main network Q and the target network Q′. The latter is used to calculate the target Q value. The parameters of the two networks are recorded as α, β, θ and α′, β′, θ′ respectively. At regular intervals, the weights of the main network are copied to the target network. The loss function of the main network is defined as the Q-value estimate Q(s,a;θ,α,β) and the target value Q tar The squared difference between:

[0072]

[0073] Specifically, s and s′ represent the current state and the next state respectively, and L(θ, α, β) represents the loss function. The above loss function can be minimized using gradient descent or its variant method. When training the source system, the Transformer neural network can effectively learn the correlation between the power system state and the reactive power shortage, that is, the appropriate state representation.

[0074] Specifically, in practical applications, unique dynamics are learned through the data of the target power system. Since the agent can only access limited data of the target power system and cannot interact with the target system directly, we need an efficient policy learning algorithm to learn the dynamics of the target power system from offline data (i.e., data obtained by interacting with the environment through other strategies). Considering the high sample efficiency of supervised learning methods, the action reward predictor is trained through supervised learning. Considering the knowledge transfer of the state feature part, the same deep Q network structure as the first stage is adopted. The reward predictor takes the state of the power system and the reactive power injection action a as input, and the output is the estimated expected gain G with discount expred :

[0075]

[0076] Where α1 represents the learning parameters of the proposed reward predictor, while θ and β represent the fixed parameters of the Q network model learned from the source system. r represents the reward for reactive power injection. The weights of the state feature extraction module of the reward predictor applied to the target system are initialized to the parameters of the feature extraction module from the Q function trained by the source system and are set to fixed values ​​to prevent overfitting on limited training samples. The above process realizes the knowledge transfer and utilization of feature extraction between power systems in different operating modes. The loss function L of the reward predictor is GIt reflects the error between the estimated expected gain and the actual gain of the state-action sample of the target system and is defined as follows:

[0077]

[0078] Specifically, D T represents the data collected from the target power system, and s r 、a r , s′ r 、r r Respectively represent from D T The sampled state, action, next state, and reward. The proposed reward predictor is trained via supervised learning to minimize the loss function L G .

[0079] Since the knowledge from both the source and target systems is exploited in the reward predictor, where the state feature extraction is trained on the source system and the dynamics of the target power system is trained based on data from the target system, the proposed scheme has better generalization ability compared to the agent trained only on the target system. In addition, since only the action features need to be trained, we achieve more efficient knowledge transfer and utilization, shortening the time it takes for the agent to adapt to the target system's operating mode.

[0080] Considering the knowledge transfer between different reactive power optimization scenarios, the intelligent agent first learns the feature mining capabilities in a wide range of reactive power optimization scenarios from the source system. Then, when the system operation state changes, a small amount of measured data can be used to achieve rapid supervised learning training and intelligent agent performance improvement without retraining the intelligent agent from scratch. This migration method greatly accelerates the learning ability of the intelligent agent in the actual environment. Combining the ideas of state feature mining and action characteristic specialized training, the proposed reactive power optimization scheme based on transfer learning can more efficiently realize the transfer and utilization of knowledge between different operation modes.

[0081] In the specific implementation process, a supervised learning algorithm is used to perform optimization control based on the above data features of the above offline related data, which can be achieved through the following steps: constructing a gain identification model, wherein the above gain identification model is trained by a supervised learning algorithm using multiple sets of training data, and each set of training data in the multiple sets of training data includes historical data features obtained within a historical time period and historical gains corresponding to the above historical data features; inputting the above data features into the above gain identification model to obtain the gains corresponding to the above data features; and using the above gains and reward functions to optimize the control of the above power system.

[0082] In this scheme, a gain identification model is constructed using a supervised learning algorithm, and the model is applied to power system optimization control. This can more accurately identify and predict the gain of the power system, thereby optimizing the control strategy and improving the efficiency and performance of the system.

[0083] Specifically, reactive power optimization is the process of adjusting reactive power flows in an AC power system while satisfying operational and safety constraints. Reactive power optimization can improve voltage distribution, reduce transmission losses, and enhance system stability. The formula for reactive power optimization is:

[0084]

[0085] where x is the vector of reactive power control variables, such as generator terminal voltage, transformer tap settings, and reactive power compensator switching; f i (x)(i=1,2,...,n) is the objective function, which respectively represents the active power loss of the transmission line, the voltage deviation and the switching cost of the reactive power compensation device; c i is the coefficient of the objective function, reflecting the weight of different objective functions; n is the number of objective functions; g(x) and h(x) are the power system operation constraints in the form of equations and inequalities, respectively. The former includes the power flow equation, and the latter includes voltage limits and safety constraints. Reactive power optimization is a nonlinear, non-convex constrained optimization problem. Traditional reactive power optimization solutions include gradient method, Newton method, interior point method, evolutionary algorithm, etc.

[0086] Specifically, supervised learning algorithms are used to optimize the control of power systems. First, it is necessary to obtain offline relevant data of the power system, which includes parameters such as current, voltage, power, phase difference, etc. Then, using these offline relevant data as input, supervised learning algorithms are used to learn and optimize the control strategy of the power system.

[0087] In the supervised learning algorithm, the control strategy is adjusted according to the current state and the reward value of the environment feedback to achieve the best control effect. Specifically, according to the characteristics of offline related data, appropriate actions are selected to optimize the operating state of the power system, such as adjusting the parameter settings of the power system, switch control, etc.

[0088] For example, if you need to optimize the power output of a power system, you can use a supervised learning algorithm to learn a control strategy based on historical data. When the power system is in a certain state, it will select appropriate actions to adjust the operating state of the power system based on the current offline related data, such as adjusting the output power of the generator or changing the connection method of the power grid. Through continuous learning and adjustment, the power output of the power system can be gradually optimized to achieve the best state.

[0089] In summary, the use of supervised learning algorithms to optimize the control of power systems can achieve optimal adjustment of the operating status of the power system based on the characteristics of offline related data and environmental feedback, thereby improving the efficiency and performance of the power system.

[0090] A gain identification model is trained using a supervised learning algorithm, which can predict future gains based on historical data features. Training this model requires multiple sets of training data, each of which includes historical data features and corresponding historical gains within a historical time period. Once the model is trained, new data features can be input into the model to obtain the corresponding gain. Then, this gain and a reward function are used to optimize the control of the power system. Specifically, the parameters and control strategies of the power system can be continuously adjusted to maximize the system gain and meet certain constraints. The reward function can be defined based on the performance indicators of the system, such as power efficiency, cost, etc.

[0091] For example, there is a power system where the goal is to maximize the power output of the system. Use the above trained gain recognition model to predict future power gains. Based on this predicted value and the reward function, the parameters of the power system, such as the output power of the generator, can be adjusted to maximize the power output of the system.

[0092] Specifically, if the gain identification model predicts that the future power gain will be high, the generator output power can be increased; if the gain is predicted to be low, the generator output power can be reduced. By continuously adjusting the parameters, the system can achieve the optimal power output under different circumstances.

[0093] In some embodiments, the above reward function is:

[0094]

[0095] r represents the above reward function, P loss represents the total active power loss of the above power system, ΣΔ v 2 represents the overall system bus voltage deviation, Σ| qinput | represents the total injected reactive power, α represents the first weight coefficient, β represents the second weight coefficient, and λ represents the third weight coefficient.

[0096] In this scheme, the reward function is intended to indicate the effectiveness of the control action. Specifically for reactive power optimization problems, the reward should take into account the impact of power loss, voltage deviation, and rescheduling cost. The reward function can motivate the optimization algorithm to continuously optimize and eventually converge to the optimal solution, so as to flexibly control the search direction and speed of the optimization algorithm to make it more in line with actual needs.

[0097] Specifically, this scheme combines the ideas of state feature mining and action characteristic special training. The proposed reactive power optimization scheme based on transfer learning can more efficiently realize the transfer and utilization of knowledge between different operating modes. After adding state feature mining and action characteristic special training, the specific process is as follows: learn a general power system state representation on the source system, and the learned features can effectively reflect the reactive power loss characteristics of the power system; in the second stage, considering the limited number of samples of the target system, in order to improve the sample efficiency, combined with the state representation learned in the first stage, supervised learning is used to train the reward predictor in this stage; finally, this reactive power optimization model is applied to the online scenario.

[0098] Specifically, the characteristics and beneficial effects of this scheme are: since the training process is only based on samples collected offline and does not interact with the real power system, the proposed framework can ensure the safety of the target system. Supervised learning improves the utilization efficiency of samples and further improves the efficiency of knowledge transfer and utilization. The main features of this scheme are: through the method of transfer reinforcement learning, efficient transmission of reactive power optimization knowledge between different systems is achieved. Compared with traditional methods, this method has higher sampling efficiency and can better adapt to the target system; transfer learning is achieved by combining the original model with a small amount of data from the target system through offline supervised learning, without the need for online interaction with the target system, which enhances the safety of the training process; a deep Q network structure suitable for extracting reactive power optimization features is proposed, which decouples the state and action of reactive power optimization and has efficient state feature extraction capabilities.

[0099] The embodiment of the present application also provides an optimization device for an electric power system. It should be noted that the optimization device for an electric power system in the embodiment of the present application is a combination of software and / or hardware with predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0100] The following is an introduction to the power system optimization device provided in the embodiments of the present application.

[0101] Figure 5 is a structural block diagram of an optimization device for a power system according to an embodiment of the present application. Figure 5 As shown, the device comprises:

[0102] A first acquisition unit 10 is used to acquire offline related data of the power system, wherein the offline related data at least includes one or more of current, voltage, power, and phase difference;

[0103] An extraction unit 20, used to extract data features of the above offline related data using a Deep Q-network algorithm;

[0104] The optimization unit 30 is used to use a supervised learning algorithm to perform optimization control based on the above data features of the above offline related data.

[0105] Through this embodiment, the data obtained are all samples collected offline and do not interact with the real online power system, so safety can be guaranteed. The Deep Q-network algorithm is a deep learning and reinforcement learning algorithm that can learn the characteristics of samples and has efficient feature extraction capabilities. The supervised learning algorithm can predict future power demand and supply based on historical data and real-time input, thereby optimizing the operation of the power system, reducing the operating cost of the power system, improving the stability of the system, achieving efficient decision-making and control, and better adapting to the power system, and can better improve the applicability of reactive power optimization of the power system under various operating conditions.

[0106] In the specific implementation process, the extraction unit includes a first construction module and a first processing module. The first construction module is used to construct a feature recognition model, wherein the feature recognition model is obtained by training through a Deep Q-network algorithm using multiple sets of training data, and each set of training data in the multiple sets of training data includes historical offline related data obtained within a historical time period and historical data features corresponding to the historical offline related data; the first processing module is used to input the offline related data into the feature recognition model to obtain the data features corresponding to the offline related data.

[0107] In this solution, the Deep Q-network algorithm is used to train the model during the process of building a feature recognition model. By inputting offline related data into the feature recognition model, the features corresponding to these data can be obtained. By building a feature recognition model, the features of the data can be identified quickly and accurately, further improving the efficiency of data processing.

[0108] In some embodiments, the device further includes a first construction unit, a second construction unit, and a third construction unit, wherein the first construction unit is used to construct a first network layer before constructing a feature recognition model, wherein the first network layer is a Transformer layer, and the first network layer is used to extract state data features, wherein the state data features are electrical feature quantities during steady-state operation of the power system, and the state data features include one or more of node active power, reactive power injection, and node voltage amplitude; the second construction unit is used to construct a second network layer, wherein the second network layer is a first fully connected layer, and the second network layer is used to extract action data features, wherein the action data features are reactive compensation amounts of reactive control devices that have been feature-encoded; the third construction unit is used to construct a third network layer, wherein the third network layer is a second fully connected layer, and the third network layer is used to extract a first value of the state data features and a second value of the action data features.

[0109] In this solution, a neural network structure is provided, which includes two network layers: the first layer is the Transformer layer, which is used to extract the features of the state data; the second layer is the first fully connected layer, which is used to extract the features of the action data. The Transformer layer performs well in processing sequence data and can better capture the relationship between data. The fully connected layer can combine and fuse different features, which helps to improve the expressiveness and generalization ability of the model. Through such a deep network structure, the features of the state data and action data can be effectively extracted, which is conducive to the network learning more effective representations.

[0110] During the specific implementation process, the above-mentioned device also includes a second acquisition unit and a processing unit. The second acquisition unit is used to obtain initial action data before building the second network layer, wherein the above-mentioned initial action data is data with a change amount in the above-mentioned offline related data; the processing unit is used to pre-process the above-mentioned initial action data using a one-Hot encoding method to obtain action data.

[0111] In this scheme, in knowledge transfer, as the number of samples is limited, the accuracy of the estimation results will decrease, and the input actions are preprocessed using one-hot encoding. This method reduces the correlation between different actions and weakens the impact of the back propagation of a single sample on other evaluation results during training.

[0112] In some embodiments, the device further includes a fourth construction unit and a fifth construction unit, wherein the fourth construction unit is used to construct a main network layer before constructing the feature recognition model, wherein the main network layer is constructed based on the Deep Q-network algorithm, and the main network layer is used to calculate the expected return value corresponding to the input state data and the action data, and in each step of training, the parameters of the main network layer are updated according to the samples in the experience playback; the fifth construction unit is used to construct a target network layer, wherein the target network layer is constructed based on the Deep Q-network algorithm, and the target network layer is used to calculate the state-action value of the target.

[0113] In this scheme, the training process can be stabilized through the network layer constructed above. The parameters of the target network layer are not updated at every step, but the parameters of the main network layer are copied to the target network at a fixed number of steps. Since the parameters of the target network remain unchanged for a period of time, it provides a relatively stable target value and avoids the instability of the training process.

[0114] In the specific implementation process, the optimization unit also includes a second construction module, a second processing module and an optimization module. The second construction module is used to construct a gain identification model, wherein the gain identification model is obtained by training with a supervised learning algorithm using multiple sets of training data, and each set of training data in the multiple sets of training data includes historical data features acquired in a historical time period and historical gains corresponding to the historical data features; the second processing module is used to input the data features into the gain identification model to obtain the gains corresponding to the data features; the optimization module is used to use the gains and reward functions to optimize the control of the power system.

[0115] In this scheme, a gain identification model is constructed using a supervised learning algorithm, and the model is applied to power system optimization control. This can more accurately identify and predict the gain of the power system, thereby optimizing the control strategy and improving the efficiency and performance of the system.

[0116] In some embodiments, the above reward function is:

[0117]

[0118] r represents the above reward function, P loss represents the total active power loss of the above power system, ∑Δ v 2 represents the overall voltage deviation of the system bus, ∑| qinput | represents the total injected reactive power, α represents the first weight coefficient, β represents the second weight coefficient, and λ represents the third weight coefficient.

[0119] In this scheme, the reward function is intended to indicate the effectiveness of the control action. Specifically for reactive power optimization problems, the reward should take into account the impact of power loss, voltage deviation, and rescheduling cost. The reward function can motivate the optimization algorithm to continuously optimize and eventually converge to the optimal solution, so as to flexibly control the search direction and speed of the optimization algorithm to make it more in line with actual needs.

[0120] The above-mentioned power system optimization device includes a processor and a memory, and the above-mentioned first acquisition unit, extraction unit and optimization unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0121] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem of poor applicability of the conventional reactive power optimization method in the prior art in a complex actual power system can be solved by adjusting kernel parameters.

[0122] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0123] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the power system optimization method.

[0124] Specifically, the optimization methods of the power system include:

[0125] Step S201, acquiring offline related data of the power system, wherein the offline related data at least includes one or more of current, voltage, power, and phase difference;

[0126] Step S202, using the Deep Q-network algorithm to extract data features of the above offline related data;

[0127] Step S203: using a supervised learning algorithm to perform optimization control based on the above data features of the above offline related data.

[0128] An embodiment of the present invention provides a processor, and the processor is used to run a program, wherein the power system optimization method is executed when the program is running.

[0129] Specifically, the optimization methods of the power system include:

[0130] Step S201, acquiring offline related data of the power system, wherein the offline related data at least includes one or more of current, voltage, power, and phase difference;

[0131] Step S202, using the Deep Q-network algorithm to extract data features of the above offline related data;

[0132] Step S203: using a supervised learning algorithm to perform optimization control based on the above data features of the above offline related data.

[0133] An embodiment of the present invention provides a device, the device including a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0134] Specifically, the optimization methods of the power system include:

[0135] Step S201, acquiring offline related data of the power system, wherein the offline related data at least includes one or more of current, voltage, power, and phase difference;

[0136] Step S202, using the Deep Q-network algorithm to extract data features of the above offline related data;

[0137] Step S203: using a supervised learning algorithm to perform optimization control based on the above data features of the above offline related data.

[0138] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0139] The present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in each embodiment of the present application are implemented.

[0140] Specifically, the optimization methods of the power system include:

[0141] Step S201, acquiring offline related data of the power system, wherein the offline related data at least includes one or more of current, voltage, power, and phase difference;

[0142] Step S202, using the Deep Q-network algorithm to extract data features of the above offline related data;

[0143] Step S203: using a supervised learning algorithm to perform optimization control based on the above data features of the above offline related data.

[0144] The present application also provides an optimization system, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the above-mentioned power system optimization methods.

[0145] Specifically, the optimization methods of the power system include:

[0146] Step S201, acquiring offline related data of the power system, wherein the offline related data at least includes one or more of current, voltage, power, and phase difference;

[0147] Step S202, using the Deep Q-network algorithm to extract data features of the above offline related data;

[0148] Step S203: using a supervised learning algorithm to perform optimization control based on the above data features of the above offline related data.

[0149] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0150] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0151] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0154] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0155] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0156] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0157] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0158] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0159] 1) In an optimization method for an electric power system of the present application, the data obtained are all samples collected offline and do not interact with the real online electric power system, so safety can be guaranteed. The Deep Q-network algorithm is a deep learning and reinforcement learning algorithm that can learn the characteristics of samples and has efficient feature extraction capabilities. The supervised learning algorithm can predict future electricity demand and supply based on historical data and real-time input, thereby optimizing the operation of the electric power system, reducing the operating cost of the electric power system, improving the stability of the system, achieving efficient decision-making and control, and better adapting to the electric power system, and can better improve the applicability of reactive power optimization of the electric power system under various operating conditions.

[0160] 2) In an optimization device for an electric power system of the present application, the data acquired are all samples collected offline and do not interact with the real online electric power system, so safety can be guaranteed. The Deep Q-network algorithm is a deep learning and reinforcement learning algorithm that can learn the characteristics of samples and has efficient feature extraction capabilities. The supervised learning algorithm can predict future electricity demand and supply based on historical data and real-time input, thereby optimizing the operation of the electric power system, reducing the operating cost of the electric power system, improving the stability of the system, achieving efficient decision-making and control, and better adapting to the electric power system, and can better improve the applicability of reactive power optimization of the electric power system under various operating conditions.

[0161] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for optimizing a power system, characterized in that: include: Acquire offline related data of the power system, wherein the offline related data includes at least one or more of current, voltage, power, and phase difference; Using a Deep Q-network algorithm to extract data features of the offline related data; A supervised learning algorithm is used to perform optimization control based on the data features of the offline related data.

2. The method according to claim 1, characterized in that: The Deep Q-network algorithm is used to extract the data features of the offline related data, including: Constructing a feature recognition model, wherein the feature recognition model is trained by a Deep Q-network algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes historical offline related data acquired within a historical time period and historical data features corresponding to the historical offline related data; The offline related data is input into the feature recognition model to obtain the data features corresponding to the offline related data.

3. The method according to claim 2, characterized in that Before constructing the feature recognition model, the method further includes: Constructing a first network layer, wherein the first network layer is a Transformer layer, and the first network layer is used to extract state data features, wherein the state data features are electrical characteristic quantities during steady-state operation of the power system, and the state data features include one or more of node active power, reactive power injection, and node voltage amplitude; Constructing a second network layer, wherein the second network layer is the first fully connected layer, and the second network layer is used to extract action data features, wherein the action data features are reactive power compensation amounts of reactive power control devices that have been feature-encoded; A third network layer is constructed, wherein the third network layer is a second fully connected layer, and the third network layer is used to extract a first value of the state data feature and a second value of the action data feature.

4. The method according to claim 3, characterized in that Before constructing the second network layer, the method further includes: Acquire initial action data, wherein the initial action data is data with a change amount in the offline related data; The initial action data is preprocessed using a one-hot encoding method to obtain action data.

5. The method according to claim 3, characterized in that: Before constructing the feature recognition model, the method further includes: Constructing a main network layer, wherein the main network layer is constructed based on the Deep Q-network algorithm, and the main network layer is used to calculate the expected return value corresponding to the input state data and the action data. In each training step, the parameters of the main network layer are updated according to the samples in the experience playback; Construct a target network layer, wherein the target network layer is constructed based on a Deep Q-network algorithm, and the target network layer is used to calculate a state-action value of a target.

6. The method according to claim 3, characterized in that Using a supervised learning algorithm to perform optimization control based on the data features of the offline related data, including: Constructing a gain identification model, wherein the gain identification model is trained by a supervised learning algorithm using multiple sets of training data, each set of training data in the multiple sets of training data includes historical data features acquired in a historical time period and historical gains corresponding to the historical data features; Inputting the data feature into the gain recognition model to obtain the gain corresponding to the data feature; The gain and reward function are used to optimize the control of the power system.

7. The method according to claim 6, characterized in that The reward function is: r represents the reward function, P loss represents the total active power loss of the power system, ΣΔv 2 represents the overall voltage deviation of the system bus, Σ|q input | represents the total reactive power injected, α represents the first weight coefficient, β represents the second weight coefficient, and λ represents the third weight coefficient.

8. An optimization device for a power system, characterized in that: include: A first acquisition unit, configured to acquire offline related data of the power system, wherein the offline related data includes at least one or more of current, voltage, power, and phase difference; An extraction unit, used to extract data features of the offline related data using a Deep Q-network algorithm; The optimization unit is used to adopt a supervised learning algorithm to perform optimization control based on the data features of the offline related data.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the power system optimization method according to any one of claims 1 to 7 are implemented.

10. An optimization system, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the optimization method of the power system described in any one of claims 1 to 7.

Images